What happened in the Reproducible Builds effort between April 17th and April 23rd 2016: Toolchain fixes Thomas Weber uploaded lcms2/2.7-1 which will not write uninitialized memory when writing color names. Original patch by Lunar. The GCC 7 development phase has just begun, so Dhole reworked his patch to make gcc use SOURCE_DATE_EPOCH if set which prompted interesting feedback, but it has not been merged yet. Alexis Bienven e submitted a patch for sphinx to strip Python object memory addresses from the generated documentation. Packages fixed The following packages have become reproducible due to changes in their build dependencies: cobertura, commons-pool, easymock, eclipselink, excalibur-logkit, gap-radiroot, gluegen2, jabref, java3d, jcifs, jline, jmock2, josql, jtharness, libfann, libgroboutils-java, libjemmy2-java, libjgoodies-binding-java, libjgrapht0.8-java, libjtds-java, liboptions-java, libpal-java, libzeus-jscl-java, node-transformers, octave-msh, octave-secs2d, openmama, rkward. The following packages have become reproducible after being fixed:

auto-multiple-choice/1.3.0-1 by Alexis Bienven e.
castle-game-engine/5.2.0-3 by Paul Gevers.
gap-autpgrp/1.5-2 by Bill Allombert.
gap-grape/4r7+ds-2 by Jerome Benoit.
letsencrypt.sh/0.1.0-2 by Mattia Rizzolo.
libindicate/0.6.92-3 by Adam Borowski.
ltrsift/1.0.2-7 by Sascha Steinbiss.
perl/5.22.1-10 by Niko Tyni.
php-pear/1:1.10.1+submodules+notgz-7 by Mathieu Parent.
python-pint/0.7.2-2 by Ond ej Nov .
spades/3.7.1+dfsg-3 by Sascha Steinbiss.
tex4ht/20090611-2 by Reiner Herrmann.

Patches submitted that have not made their way to the archive yet:

#821356 against emoslib by boyska: use echo in a portable manner across shells.
#822268 against transdecoder by Dhole: set PERL_HASH_SEED=0 when calling the scripts that generate samples.

tests.reproducible-builds.org

Steven Chamberlain investigated the performance of our armhf boards which also provided a nice overview of our armhf build network.
As i386 has almost been completely tested the order of the architectures displayed has been changed to reflect the fact that i386 is now the 2nd most popular architecture in Debian. (h01ger)
In order to decrease the number of blacklisted packages, the first build is now run with a timeout of 18h (previously: 12h) and the 2nd with 24h timeout (previously: 18h). (h01ger)
We now also vary the CPU model on amd64 (and soon on i386 too) so that one build is performed using a "AMD Opteron 62xx class CPU" while the other is done using a "Intel Core Processor (Haswell)". This is now possible as proftitbricks.com offers VMs running both types of CPU and have generously increased their sponsorship once more. (h01ger)
Profitbricks increased our storage space by 400 GB which will be used to setup a 2nd build node for the coreboot/OpenWrt/NetBSD/Arch Linux/Fedora tests. This 2nd build node will run 398 days in the future for testing reproducibility on a different date.

diffoscope development diffoscope 52 was released with changes from Mattia Rizzolo, h01ger, Satyam Zode and Reiner Herrmann, who also did the release. Notable changes included:

Drop transitional debbindiff package.
Let objdump demangle symbols for better readability.
Install bin/diffoscope instead of auto-generated script. (Closes: #821777)

As usual, diffoscope 52 is available on Debian, Archlinux and PyPI, other distributions will hopefully soon update. Package reviews 28 reviews have been added, 11 have been updated and 94 have been removed in this week. 14 FTBFS bugs were reported by Chris Lamb (one being was a duplicate of a bug filed by Sebastian Ramacher an hour earlier). Misc. This week's edition was written by Lunar, Holger 'h01ger' Levsen and Chris Lamb and reviewed by a bunch of Reproducible builds folks on IRC.

As I'm nearing the point where Nageru, my live video mixer, can produce a stream directly that is actually suitable to streaming directly to clients (without a transcoding layer in the chain), it struck me the other day how much of the chain I've actually had to touch: In my test setup, the signal comes into a Blackmagic Intensity Shuttle. At some point, I found what I believe is a bug in the card's firmware; I couldn't fix it, but a workaround was applied in the Linux kernel. (I also have some of their PCI cards, in which I haven't found any bugs, but I have found bugs in their drivers.) From there, it goes into bmusb, a driver I wrote myself. bmusb uses libusb-1.0 to drive the USB card from userspace but for performance and stability reasons, I patched libusb to use the new usbfs zerocopy support in the Linux kernel. (The patch is still pending review.) Said zerocopy support wasn't written by me, but I did the work to clean up the support and push it upstream (it's in the 4.6-rc* series). Once safely through bmusb, it goes of course into Nageru, which I wrote myself. Nageru uses Movit for pixel processing, which I also wrote myself. Movit in turn uses OpenGL; I've found bugs in all three major driver implementations, and fixed a Nageru-related one in Mesa (and in the process of debugging that, found bugs in apitrace, a most useful OpenGL debugger). Sound goes through zita-resampler to stretch it ever so gently (in case audio and video clocks are out of sync), which I didn't wrote, but patched to get SSE support (patch pending upstream). So now Nageru chews a bit on it, and then encodes the video using x264 (that's the new part in 1.3.0 of course, you need a fast CPU to do that as opposed to using Quick Sync). I didn't write x264, but I had to redo parts of the speedcontrol patch (not part of upstream; awaiting review semi-upstream) because of bugs and outdated timings, but I also found a bug in x264 proper (fixed by upstream, pending inclusion). Muxing is done through ffmpeg, where I actually found multiple bugs in the muxer (some of which are still pending fixes). Once the stream is safely encoded and hopefully reasonably standards-conforming (that took me quite a while), it goes to Cubemap, which I wrote, for reflection to clients. For low-bitrate clients, it takes a detour through VLC to get a re-encode on a faster machine to lower bitrate I've found multiple bugs in VLC's streaming support in the past (and also fixed some of them, plus written the code that interacts with Cubemap). From there it goes to any of several clients, usually a browser. I didn't write any browsers (thank goodness!), but I wrote the client-side JavaScript that picks the closest relay, and the code for sending it to a Chromecast. I also found a bug in Chrome for Android (will be fixed in version 50 or 51, although the fix was just about turning on something that was already in the works), and one in Firefox for Linux (fixed by patching GStreamer's MP4 demuxer, although they've since switched away from that to something less crappy). IE/Edge also broke at some point, but unfortunately I don't have a way to report bugs to Microsoft. There's also at least one VLC bug involved on the client side (it starts decoding frames too late if they come with certain irregular timestamps, which causes them to drop), but I want to verify that they still persist even after the muxer is fixed before I go deep on that. Moral of the story: If anyone wants to write a multimedia application and says I'll just use <framework, language or library XYZ>, and I'll get everything for free; I just need to click things together! , they simply don't know what they're talking about and are in for a rude awakening. Multimedia is hard, an amazing amount of things can go wrong, complex systems have subtle bugs, and there is no silver bullet.

Two weeks ago was the GNOME Software hackfest in London, and I ve been there! And I just now found the time to blog about it, but better do it late than never

. Arriving in London and finding the Red Hat offices After being stuck in trains for the weekend, but fortunately arriving at the airport in time, I finally made it to London with quite some delay due to the slow bus transfer from Stansted Airport. After finding the hotel, the next issue was to get food and a place which accepted my credit card, which was surprisingly hard in defence of London I must say though, that it was a Sunday, 7 p.m. and my card is somewhat special (in Canada, it managed to crash some card readers, so they needed a hard-reset). While searching for food, I also found the Red Hat offices where the hackfest was starting the next day by accident. My hotel, the office and the tower bridge were really close, which was awesome! I have been to London in 2008 the last time, and only for a day, so being that close to the city center was great. The hackfest didn t leave any time to visit the city much, but by being close to the center, one could hardly avoid the London experience

. Cool people working on great stuff

That s basically the summary for the hackfest

. It was awesome to meet with Richard Hughes again, since we haven t seen each other in person since 2011, but work on lots of stuff together. This was especially important, since we managed to solve quite some disagreements we had over stuff Richard even almost managed to make me give in to adding <kudos/> to the AppStream spec, something which I was pretty against supporting (it didn t make it yet, but I am no longer against the idea of having that the remaining issues are solvable). Meeting Iain Lane again (after FOSDEM) was also very nice, and also seeing other people I ve only worked with over IRC or bug reports (e.g. William, Kalev, ) was great. Also lots of new people were there, like guys from Endless, who build their low-budget computer for developing/emerging countries on top of GNOME and Linux technologies. It s pretty cool stuff they do, you should check out their website! (they also build their distribution on top of Debian, which is even more awesome, and something I didn t know before (because many Endless people I met before were associated with GNOME or Fedora, I kind of implicitly assumed the system was based on Fedora

)). The incarnation of GNOME Software used by endless looks pretty different from what the normal GNOME user sees, since it s adjusted for a different audience and input method. But it looks great, and is a good example for how versatile GS already is! And for upstream GNOME, we ve seen some pretty great mockups done by Endless too I hope those will make it into production somehow.

Ironically, a snapstore was close to the office ;-)

XdgApp and sandboxing of apps was also a big topic, aside from Ubuntu and Endless integration. Fortunately, Alexander Larsson was also there to answer all the sandboxing and XdgApp-questions. I used the time to follow up on a conversation with Alexander we started at FOSDEM this year, about the Limba vs. XdgApp bundling issue. While we are in-line on the sandboxing approach, the way how software is distributed is implemented differently in Limba and XdgApp, and it is bad to have too many bundling systems around (doesn t make for a good story where we can just tell developers ship as this bundling format, and it will be supported everywhere ). Talking with Alex about this was very nice, and I think there is a way out of the too-many-solutions dilemma, at least for Limba and XdgApp I will blog about that separately soon. On the Ubuntu side, a lot of bugs and issues were squashed and changes upstreamed to GNOME, and people were generally doing their best to reduce Richard s bus-factor on the project a little

. I mainly worked on AppStream issues, finishing up the last pieces of appstream-generator and running it against some sample package sets (and later that week against the whole Debian archive). I also started to implement support for showing AppStream issues in the Debian PTS (this work is not finished yet). I also managed to solve a few bugs in the old DEP-11 generator and prepare another release for Ubuntu. We also enjoyed some good Japanese food, and some incredibly great, but also suddenly very expensive Indian food (but that s a different story

). The most important thing for me though was to get together with people actually using AppStream metadata in software centers and also more specialized places. This yielded some useful findings, e.g. that localized screenshots are not something weird, but actually a wanted feature of Endless for their curated AppStore. So localized screenshots will be part of the next AppStream spec. Also, there seems to be a general need to ship curation information for software centers somehow (which apps are featured? how are they styled? added special banners for some featured apps, app of the day features, etc.). This problem hasn t been solved, since it s highly implementation-specific, and AppStream should be distro-agnostic. But it is something we might be able to address in a generic way sooner or later (I need to talk to people at KDE and Elementary about it). In summary It was a great event! Going to conferences and hackfests always makes me feel like it moves projects leaps ahead, even if you do little coding. Sorting out issues together with people you see in person (rather than communicating with them via text messages or video chat), is IMHO always the most productive way to move forward (yeah, unless you do this every week, but I think you get my point

). For me, being the only (and youngest ^^) developer at the hackfest who was not employed by any company in the FLOSS business, the hackfest was also motivating to continue to invest spare time into working on these projects. So, the only thing left to do is a huge shout out of THANK YOU to the Ubuntu Community Fund and therefore the Ubuntu community for sponsoring me! You rock! Also huge thanks to Canonical for organizing the sponsoring really quickly, so I didn t get into trouble with paying my flights.

Laney and attente walking on the Millennium Bridge after we walked the distance between Red Hat and Canonical's offices.

Laney and attente on the Millennium Bridge after we walked the distance between Red Hat and Canonical s offices.

To worried KDE people: No, I didn t leave the blue side I just generally work on cross-desktop stuff, and would like all desktops to work as well as possible

While there is real progress being made towards eliminating insecure HTTP traffic, DNS is a fundamental Internet service that still usually relies on unauthenticated cleartext. There are however a few efforts to try and fix this problem. Here is the setup I use on my Debian laptop to make use of both DNSSEC and DNSCrypt.

DNSCrypt DNSCrypt was created to enable end-users to encrypt the traffic between themselves and their chosen DNS resolver. To switch away from your ISP's default DNS resolver to a DNSCrypt resolver, simply install the dnscrypt-proxy package and then set it as the default resolver either in /etc/resolv.conf:

nameserver 127.0.2.1

if you are using a static network configuration or in /etc/dhcp/dhclient.conf:

supersede domain-name-servers 127.0.2.1;

if you rely on dynamic network configuration via DHCP. There are two things you might want to keep in mind when choosing your DNSCrypt resolver:

whether or not they keep any logs of the DNS traffic
whether or not they support DNSSEC

I have personally selected a resolver located in Iceland by setting the following in /etc/default/dnscrypt-proxy:

DNSCRYPT_PROXY_RESOLVER_NAME=ns0.dnscrypt.is

DNSSEC While DNSCrypt protects the confidentiality of our DNS queries, it doesn't give us any assurance that the results of such queries are the right ones. In order to authenticate results in that way and prevent DNS poisoning, a hierarchical cryptographic system was created: DNSSEC. In order to enable it, I have setup a local unbound DNSSEC resolver on my machine and pointed /etc/resolv.conf (or /etc/dhcp/dhclient.conf) to my unbound installation at 127.0.0.1. Then I put the following in /etc/unbound/unbound.conf.d/dnscrypt.conf:

server:
    # Remove localhost from the donotquery list
    do-not-query-localhost: no
forward-zone:
    name: "."
    forward-addr: 127.0.2.1@53

to stop unbound from resolving DNS directly and to instead go through the encrypted DNSCrypt proxy.

Reliability In my experience, unbound and dnscrypt-proxy are fairly reliable but they eventually get confused (presumably) by network changes and start returning errors. The ugly but dependable work-around I have found is to create a cronjob at /etc/cron.d/restart-dns.conf that restarts both services once a day:

0 3 * * *    root    /usr/sbin/service dnscrypt-proxy restart
1 3 * * *    root    /usr/sbin/service unbound restart

Captive portals The one remaining problem I need to solve has to do with captive portals. This can be quite annoying when travelling because it requires me to use the portal's DNS resolver in order to connect to the splash screen that unlocks the wifi connection. The dnssec-trigger package looked promising but when I tried it on my jessie laptop, it wasn't particularly reliable. My temporary work-around is to comment out this line in /etc/dhcp/dhclient.conf whenever I need to connect to such annoying wifi networks:

#supersede domain-name-servers 127.0.0.1;

If you've found a better solution to this problem, please leave a comment!

Debian Long Term Support (LTS) This is my 5th month working on Debian LTS, started by Raphael Hertzog at Freexian. This is my largest month so far, in which I worked on completing the Xen and NSS packages updates from last month, but also spent a significant amount of time working on phpMyAdmin and libidn.

Updates to NSS and Xen completed This basically consisted on following up on the reviews from other security people. I basically continued building up on Brian's work and tested the package on a test server at Koumbit, which seems to have survived well the upgrade. The details are in this post to the debian-lts mailing list. As for NSS, the package was mostly complete, but I forgot to ship the tests for some reason, so I added them back. I also wrote the release team to see if it was possible to update NSS to the same version in all suites. Unfortunately, this issue is still pending, but I still hope we can find better ways of managing that package in the long term.

IDN and phpMyAdmin Most of my time this month was spent working on IDN and phpMyAdmin. Unfortunately, it turns out that someone else had worked on the libidn package. This is partly my fault: I forgot to check in the `dsa-needed.txt` file for assignment before working on the package. But considering how in flux the workflow currently is with the switch between the main security team and the LTS team for the wheezy maintenance, I don't feel too bad. Still, I prepared a package which was a bit more painful than it should have been because of GNUlib. I didn't even know about GNUlib before, oddly enough, and after that experience, I feel that it should just not exist at all anymore. I have filed a bug to remove that dependency at the very least, but I do not clearly see how such a tool is necessary on Debian at this point in time. But phpMyAdmin, no one had worked on that. And I understand why: even though it's a very popular package, there were quite a few outstanding issues (8!) in Wheezy, with 10-15 patches to be ported. Plus, it's ... well, PHP. And old PHP at that, with parts of it with modern autoloader classes, and other with mixed HTML and PHP content, `require` (and not `require_once`) and all sorts of nasty things you still find in the PHP universe. I nevertheless managed to produce a new Debian package for wheezy and test it on Koumbit's test servers. Hopefully, that can land into Wheezy soon.

Long term software support I am a little worried that we are, both in Jessie and Wheezy sitting in between stable releases for phpMyAdmin, something that is a recurring issue for a lot of packages in Debian stable or LTS. Sometimes, it just happens that the version that happens to be in Debian testing when it is released as stable is just not a supported release upstream. It's the case for phpMyAdmin in Jessie (4.3, whereas 4.0 and 4.4 are supported) and Wheezy (3.4, although it's unclear how long that was supported upstream). But even if the next Debian stable (Stretch), would pick a stable release upstream, there is actually no phpMyAdmin release that has a support window as long as Debian stable (roughly 3 years), let alone as long as Debian LTS (5 years). This is a similar problem with NSS: upstream is simply not supporting their product in the long term, or at least not in the terms we are used to in the Debian community (ie. only security fixes or regressions). This is, in my opinion, a real concern for the reliability and sustainability of the computing infrastructure we are creating. While some developers are of the opinion that software older than 18 months is too old, here we are shipping hardware and software in space or we have Solaris, which is supported for 16 years! Now that is a serious commitment and something we can rely on. 18 months is really, really, a tiny short time in the history of our civilization. I know computer programmers and engineers like to think of themselves in elitist terms, that they are changing the world every other year. But the truth is that things have not changed much in the last 4 decades where computing has existed, both in terms of security or functionality. Two quotes from my little quotes collection come to mind here:
Software gets slower faster than hardware gets faster. - Wirth's law The future is already here it's just not very evenly distributed. - William Gibson
Because of course, the response to my claims that computing is not really advancing is "but look! we have supercomputers in our pockets now!" Yes, of course, we do have those fancy phones and they are super-connected, but they are a threat to our fundamental rights and freedom. And those glittering advances always pale in comparison to what could be done if we wouldn't be wasting our time rewriting the same software over and over again on different, often proprietary, platforms, simply because of the megalomaniac fantasies of egoistic programmers. It would be great to build something that lasts, for a while. Software that does not need to be updated every 16 months. You'd think that something as basic as a screensaver release could survive what is basically the range of human infancy. Yet it seems we like to run mindlessly in the minefield of software development, one generation following the other without questioning the underlying assumption of infinite growth that permeates our dying civilization. I have talked about this before of course, but working with the LTS project just unnerves me so bad that I just needed to let out another rant. (For the record, I really have a lot of respect for JWZ and all the work he has done in the free software world. I frequently refer to his "no-bullshit" backup guide and use Xscreensaver daily. But I do think he was wrong in this case: asking Debian to remove Xscreensaver is just too much. The response from the maintainer was exemplary of how to handle such issues in the future. I restarted Xscreensaver after the stable update, and the message is gone, and things are still all fine. Thanks JWZ and Tormod for keeping things rolling.)

Other free software work With that in mind, I obviously didn't stop doing other computing work this month. In fact, I did a lot of work to try to generally fix the internet, that endless and not-quite-gratifying hobby so many of us are destroying our bodies to accomplish.

Build tools I have experimented a bit more with different build tools. I got worried because at some point cowbuilder got orphaned and I figured I could refresh the way I build packages for LTS. I looked into sbuild, but that ended up not bringing much improvements over my current cowbuilder setup (which I really need to document better). I was asked by the new maintainer to open a bug report to make such setups easier by guessing the basepath better, so we'll see how that goes. I did enjoy the simplicity of gitpkg and discovered cowpoke which made it about 2 times faster to build packages because I could use another server to build larger packages. I also found that gitpkg doesn't use `-n` by default when calling gzip, which makes it harder to reproduce tarballs when they are themselves built reproducibly, which is the case for Github tarballs (really nice of them). So I filed bug #820842 about that. It would be pretty awesome if such buildds would be available for Debian Developers to do their daily tasks. It could simply be a machine that would spin up a chroot with `cowbuilder` or it could even be a full, temporary VM although that would take way more resources than a simple VM with a cowbuilder setup. In the meantime, I should probably look at whalebuilder as an alternative to cowbuilder. It is a tool that supports building packages within a Docker chroot, which means that packages are built from a clean environment like pbuilder, and using COW optimisations but also without privileges or network access, which is a huge plus especially when you build untrusted packages.

Ikiwiki admonitions I have done some work to implement Moinmoin-like admonitions in Ikiwiki, something I am quite happy about since it's something I was really missing about Moinmoin. Admonitions bring a really nice way to outline certain blocks with varying severity levels and distinct styles. For example:
Admonitions are great!
This was done with a macro, but basically, since Markdown allows more or less arbitrary HTML, this can also be done with the `<div>` tag. I like that we don't have a new weird markup here. Yet, I still liked the sub-parser feature of MoinMoin, something that can be implemented in Ikiwiki, but it's a little more clunky. Normally, you'd probably do this with the inline macro and subpages, but it's certainly less intuitive that directly inlined content.

Ereader I got a new e-reader! I was hesitant between the Kobo Aura H20 and the Kobo Glo HD, which were the ones available at Bestbuy. So I bought both and figured I would return the other. That was a painful decision! In the end, both machines are pretty nice:

Aura H2O

Pros:

waterproof

larger screen (makes it easier to see web pages and better for the eyes)

naturally drawn to it

Cons:

heavier

larger (fits in outside pocket though)

port cover finicky

more expensive (180$) - prices may go down in future

Aura Glo HD

Pros

smaller (fits in inside pocket of both coats)

better resolution in theory (can't notice in practice)

cheaper (100$)

may be less common on the future (larger models more common? just a guess)

Cons

no SD card

smaller screen

power button in the middle

... but in the end, I ended up settling down on the Glo, mostly for the price. Heck, I saved around 100$, so for that amount, I could have gotten two machines so that if one breaks I would still have the other. I have ordered a cover for it on Deal Extreme about two weeks ago, and it still hasn't arrived. I suspect it's going to take a few more months to get there, by which point I may have changed e-reader again. Note that both e-readers needed an update to calibre, so I started working on a calibre backport (#818309) which I will complete soon. So anyways, I looked into better ways of transferring articles from the web to the e-reader, something which I do quite a bit to avoid spending too much time on the computer. Since the bookmark manager I use (Bookie) is pretty much dead, I started looking at other alternatives. And partly inspired by Framasoft's choice of Wallabag for their bookmarking service (Framabag), I started to look into that software, especially since my friend who currently runs the Bookie instance is thinking of switching to Wallabag as well. It seems the simplest way to browse articles remotely through a standard protocol is by implementing OPDS support in Wallabag. OPDS is a standard developed in part by the Internet Archive and it allows for browsing book collections and downloading them. Articles and bookmarks would then be presented as individual books that would be accessible from any OPDS-compatible device. Unfortunately, the Kobo e-readers don't support OPDS out of the box: you need to setup some OPDS-compatible reader like Koreader. And that I found nearly impossible to do: I was able to setup KSM (the "start menu", not to be confused with that KSM), but not Koreader in KSM. Besides, I do not want a separate OS running here on the tablet: I just want to start Koreader every once in a while. KSM just starts another system when you reboot the e-reader, something which is not really convenient on the Kobo. Basically, I just want to add Koreader as a tile in the home screen on the e-reader. I found the documentation on that topic to be sparse and hard to follow. It is often dispersed across multiple forum threads and involves uploading random binaries, often proprietary, to the e-reader. It had been a long time since I was asked to "register my software" frenetically, and I hadn't missed that one bit. So I decided to stay away from this until that solution and its documentation matures a bit.

Streaming re-established I have worked a bit on my home radio stream. I simply turned the Liquidsoap stream back online, and did a few tweaks to the documentation that I had built back then. All that experimenting led me to do two NMUs. One was for gmpc-plugins to fix a FTBFS (bug #807735) and to properly kill the shout streamer when completing the playback (bug #820908). The other was to fix the ezstream manpage (bug #573928), a patch that had been sitting there for 5 years! This was to try to find an easy way to stream random audio (say from a microphone) to the Icecast server, something which is surprisingly difficult, consider how basic that functionality is. I was surprised to see that Darkice just completely fails to start (bug #821040) and I had to fallback to the simplest ices2 software to stream the audio. I am still having issues with Liquidsoap: it's really unstable! As a server, I would expect it to keep running for days if not years. Unfortunately, there's always something that makes it crash. I had assertion failed (bug #821112) and I keep seeing it crash after 2-3 days fairly reliably, a bug I reported 3 years ago and that is still not fixed (bug #727307). Switching back the stream to Vorbis (because I was having problems with the commandline mp3 players and `ogg123` is much more lightweight) created another set of problems too, this time with the phone. It seems that Android cannot stream Vorbis at all, something that is even worse in Cyanogenmod... I also had to tweak my MPD config to make the Android client be able to load the larger playlists (see dmix buffer is full).

Android apps So I have also done quite a bit of work again on my phone. I finally was told how to access from Termux from adb shell which is pretty cool because now I can start a screen on my phone and then, when I'm tired of tapping to type, I can just reconnect to it when I plug in a USB cable on my laptop. I sent a pull request to fix the documentation regarding that. I also tried to see how my SMS and Signal situation could be improved. Right now, I have two different apps to do SMS on my phone: I use both Signal and the VoIP.ms SMS client, because I do not have a contract or SIM card in my phone. Both work well independently, but it's somewhat annoying to have to switch between the two. (In fact, I feel that Signal itself has an issue with how it depends on the network to send encrypted messages: I often have to "ping" people in clear text (and therefore in the other app) so that they connect to their data plan to fetch my "secure" signal messages...) Anyways, I figured it would be nice if at least there would be a SMS fallback in Signal that would allow me to send regular text messages from signal through Voip.MS. That was dismissed really fast. Moxie even closed the conversation completely, something I had never experienced before, and doesn't feel exactly pleasant. The Voip.MS side was of course not impressed and basically shrugged it off because Signal was not receptive. I also tried to clear up the Libresignal confusion: there are 3 different projects named "Libresignal", and I am not sure I figured out the right thing, even though the issue is now closed and there's a FAQ that is supposed to make all that clear. Nevertheless, I opened two distinct feature requests to try to steer the conversation into a more productive direction: GCM-less calls and GCM support. But I'm not sure neither will go anywhere. In the meantime, I am using the official signal client, which I downloaded using gplaycli and which I keep up to date with Aptoide. Even though the reliability of that site is definitely questionable, it seems that once you have a trusted version, it is safe to upgrade, regardless of the source because there is a trust path between you and the developer. I also filed a few issues with varying levels of success/response from the community:

Green address feature requests: pin change and pin confirmation

AntennaPod icons are reversed, but apparently, I'm the one that's backwards, go figure

Asked for NYC support in the excellent Transportr app, it seems that's not going to happen until I find the space to run the Android dev environment...

Random background radiation And then there's of course the monthly background noise of all the projects I happened to not only stumble on, but also file bugs or patches against:

neomutt credits improvements: Neomutt is an amazing effort to converge all the Mutt third party patches in a common tree

request to improve acme comparison: acme is an interesting go client for Let's Encrypt

a small set of issues and patches for borg, mostly documentation:

borg diff with files

mention jessie-backports in install docs

mention openbsd and OpenSUSE in install docs

finally filed a bug report in debian about symlink support in ncdu

internationalization SNAFUs:

gitweb should default to utf8 instead of latin1

Apache's AddCharset config should state more clearly which documentation to read

classic "my printer is not working" never ending story: printing fails by default with "cannot open printer description file"

small fixes to the tuptime project, which i mentionned before

last nit-picking: znc homepage errors

Explaining G del s theorems to students is a pain. Period. How can those poor creatures crank their mind around a Completeness and an Incompleteness Proof I understand that. But then, there are brave souls using G del s theorems to explain the world of demons to writers, in particular to answer the question:

You can control a Demon by knowing its True Name, but why?

Very impressive. Found at worldbuilding.stackexchange.com, pointed to me by a good friend. I dare to full quote author Cort Ammon (nothing more is known), to preserve this masterpiece. Thanks!!!!

Use of their name forces them to be aware of the one truth they can never know. Tl/Dr: If demons seek permanent power but trust no one, they put themselves in a strange position where mathematical truisms paint them into a corner which leaves their soul small and frail holding all the strings. Use of their name suggests you might know how to tug at those strings and unravel them wholesale, from the inside out! Being a demon is tough work. If you think facing down a 4000lb Glabrezu without their name is difficult, try keeping that much muscle in shape in the gym! Never mind how many manicurists you go through keeping the claws in shape! I don t know how creative such demons truly are, but the easy route towards the perfect French tip that can withstand the rigors of going to the gym and benching ten thousand pounds is magic. Such a demon might learn a manicure spell from the nearby resident succubi. However, such spells are often temporary. No demon worth their salt is going to admit in front of a hero that they need a moment to refresh their mani before they can fight. The hero would just laugh at them. No, if a demon is going to do something, they re going to do it right, and permanently. Not just nice french tips with a clear lacquer over the top, but razor sharp claws that resharpen themselves if they are blunted and can extend or retract at will! In fact, come to think of it, why even go to the gym to maintain one s physique? Why not just cast a magic spell which permanently makes you into the glorious Hanz (or Franz) that the trainer keeps telling you is inside you, just waiting to break free. Just get the spell right once, and think of the savings you could have on gym memberships. Demons that wish to become more powerful, permanently, must be careful. If fairy tales have anything to teach is, it s that one of the most dangerous things you can do is wish for something forever, and have it granted. Forever is a very long time, and every spell has its price. The demon is going to have to make sure the price is not greater than the perks. It would be a real waste to have a manicure spell create the perfect claws, only to find that they come with a peculiar perchance to curve towards one s own heart in an attempt to free themselves from the demon that cast them. So we need proofs. We need proofs that each spell is a good idea, before we cast it. Then, once we cast it, we need proof that the spell actually worked intended. Otherwise, who knows if the next spell will layer on top perfectly or not. Mathematics to the rescue! The world of First Order Logic (FOL, or herefter simply logic ) is designed to offer these guarantees. With a few strokes of a pen, pencil, or even brush, it can write down a set of symbols which prove, without a shadow of a doubt, that not only will the spell work as intended, but that the side effects are manageable. How? So long as the demon can prove that they can cast a negation spell to undo their previous spell, the permanency can be reverted by the demon. With a few more fancy symbols, the demon can also prove that nobody else outside of the demon can undo their permanency. It s a simple thing for mathematics really. Mathematics has an amazing spell called reductio ad infinitum which does unbelievable things. However, there is a catch. There is always a catch with magic, even when that magic is being done through mathematics. In 1931, Kurt G del published his Incompleteness Theorems. These are 3 fascinating works of mathematical art which invoke the true names of First Order Logic and Set Theory. G del was able to prove that any system which is powerful enough to prove out all of algebra (1 + 1 = 2, 2 + 1 = 3, 3 * 5 = 15, etc.), could not prove its own validity. The self referential nature of proving itself crossed a line that First Order Logic simply could not return from. He proved that any system which tries must pick up one of these five traits:

Incomplete they missed a detail when trying to prove everything
Incorrect They got everything, but at least one point is wrong
Unprovable They might be right, but they can never prove it
Intractable If you re willing to sit down and write down a proof that takes longer than eternity, you can prove a lot. Proofs that fit into eternity have limits.
Illogical Throw logic to the wind, and you can prove anything!

If the demon wants itself to be able to cancel the spell, his proof is going to have to include his own abilities, creating just the kind of self referential effects needed to invoke G del s incompleteness theorems. After a few thousand years, the demon may realize that this is folly. A fascinating solution the demon might choose is to explore the incomplete solution to G del s challenge. What if the demon permits the spell to change itself slightly, but in an unpredictable way. If the demon was a harddrive, perhaps he lets a single byte get changed by the spell in a way he cannot expect. This is actually enough to sidestep G del s work, by introducing incompleteness. However, now we have to deal with pesky laws of physic and magics. We can t just create something out of nothing, so if we re going to let the spell change a single byte of us, there must be a single byte of information, its dual, that is unleashed into the world. Trying to break such conservation laws opens up a whole can of worms. Better to let that little bit go free into the world. Well, almost. If you repeat this process a whole bunch of times, layering spells like a Matryoska doll, you re eventually left with a soul that is nothing but the leftover bits of your spells that you simply don t know enough about to use. If someone were collecting those bits and pieces, they might have the undoing of your entire self. You can t prove it, of course, but its possible that those pieces that you sent out into the world have the keys to undo your many layers of armor, and then you know they are the bits that can nullify your soul if they get there. So what do you do? You hide them. You cast your spells only on the darkest of nights, deep in a cave where no one can see you. If you need assistants, you make sure to ritualistically slaughter them all, lest one of them know your secret and whisper it to a bundle of reeds, The king has horns, if you are familiar with the old fairy tale. Make it as hard as possible for the secret to escape, and hope that it withers away to nothingness before someone discovers it, leaving you invincible. Now we come back to the name. The demon is going to have a name it uses to describe its whole self, including all of the layers of spellcraft it has acquired. This will be a great name like Abraxis, the Unbegotten Father or Satan, lord of the underworld. However, they also need to keep track of their smaller self, their soul. Failure to keep track of this might leave them open to an attack if they had missed a detail when casting their spells, and someone uncovered something to destroy them. This would be their true name, potentially something less pompous, like Gaylord Focker or Slartybartfarst. They would never use this name in company. Why draw attention to the only part of them that has the potential to be weak. So when the hero calls out for Slartybartfarst, the demon truly must pay attention. If they know the name the demon has given over the remains of their tattered soul, might they know how to undo the demon entirely? Fear would grip their inner self, like a child, having to once again consider that they might be mortal. Surely they would wish to destroy the hero that spoke the name, but any attempt runs the risk of falling into a trap and exposing a weakness (surely their mind is racing, trying to enumerate all possible weaknesses they have). It is surely better for them to play along with you, once you use their true name, until they understand you well enough to confidently destroy you without destroying themselves. So you ask for answers which are plausible. This one needs no magic at all. None of the rules are invalid in our world today. Granted finding a spell of perfect manicures might be difficult (believe me, some women have spent their whole life searching), but the rules are simply those of math. We can see this math in non-demonic parts of society as well. Consider encryption. An AES-256 key is so hard to brute force that it is currently believed it is impossible to break it without consuming 3/4 of the energy in the Milky Way Galaxy (no joke!). However, know the key, and decryption is easy. Worse, early implementations of AES took shortcuts. They actually left the signature of the path they took through the encryption in their accesses to memory. The caches on the CPU were like the reeds from the old fable. Merely observing how long it took to read data was sufficient to gather those reeds, make a flute, and play a song that unveils the encryption key (which is clearly either The king has horns or 1-2-3-4-5 depending on how secure you think your luggage combination is). Observing the true inner self of the AES encryption implementations was enough to completely dismantle them. Of course, not every implementation fell victim to this. You had to know the name of the implementation to determine which vulnerabilities it had, and how to strike at them. Or, more literally, consider the work of Alfred Whitehead, Principia Mathematica. Principia Mathematica was to be a proof that you could prove all of the truths in arithmetic using purely procedural means. In Principia Mathematica, there was no manipulation based on semantics, everything he did was based on syntax manipulating the actual symbols on the paper. G del s Incompleteness Theorem caught Principia Mathematica by the tail, proving that its own rules were sufficient to demonstrate that it could never accomplish its goals. Principia Mathematica went down as the greatest Tower of Babel of modern mathematical history. Whitehead is no longer remembered for his mathematical work. He actually left the field of mathematics shortly afterwards, and became a philosopher and peace advocate, making a new name for himself there. (by Cort Ammon)

We're excited to announce that Debian has selected 29 interns to work with us this summer: 4 in Outreachy, and 25 in the Google Summer of Code. Here is the list of projects and the interns who will work on them: Android SDK tools in Debian:

APT - dpkg communications rework:

David Kalnischkies

Continuous Integration for Debian-Med packages:

Extending the Debian Developer Horizon:

Harsh Daftary

Improving and extending AppRecommender:

Improving the debsources frontend:

Aaron Delaney

Improving voice, video and chat communication with Free Software:

MIPS and MIPSEL ports improvements:

Jonathan Jackson

Reproducible Builds for Debian and Free Software:

Support for KLEE in Debile:

Marko Dimja evi

The Google Summer of Code and Outreachy programs are possible in Debian thanks to the effort of Debian developers and contributors that dedicate part of their free time to mentor students and outreach tasks. Join us and help extend Debian! You can follow the students weekly reports on the debian-outreach mailing-list, chat with us on our IRC channel or on each project's team mailing lists. Congratulations to all of them!

Ubuntu 16.04 was released today, with one of the highlights being the new Snap package format. Snaps are intended to make it easier to distribute applications for Ubuntu - they include their dependencies rather than relying on the archive, they can be updated on a schedule that's separate from the distribution itself and they're confined by a strong security policy that makes it impossible for an app to steal your data.

At least, that's what Canonical assert. It's true in a sense - if you're using Snap packages on Mir (ie, Ubuntu mobile) then there's a genuine improvement in security. But if you're using X11 (ie, Ubuntu desktop) it's horribly, awfully misleading. Any Snap package you install is completely capable of copying all your private data to wherever it wants with very little difficulty.

The problem here is the X11 windowing system. X has no real concept of different levels of application trust. Any application can register to receive keystrokes from any other application. Any application can inject fake key events into the input stream. An application that is otherwise confined by strong security policies can simply type into another window. An application that has no access to any of your private data can wait until your session is idle, open an unconfined terminal and then use curl to send your data to a remote site. As long as Ubuntu desktop still uses X11, the Snap format provides you with very little meaningful security. Mir and Wayland both fix this, which is why Wayland is a prerequisite for the sandboxed xdg-app design.

I've produced a quick proof of concept of this. Grab XEvilTeddy from git, install Snapcraft (it's in 16.04), snapcraft snap, sudo snap install xevilteddy*.snap, /snap/bin/xevilteddy.xteddy . An adorable teddy bear! How cute. Now open Firefox and start typing, then check back in your terminal window. Oh no! All my secrets. Open another terminal window and give it focus. Oh no! An injected command that could instead have been a curl session that uploaded your private SSH keys to somewhere that's not going to respect your privacy.

The Snap format provides a lot of underlying technology that is a great step towards being able to protect systems against untrustworthy third-party applications, and once Ubuntu shifts to using Mir by default it'll be much better than the status quo. But right now the protections it provides are easily circumvented, and it's disingenuous to claim that it currently gives desktop users any real security.

comments

Back in 2014, I wrote about some negative first impressions of systemd. I also had a plea to debian-project to end all the flaming, pointing out that jessie will still boot , noting that my preference was for sysvinit but things are what they are and it wasn t that big of a deal. Although I still have serious misgivings about the systemd upstream s attitude, I ve got to say I find the system rather refreshing and useful in practice. Here s an example. I was debugging the boot on a server recently. It mounts a bunch of NFS filesystems and runs a third-party daemon that is started from an old-style /etc/init.d script. We had a situation where the NFS filesystems the daemon required didn t mount on boot. The daemon then was started, and unfortunately it basically does a mkdir -p on startup. So it started running and processing requests with negative results. So there were two questions: why did the NFS filesystems fail to start, and how could we make sure the daemon wouldn t start without them mounted? For the first, journalctl -xb was immensely helpful. It logged the status of each individual mount, and it turned out that it looked like a modprobe or kernel race condition when a bunch of NFS mounts were kicked off in parallel and all tried to load the nfsv4 module at the same time. That was easy enough to work around by adding nfsv4 to /etc/modules. Now for the other question: refusing to start the daemon if the filesystems weren t there. With systemd, this was actually trivial. I created /etc/systemd/system/mydaemon.service.requires (I ll call the service mydaemon here), and in it I created a symlink to /lib/systemd/system/remote-fs.target. Then systemctl daemon-reload, and boom, done. systemctl list-dependencies mydaemon will even show the the dependency tree, color-coded status of each item on it, and will actually show every single filesystem that remote-fs requires and the status of it in one command. Super handy. In a non-systemd environment, I d probably be modifying the init script and doing a bunch of manual scripting to check the filesystems. Here, one symlink and one command did it, and I get tools to inspect the status of the mydaemon prerequisites for free. I ve got to say, as someone that has occasionally had to troubleshoot boot ordering and update-rc.d symlink hell, troubleshooting this stuff in systemd is considerably easier and the toolset is more powerful. Yes, it has its set of poorly-documented complexity, but then so did sysvinit. I never thought the world is falling folks were right, but by now I can be counted among those that feels like systemd has matured to the point where it truly is superior to sysvinit. Yes, in 2014 it had some bugs, but by here in 2016 it looks pretty darn good and I feel like Debian s decision has been validated through my actual experience with it.

alberoVita The accelerated world of the Western or Westernized countries seems to be fed by an insidious food, which generates a kind of psychological dependence: anxiety. The economy of global markets cannot help it, it has a structural need of it to feed their iron logic of survival. The anxiety generated in the masses of consumers and in market competitors is crucial for Companies fighting each other and now they can only live if men are projected to objective targets continuously moving forward, without ever allowing them to achieve a stable destination.

The consumer is thus constantly maintained in a state of perpetual breathlessness, always looking for the fresh air of liberation that could eventually reduce his tension. It is a state of anxiety caused by false needs generated by advertising campaigns whose primary purpose is to create a need, to interpret to their advantage a still confused psychological demand leading to the destination decided by the market <Read More [by Fabio Marzocca]>

Last week, someone posted a request for help on the popular Server Fault Q&A site: they had apparently accidentally deleted their entire web hosting business, and all their backups. The post (now itself deleted) was a reasonably obvious fake, but mainstream media reported on it anyway, and then life imitated art and 123-reg went and did actually delete all their hosted VMs, and their backups. I was chatting to some friends from $job-2 and we had a brief smug moment that we had never done anything this bad, before moving on to incredulity that we had never done anything this bad in the 5 years or so we were running the University web servers. Some time later I realised that my personal backups were at risk from something like this because I have a permanently mounted /backup partition on my home NAS. I decided to fix it. I already use Systemd to manage mounting the /backup partition (via a backup.mount file) and its dependencies. I'll skip the finer details of that for now. I planned to define some new Systemd units for each backup job which was previously scheduled via Cron in order that I could mark them as depending on the /backup mount. I needed to adjust that mount definition by adding StopWhenUnneeded=true. This ensures that /backup will be unmounted when it is not in use by another job, and not at risk of a stray rm -rf. The backup jobs are all simple shell scripts that convert quite easily into services. An example: backup-home.service:

[Unit]
Requires=backup.mount
After=backup.mount
[Service]
User=backupuser
Group=backupuser
ExecStart=/home/backupuser/bin/phobos-backup-home

To schedule this, I also need to create a timer: backup-home.timer:

[Timer]
OnCalendar=*-*-* 04:01:00
[Install]
WantedBy=timers.target

To enable the timer, you have to both enable and start it:

systemctl enable backup-home.timer
systemctl start backup-home.timer

I created service and timer units for each of my cron jobs. The other big difference to driving these from Cron is that by default I won't get any emails if the jobs generate output - in particular, if they fail. I definitely do want mail if things fail. The Arch Wiki has an interesting proposed solution to this which I took a look at. It's a bit clunky, and my initial experiments with a derivation from this (using mail(1) not sendmail(1)) have not yet generated any mail. Pros and Cons The Systemd timespec is more intuitive than Cron's. It's a shame you need a minimum of three more lines of boilerplate for the simplest of timers. I think WantedBy=timers.target should probably be an implicit default for all .timer type units. Here I think clarity suffers in the name of consistency. With timers, start doesn't kick-off the job, it really means "enable" in the context of timers, which is clumsy considering the existing enable verb, which seems almost superfluous, but is necessary for consistency, ~~since Systemd units need to be enabled before they can be started~~ As Simon points out in the comments, this is not true. Rather, "enable" is needed for the timer to be active upon subsequent boots, but won't enable it in the current boot. "Start" will enable it for the current boot, but not for subsequent ones. Since I need a .service and a .unit file for each active line in my crontab, that's a lot of small files (twice as many as the number of jobs being defined) and they're all stored in system-wide folder because of the dependency on the necessarily system-level units defining the mount. It's easy to forget the After= line for the backup services. On the one hand, it's a shame that After= doesn't imply Require=, so you don't need both; or alternatively there was a convenience option that did both. On the other hand, there are already too many Systemd options and adding more conjoined ones would just make it even more complicated. It's a shame I couldn't use user-level units to achieve this, but they could not depend on the system-level ones, nor activate /backup. This is a sensible default, since you don't want any user to be able to start any service on-demand, but some way of enabling it for these situations would be good. I ruled out systemd.automount because a stray rm -rf would trigger the mount which defeats the whole exercise. Apparently this might be something you solve with Polkit, as the Arch Wiki explains, which looks like it has XML disease. I need to get mail-on-error working reliably.

What happened in the reproducible builds effort between April 10th and April 16th 2016: Toolchain fixes

Roland Rosenfeld uploaded transfig/1:3.2.5.e-6 which honors SOURCE_DATE_EPOCH. Original patch by Alexis Bienven e.
Bill Allombert uploaded gap/4r8p3-2 which makes convert.pl honor SOURCE_DATE_EPOCH. Original patch by Jerome Benoit, duplicate patch by Dhole.
Emmanuel Bourg uploaded ant/1.9.7-1 which makes the Javadoc task use UTF-8 as the default encoding if none was specified and SOURCE_DATE_EPOCH is set.

Antoine Beaupr suggested that gitpkg stops recording timestamps when creating upstream archives. Antoine Beaupr also pointed out that git-buildpackage diverges from the default gzip settings which is a problem for reproducibly recreating released tarballs which were made using the defaults. Alexis Bienven e submitted a patch extending sphinx SOURCE_DATE_EPOCH support to copyright year. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330, avis, brailleutils, charactermanaj, classycle, commons-io, commons-javaflow, commons-jci, gap-radiroot, jebl2, jetty, libcommons-el-java, libcommons-jxpath-java, libjackson-json-java, libjogl2-java, libmicroba-java, libproxool-java, libregexp-java, mobile-atlas-creator, octave-econometrics, octave-linear-algebra, octave-odepkg, octave-optiminterp, rapidsvn, remotetea, ruby-rinku, tachyon, xhtmlrenderer. The following packages became reproducible after getting fixed:

autopkgtest/3.20.3 uploaded by Martin Pitt, original patch by Alexis Bienven e.
fop/1:2.1-3 uploaded by Mathieu Malaterre, original patch by Alexis Bienven e.
fwupdate/0.5-4 by Mario Limonciello.
gem/1:0.93.3-10 by IOhannes m zm lnig.
imview/1.1.9c-16 by Anton Gladky.
libappindicator/0.4.92-4 by Adam Borowski.
libjlha-java/0.0.20050504-9 by Emmanuel Bourg.
lwjgl/2.9.3+dfsg-1 by Markus Koschany.
lxc/1:2.0.0~rc15-1 uploaded by Evgeni Golov, original patch by Reiner Herrmann.
manpages-de/1.12-1 uploaded by Tobias Quathamer, original patch by Reiner Herrmann.
pd-mrpeach/0.1~svn17615-1 by IOhannes m zm lnig.
pyhoca-gui/0.5.0.6-1 uploaded by Mike Gabriel, original patch by Chris Lamb.
r-cran-spatstat/1.45-0-1 by Andreas Tille.
s5/1.1.dfsg.2-6 uploaded by Peter Pentchev, original patch by Juan Picca.
samba/2:4.3.8+dfsg-1 by Jelmer Vernoo .
sim4/0.0.20121010-3 uploaded by Andreas Tille, original patch by Alexis Bienven e.
svxlink/15.11-1 uploaded by Felix Lechner, fixed upstream.
tinc/1.0.28-1 by Guus Sliepen, fixed upstream.
ucblogo/6.0+dfsg-1 by Reiner Herrmann.

Some uploads fixed some reproducibility issues, but not all of them:

mm-common/0.9.10-1 by Michael Biebl.

Patches submitted which have not made their way to the archive yet:

#820603 on viking by Alexis Bienven e: fix icon headers inclusion order.
#820661 on nullmailer by Alexis Bienven e: fix the order in which files are included in the static archive.
#820668 on sawfish by Alexis Bienven e: fix file ordering in theme archives, strip hostname and username from the config.h file, and honour SOURCE_DATE_EPOCH when creating the config.h file.
#820740 on bless by Alexis Bienven e: always use /bin/sh as shell.
#820742 on gmic by Alexis Bienven e: strip the build date from help messages.
#820809 on wsdl4j by Alexis Bienven e: use a plain text representation of the copyright character.
#820815 on freefem++ by Alexis Bienven e: fix the order in which files are included in the .edp files, and honour SOURCE_DATE_EPOCH when using the build date.
#820869 on pyexiv2 by Alexis Bienven e: honour the SOURCE_DATE_EPOCH environment variable through the ustrftime function, to get a reproducible copyright year.
#820932 on fim by Alexis Bienven e: fix the order in which files are joined in header files, strip the build date from fim binary, make the embeded vim2html script honour SOURCE_DATE_EPOCH variable when building the documentation, and force language to be English when using bison to make a grammar that is going to be parsed using English keywords.
#820990 on grib-api by Santiago Vila: always call dh-buildinfo.

diffoscope development Zbigniew J drzejewski-Szmek noted in #820631 that diffoscope doesn't work properly when a file contains several cpio archives. Package reviews 21 reviews have been added, 14 updated and 22 removed in this week. New issue found: timestamps_in_htm_by_gap. Chris Lamb reported 10 new FTBFS issues. Misc. The video and the slides from the talk "Reproducible builds ecosystem" at LibrePlanet 2016 have been published now. This week's edition was written by Lunar and Holger Levsen. h01ger automated the maintenance and publishing of this weekly newsletter via git.

Switching from one GnuPG master key to the usage of subkeys was long on my list of things I wanted to do, but never came around. With the advent of a YubiKey NEO in my pocket I finally took the plunge: reading through lots of web pages (and adding one here for confusion), trying to understand the procedures, and above all, understanding my own requirements!

To sum up a long story, it was worth the plunge, and all over the security level of my working environment has improved considerable. While the advantages of subkeys are well documented (e.g., Debian Wiki), at the end of the day I was like probably many Debian Developers having one master key that was used for every action: mail decryption and signing, signing of uploads, etc. Traveling a lot I always felt uncomfortable. Despite a lengthy passphrase, I still didn t want my master key to get into wrong hands in case the laptop got stolen. Furthermore, I had my master key on several computers (work, laptop, mail server), which didn t help a lot either. With all this, I started to compile a list of requirements/objectives I wanted to have:

master key is only available on offline medium (USB sticks)
subkeys for signing, encryption, authentication
possibility to sign and decrypt my emails on the server where I read emails (ssh/mutt)
laptop does not contain any keys, instead use Yubikey
all keys with expiry date (1y)
mixture of gpg versions: local laptop: gpg2.1, mail server: gpg1

Warning Before we start a word of caution make backups, best is to make backups at every stage. You don t want that an erroneous operations wipes out your precious keys without a backup! Preparation In the following I will assume that MASTERKEY environment variable contains the id of the master key to be converted. Furthermore, I have followed some of the advice here, so key ids will be shown in long format. Let us start with the current situation:

$ gpg -K $MASTERKEY
sec   4096R/0x6CACA448860CDC13 2010-09-14
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14

In the following we will go through the following steps:

Prepare the Yubikey NEO (forthcoming blog>
Edit to current key: add expiry, add photo, and above all add subkeys
Create revocation certificate
Create gpg2.1 structure
Backup to USB media
Move subkeys to Yubikey NEO
Remove master keys
Separate gpg1 (for mail server) and gpg2 (for laptop)
Upload to key servers

Yubikey SmartCard setup There are several guides out there, but I will in very near future write one about using the NEO for various usage scenaria including GPG keys. Edit the current key The following can be done in one session or in different sessions, the screen logs are after starting with:

$ gpg --expert --edit-key $MASTERKEY

add expiry date Having an expiry date on your key serves two purposes: If you loose it, it will solve itself automatically, and furthermore, you are forced to deal with the key and refresh your gpg knowledge at least once a year. That are two perfect reasons to set expiry to one year. The following log selects each key in turn and sets its expiry date.

$ gpg --expert --edit-key $MASTERKEY
gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: never       usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> expire
Changing expiration time for the primary key.
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:09:16 PM JST
Is this correct? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> key 1
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: never       usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
 
gpg> expire
Changing expiration time for a subkey.
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:09:27 PM JST
Is this correct? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>

Add a photo Not strictly necessary, but an interesting feature. gpg suggests 240 288, I resized a photo of my head, greyscaled it, and optimized it with jpegoptim -s -m40 my-photo.jpg. The parameter 40 is the quality, I played around a bit to find the best balance between size and quality. The size should not be too big as the photo will be part of the key!

gpg> addphoto
 
Pick an image to use for your photo ID.  The image must be a JPEG file.
Remember that the image is stored within your public key.  If you use a
very large picture, your key will become very large as well!
Keeping the image close to 240x288 is a good size to use.
 
Enter JPEG filename for photo ID: GPG/norbert-head.jpg
Is this photo correct (y/N/q)? y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]

Add subkeys of 2048bit for signing/encryption/authentication Now comes the interesting part, adding three subkeys: one for signing, one for encrypting, and one for authentication. The one for signing is the one you will use for signing your uploads to Debian as well as emails. The authentication key will later be used to provide ssh authentication. Note that you have to use the --expert expert option to edit-key (as shown above), otherwise gpg does not allow to do this. As I want to move the subkeys to the Yubikey NEO, a keysize of 2048bits is necessary. First for the signing:

gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:06 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
....+++++
..........+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]

Now the same for encryption key:

gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:20 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
..+++++
........+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]

Finally for the authentication key. Note that only here the --expert is necessary! We use (8) RSA (set your own capabilities) and then toggle sign and encryption capabilities off, and authentication on.

gpg> addkey
Key is protected.
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
Your selection? 8
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Sign Encrypt 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? s
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Encrypt 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? e
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? a
 
Possible actions for a RSA key: Sign Encrypt Authenticate 
Current allowed actions: Authenticate 
 
   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished
 
Your selection? q
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Mon 06 Feb 2017 08:10:34 PM JST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
......+++++
+++++
 
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub* 4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
sub  2048R/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ unknown] (5)  [jpeg image of size 4185]
 
gpg> save

Check the current status Good point to take a break and inspect the current status. We should have one main key and three subkeys, all with expiry dates of 1 year ahead, and a photo also attached to the key:

$ gpg --expert --edit-key $MASTERKEY
gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
gpg: checking the trustdb
gpg: public key 0x0FC3EC02FBBB8AB1 is 58138 seconds newer than the signature
gpg: 3 marginal(s) needed, 1 complete(s) needed, classic trust model
gpg: depth: 0  valid:   2  signed:  28  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: depth: 1  valid:  28  signed:  41  trust: 28-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2016-11-02
pub  4096R/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06  usage: SC  
                               trust: ultimate      validity: ultimate
sub  4096R/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06  usage: E   
sub  2048R/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06  usage: S   
sub  2048R/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06  usage: E   
sub  2048R/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg>

Create revocation certificate In case something happens, like all your backups are burned, your computers are destroyed, or all data stolen by the NSA, it is a good idea to have an old fashioned paper print out of a revocation certificate which allows you to revoke the key even if you are not in possession of it. This should be printed out and kept in a safe place.

$ gpg --gen-revoke $MASTERKEY > GPG/revoke-certificate-$MASTERKEY.txt
 
sec  4096R/0x6CACA448860CDC13 2010-09-14 Norbert Preining <norbert@preining.info>
 
Create a revocation certificate for this key? (y/N) y
Please select the reason for the revocation:
  0 = No reason specified
  1 = Key has been compromised
  2 = Key is superseded
  3 = Key is no longer used
  Q = Cancel
(Probably you want to select 1 here)
Your decision? 1
Enter an optional description; end it with an empty line:
> 
Reason for revocation: Key has been compromised
(No description given)
Is this okay? (y/N) y
 
You need a passphrase to unlock the secret key for
user: "Norbert Preining <norbert@preining.info>"
4096-bit RSA key, ID 0x6CACA448860CDC13, created 2010-09-14
 
Enter passphrase:
 
ASCII armored output forced.
Revocation certificate created.

Please move it to a medium which you can hide away; if the NSA or KGB or Mossad gets access to this certificate, they can use it to make your key unusable. It is smart to print this certificate and store it away, just in case your media become unreadable. Create gpg 2.1 structure There are currently three versions of gpg available: classic (version 1) which is one static binary, perfect for servers or scripting tasks; stable (version 2.0) which is the modularized version supporting OpenPGP, S/MIME, and Secure Shell; and finally modern (version 2.1 and up) with enhanced features like support for Elliptic Curve cryptography. Debian currently ships version 1 as standard, and also the modern version (but there are traces in experimental of a pending transition). The newer versions of GnuPG are modularized and use an agent. For the following we need to kill any running instance of gpg-agent.

$ killall gpg-agent

After that a simple call to gpg2 to list the secret keys will convert the layout to the new standard:

$ gpg2 -K $MASTERKEY
gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf
gpg: starting migration from earlier GnuPG versions
gpg: porting secret keys from '/home/norbert/.gnupg/secring.gpg' to gpg-agent
gpg: key 0xD2BF4AA309C5B094: secret key imported
gpg: key 0x6CACA448860CDC13: secret key imported
gpg: migration succeeded
sec   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb   rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb   rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb   rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]

After this there will be new files/directories in the .gnupg directory, in particular: .gnupg/private-keys-v1.d/ which contains the private keys. Creating backup Now your .gnupg directory contains still all the keys, available for gpg1 and gpg2.1. You MUST MAKE A BACKUP NOW!!! on at least 3 USB sticks and maybe some other offline media. Keep them in a safe place, better in different and safe places, you will need them for extending the expiry date, signing other keys, etc. Warning concerning USB and vfat file systems gpg >= 2.1 requires gpg-agent which in turn needs a socket. If you have the backup on an USB drive (most often with vfat file system), you need to redirect the socket, as vfat does not support sockets! Edit /USBSTICK/gnupghome/S.gpg-agent and enter there

%Assuan%
socket=/dev/shm/S.gpg-agent

After that the socket will be created in /dev/shm/ instead and invoking gpg with gpg2 --homedir /USBSTICK/gnupghome will work. You have done your backups, right? Move sub keys to card As I mentioned, I want to have no keys on my laptop which I carry around to strange countries, instead I want to have them all on a Yubikey NEO. I will describe the setup and usage in details soon, but mention here only how to move the keys to the card. This requires a finished setup including change of pins. Note that when using gpg2 to move the keys to the card, the local copies are actually deleted, but only for the gpg2(.1) files. The gpg1 secret keys are still all in place.

$ gpg2 --edit-key $MASTERKEY
gpg (GnuPG) 2.1.11; Copyright (C) 2016 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Secret key is available.
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb* rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (1) Signature key
   (3) Authentication key
Your selection? 1
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb* rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb* rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (2) Encryption key
Your selection? 2
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb* rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 4
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb* rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> keytocard
Please select where to store the key:
   (3) Authentication key
Your selection? 3
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb* rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> key 4
 
sec  rsa4096/0x6CACA448860CDC13
     created: 2010-09-14  expires: 2017-02-06  usage: SC  
     trust: ultimate      validity: ultimate
ssb  rsa4096/0xD1D2BD14810F62B3
     created: 2010-09-14  expires: 2017-02-06  usage: E   
ssb  rsa2048/0xEC00B8DAD32266AA
     created: 2016-02-07  expires: 2017-02-06  usage: S   
ssb  rsa2048/0xBF361ED434425B4C
     created: 2016-02-07  expires: 2017-02-06  usage: E   
ssb  rsa2048/0x9C7CA4E294F04D49
     created: 2016-02-07  expires: 2017-02-06  usage: A   
[ultimate] (1). Norbert Preining <norbert@preining.info>
[ultimate] (2)  Norbert Preining <preining@logic.at>
[ultimate] (3)  Norbert Preining <preining@debian.org>
[ultimate] (4)  Norbert Preining <preining@jaist.ac.jp>
[ultimate] (5)  [jpeg image of size 4185]
 
gpg> save

Note the repetition of selecting and deselecting keys. Current status After this procedure we are now in the following situation:

gpg1: all keys are still available
gpg2: sub keys are moved to yubikey (indicated below by ssb>), and master key is still available

In gpg words it looks like this:

$ gpg2 -K $MASTERKEY
gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf
sec   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb>  rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb>  rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb>  rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
 
$ gpg -K $MASTERKEY
sec   4096R/0x6CACA448860CDC13 2010-09-14 [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
uid                            [jpeg image of size 4185]
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14 [expires: 2017-02-06]
ssb   2048R/0xEC00B8DAD32266AA 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0xBF361ED434425B4C 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0x9C7CA4E294F04D49 2016-02-07 [expires: 2017-02-06]
 
$ gpg2 --card-status
 
....
Name of cardholder: Norbert Preining
....
PIN retry counter : 3 3 3
Signature counter : 0
Signature key ....: 5871 F824 2DCC 3660 2362  BE7D EC00 B8DA D322 66AA
      created ....: 2016-02-07 11:10:06
Encryption key....: 2501 195C 90AB F4D2 3DEA  A303 BF36 1ED4 3442 5B4C
      created ....: 2016-02-07 11:10:20
Authentication key: 9CFB 3775 C164 0E99 F0C8  014C 9C7C A4E2 94F0 4D49
      created ....: 2016-02-07 11:10:34
General key info..: sub  rsa2048/0xEC00B8DAD32266AA 2016-02-07 Norbert Preining <norbert@preining.info>
sec   rsa4096/0x6CACA448860CDC13  created: 2010-09-14  expires: 2017-02-06
ssb   rsa4096/0xD1D2BD14810F62B3  created: 2010-09-14  expires: 2017-02-06
ssb>  rsa2048/0xEC00B8DAD32266AA  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
ssb>  rsa2048/0xBF361ED434425B4C  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
ssb>  rsa2048/0x9C7CA4E294F04D49  created: 2016-02-07  expires: 2017-02-06
                                  card-no: 0006 03645719
$

Remove private master keys You are sure that you have a working backup? Did you try it with gpg --homedir ...? Only if you are really sure, continue. We are now removing the master key from both the gpg2 and gpg1 setup. removal for gpg2 gpg2 keeps the private keys in ~/.gnupg/private-keys-v1.d/KEYGRIP.key and the KEYGRIP can be found by adding --with-keygrip to the key listing. Be sure to delete the correct file, the one related to the master key.

$ gpg2 --with-keygrip --list-key $MASTERKEY
pub   rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
      Keygrip = 9DC1E90703856C1DE0EAC970CED7ABF5EE5EF79D
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
sub   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
      Keygrip = 4B8FF57434DD989243666377376903281D861596
sub   rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
      Keygrip = 39B14EF1392F2F251863A87AE4D44CE502755C39
sub   rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
      Keygrip = E41C8DDB2A22976AE0DA8D7D11F586EA793203EA
sub   rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
      Keygrip = A337DE390143074C6DBFEA64224359B9859B02FC
 
$ rm ~/.gnupg/private-keys-v1.d/9DC1E90703856C1DE0EAC970CED7ABF5EE5EF79D.key
$

After that the missing key is shown in gpg2 -K with an additional # meaning that the key is not available:

$ gpg2 -K $MASTERKEY
sec#  rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
...

removal for gpg1 Up to gpg v2.0 there is no simple way to delete only one part of the key. We export the subkeys, delete the private key, and reimport the subkeys:

$ gpg --output secret-subkeys --export-secret-subkeys $MASTERKEY
 
$ gpg --delete-secret-keys $MASTERKEY
 
sec  4096R/0x6CACA448860CDC13 2010-09-14 Norbert Preining <norbert@preining.info>
 
Delete this key from the keyring? (y/N) y
This is a secret key! - really delete? (y/N) y
 
$ gpg --import secret-subkeys
gpg: key 0x6CACA448860CDC13: secret key imported
gpg: key 0x6CACA448860CDC13: "Norbert Preining <norbert@preining.info>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
gpg:       secret keys read: 1
gpg:   secret keys imported: 1
 
$

Current status We are basically at the stage we wanted to achieve: For gpg2.1 only the old encryption key is available, the master key is not, and the other sub keys are moved to the yubikey:

$ gpg2 -K $MASTERKEY
sec#  rsa4096/0x6CACA448860CDC13 2010-09-14 [SC] [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                   [ultimate] Norbert Preining <norbert@preining.info>
uid                   [ultimate] Norbert Preining <preining@logic.at>
uid                   [ultimate] Norbert Preining <preining@debian.org>
uid                   [ultimate] Norbert Preining <preining@jaist.ac.jp>
uid                   [ultimate] [jpeg image of size 4185]
ssb   rsa4096/0xD1D2BD14810F62B3 2010-09-14 [E] [expires: 2017-02-06]
ssb>  rsa2048/0xEC00B8DAD32266AA 2016-02-07 [S] [expires: 2017-02-06]
ssb>  rsa2048/0xBF361ED434425B4C 2016-02-07 [E] [expires: 2017-02-06]
ssb>  rsa2048/0x9C7CA4E294F04D49 2016-02-07 [A] [expires: 2017-02-06]
$

And for gpg <= 2.0 the old encryption key and the sub keys are available, but the master key is not:

$ gpg -K $MASTERKEY
sec#  4096R/0x6CACA448860CDC13 2010-09-14 [expires: 2017-02-06]
      Key fingerprint = F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
uid                            Norbert Preining <norbert@preining.info>
uid                            Norbert Preining <preining@logic.at>
uid                            Norbert Preining <preining@debian.org>
uid                            Norbert Preining <preining@jaist.ac.jp>
uid                            [jpeg image of size 4185]
ssb   4096R/0xD1D2BD14810F62B3 2010-09-14 [expires: 2017-02-06]
ssb   2048R/0xEC00B8DAD32266AA 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0xBF361ED434425B4C 2016-02-07 [expires: 2017-02-06]
ssb   2048R/0x9C7CA4E294F04D49 2016-02-07 [expires: 2017-02-06]
 
$

Split the .gnupg directory for mail server and laptop As mentioned, I want to have a gpg1 version available at the server where I read my emails, and be able to sign/encrypt emails there, while on my laptop no secret key is available. Thus I prepare two gnupg directories. For the mailserver the gpg2 specific files are removed:

$ cp -a .gnupg .gnupg-mail
$ cd .gnupg-mail
$ rm -rf private-keys-v1.d/ pubring.gpg~ reader_0.status
$ rm -rf S.gpg-agent* S.scdaemon .gpg-v21-migrated

On my laptop, where I did all this operation, I remove the gpg1 files, namely the outdated secring.gpg:

$ cd $HOME/.gnupg
$ rm secring.gpg

As a last step I move the .gnupg-mail directory to my mail server. Once could *expire* the old encryption key, but for now I leave it as is. Upload keys to keyservers If you are a Debian Developer, a simple update of your master key will suffice:

gpg --keyserver hkp://keyring.debian.org --send-key YOURMASTERKEYID

Note that the update from the keyring server to the actual Debian keyring takes up to one month. Until that time either do not upload anything, or use the (offline) master key for signing. After your key has been updated in the Debian keyring, signatures made with the signing subkey will be accepted for uploading to Debian. It might be also a good idea to upload your new keys to some keyservers like:

gpg --keyserver hkp://pool.sks-keyservers.net --send-key $MASTERKEY

Now you an also fix the configuration file skew between gpg1 and gpg2. Further remark I am currently trying to use the authentication key from my Yubikey NEO as ssh key, but bugs (see #795368 and #818969) prohibit it at the moment. Raphael Herzog gave a possible fix by killing the gpg-agent and restarting it with gpg-agent --daemon from an X terminal, and I can confirm that this worked. After one year before the key expires I need to extend the key validity for another year. For this you need the offline master key. I will describe the process when it becomes necessary. Reading list The following web sites have been useful in collecting the necessary information:

https://iain.learmonth.me/yubikey-neo-gpg/
https://iain.learmonth.me/yubikey-udev/
http://blog.josefsson.org/2014/06/23/offline-gnupg-master-key-and-subkeys-on-yubikey-neo-smartcard/
https://wiki.debian.org/Subkeys
https://jclement.ca/articles/2015/gpg-smartcard/ as modernized version of (3)
https://www.esev.com/blog/post/2015-01-pgp-ssh-key-on-yubikey-neo/ similar style, with ssh and gnome-keyring infos
http://karlgrz.com/2fa-gpg-ssh-keys-with-pass-and-yubikey-neo/ also good reading
https://help.riseup.net/en/security/message-security/openpgp/best-practices good and concise advise on gpg practices

My writing is mostly based on (5) with additions from (4). Please let me know of any errors, improvements, and fixes. I hope this walk-through might help others in the same situation.

Although, AWS provides load balancer service in the form of Elastic Load Balancer (ELB), a common trick is to use HAProxy in the middle to provide SSL offloading, complex routing and better logging.
In this scenario, a public ELB is the frontier of all the traffic, HAProxy farm in the middle is managed by an Auto Scaling Group, and one (or more) internal backend ELBs stay in front of Web farm. haproxy

I think that HAProxy does not need any introductions here. It is highly scalable and reliable piece of software. There is however a small caveat when you use it with domain names and not IP addresses. To speed up things, HAProxy resolves all the domain named during startup (during config file parsing in fact). Hence, when the IP of a domain changes, you end up with a lot of 503s (Service Unavailable). Why is this important ? In AWS, ELB's IP can change over time, so it is recommended to use ELB's domain name. Now, when you use this domain name in HAProxy's backend, you can end up with 503s. ELB IPs do not change so often but still you would not want any downtimes. The solution is to configure runtime resolvers in HAProxy and use them in the backend (unforntunatelly this works only in HAProxy 1.6):

 ::haproxy
 resolvers myresolver
      nameserver dns1 10.10.10.10:53
      resolve_retries       30
      timeout retry         1s
      hold valid           10s
  backend mybackend
      server myelb-internal.123456.eu-west-1.elb.amazonaws.com check resolvers myresolver

Now HAProxy will check the domain at runtime, no more 503s.

What happened in the reproducible builds effort between April 3rd and April 9th 2016: Media coverage Emily Ratliff wrote an article for SecurityWeek called Establishing Correspondence Between an Application and its Source Code - How Combining Two Completely Separate Open Source Projects Can Make Us All More Secure. Tails have started work on a design for freezable APT repositories to make it easier and practical to perform reproductions of an entire distribution at a given point in time, which will be needed to create reproducible installation- or live-media. Toolchain fixes Alexis Bienven e submitted patches adding support for SOURCE_DATE_EPOCH in several tools: transfig, imagemagick, rdtool, and asciidoctor. boyska submitted one for python-reportlab. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330 brailleutils cglib3 gnugo libcobra-java libgnumail-java libjchart2d-java libjcommon-java libjfreechart-java libjide-oss-java liblaf-widget-java liblastfm-java liboptions-java octave-control octave-mpi octave-nan octave-parallel octave-stk octave-struct octave-tsa oar The following packages became reproducible after getting fixed:

apt-listchanges/2.88 by Robert Luberda.
cgal/4.8-1 by Joachim Reichel.
erlang-bitcask/2.0.2+dfsg-1 by Nobuhiro Iwamatsu.
geneweb/6.08+git20160228+dfsg-2 by Guillaume Brochu.
glance/2:12.0.0~rc2-1 uploaded by Thomas Goirand, original patch by Chris Lamb.
gle-graphics/4.2.5-4 by Christian T. Steigies.
maude uploaded by Andreas Tille, patch by Alexis Bienven e.
psqlodbc/1:09.05.0100-3 by Christoph Berg.
resource-agents/1:3.9.7-3 by Christoph Berg.
vgabios/0.7a-6 by Reiner Herrmann.
vim/2:7.4.1689-2 by James McCoy.
xmhtml/1.1.10-2 by Graham Inggs with Reiner Herrmann's help.
xscreensaver/5.34-2 uploaded by Tormod Volden, original patch by Sascha Steinbiss.

Several uploads fixed some reproducibility issues, but not all of them:

rkward/0.6.5-1 uploaded by Thomas Friedrichsmeier, original patch by Philip Rinn.
mailfilter/0.8.4-1 uploaded by Elimar Riesebieter, original patch by Chris Lamb.
bind9/1:9.10.3.dfsg.P4-6 uploaded by Michael Gilbert, original patch by Reiner Herrmann.
bzr/2.7.0- 3,4 by Jelmer Vernooij.
samba/2:4.3.6+dfsg-2 uploaded by Mathieu Parent, fix by Jelmer Vernooij.
fwupdate/0.5-3 by Mario Limonciello.
paraview/5.0.1+dfsg1-1 by Anton Gladky.

Patches submitted which have not made their way to the archive yet:

#819883 on debootstrap by Reiner Herrmann: tell tar to sort the archive members.
#819885 on chktex by Sascha Steinbiss: use the time of latest debian/changelog entry as documentation timestamp.
#819915 on kannel by Alexis Bienven e: use the time of latest debian/changelog entry as documentation timestamp.
#819921 on basket by Alexis Bienven e: remove build date from debug info.
#819965 on openarena-data by Alexandre Detiste: normalize file permissions before creating .pk3 archive.
#820016 on gabedit by Alexis Bienven e: sort object files used to build the executable.
#820032 on bibledit-gtk by Alexis Bienven e: remove useless included Makefile.
#820072 on synfig by Alexis Bienven e: remove build date from info output.
#820148 on autopkgtest by Alexis Bienven e: fix install order to cope with locales with case insensitive globbing.
#820152 on anope by Alexis Bienven e: remove build date from the version string.
#820179 on aodh by Alexis Bienven e: remove build date from the documentation.
#820183 on cython by Alexis Bienven e: add support SOURCE_DATE_EPOCH.
#820194 on nasm by rain1: sorts keys when traversing hash tables used to build the documentation.
#820226 on chrony by Alexis Bienven e: add support for SOURCE_DATE_EPOCH to preset the ntp_era_split parameter.
#820457 on recode by Alexis Bienven e: use system help2man.
#820522 on gtkspell by Alexis Bienven e: force shell to /bin/sh in example Makefile.

Other upstream fixes Alexander Batischev made a commit to make newsbeuter reproducible. tests.reproducible-builds.org

An architecture agnostic summary has been added to the reproducible-tracker.json by Valerie Young to make it easy to parse whether a package is unreproducible anywhere.
To find more reproducibility issues a new variation was added to the i386 builders, so that one build is done using a 32 bit kernel (686-PAE) and the other build is using a 64 bit kernel (amd64). (h01ger)
- Niko Tyni was the first to notice a bug due to this: #821182 perl: embeds kernel architecture information
The 2nd builds are now done in fr_CH on amd64, de_CH on i386 and it_CH on armhf. (h01ger)
The variation table has been updated to reflect the recent changes and various small bugs have been fixed. (h01ger)

Package reviews 93 reviews have been removed, 66 added and 21 updated in the previous week. 12 new FTBFS bugs have been reported by Chris Lamb and Niko Tyni. Misc. This week's edition was written by Lunar, Holger Levsen, Reiner Herrmann, Mattia Rizzolo and Ximin Luo. With the departure of Lunar as a full-time contributor, Reproducible Builds Weekly News (this thing you're reading) has moved from his personal Debian blog on Debian People to the Reproducible Builds team web site on Debian Alioth. You may want to update your RSS or Atom feeds. Very many thanks to Lunar for writing and publishing this weekly news for so long, well & continously!

It is days like today I am really happy to be a member of the Norwegian Unix User group, a member association for those of us believing in free software, open standards and unix-like operating systems. NUUG announced today it will try to bring the seizure of the DNS domain popcorn-time.no as unlawful, to stand up for the principle that writing about a controversial topic is not infringing copyrights, and censuring web pages by hijacking DNS domain should be decided by the courts, not the police. The DNS domain was seized by the Norwegian National Authority for Investigation and Prosecution of Economic and Environmental Crime a month ago. I hope this bring more paying members to NUUG to give the association the financial muscle needed to bring this case as far as it must go to stop this kind of DNS hijacking.

A few days ago a new upstream version "2.0" of CCTZ was released. See here for the corresponding post on the Google OpenSource Blog. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. This new version adds more support to the notion of civil time representation -- i.e. independent of timezones -- which can then be mapped to different time zone representations. Changes in this version are summarized here:

Changes in version 0.0.4 (2016-04-17)

Synchronized with CCTZ v2 upstream.

Updated examples.cpp accordingly

A quick example is provided here where we look at the time when Neil Armstrong first stepped on the moon as an absolute ("civil") time and two local representations:

// from examples/hello.cc
// 
// [[Rcpp::export]]
int helloMoon()  
    cctz::time_zone syd;
    if (!cctz::load_time_zone("Australia/Sydney", &syd)) return -1;
    // Neil Armstrong first walks on the moon
    const auto tp1 = cctz::convert(cctz::civil_second(1969, 7, 21, 12, 56, 0), syd);
    const std::string s = cctz::format("%F %T %z", tp1, syd);
    Rcpp::Rcout << s << "\n";
    cctz::time_zone nyc;
    cctz::load_time_zone("America/New_York", &nyc);
    const auto tp2 = cctz::convert(cctz::civil_second(1969, 7, 20, 22, 56, 0), nyc);
    return tp2 == tp1 ? 0 : 1;

We can call this from R, and get the expected result (of equivalence between the dates):

R> library(RcppCCTZ)
R> helloMoon()
1969-07-21 12:56:00 +1000
[1] 0
R>

We also have a diff to the previous version thanks to CRANberries. More details, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

As a rule, I avoid writing publicly on political topics, but I m making an exception. In case you haven t been following it, the senior Republican and the senior Democrat on the Senate Intelligence Committee recently announced a legislative proposal misleadingly called the Compliance with Court Orders Act of 2016. The full text of the draft can be found here. It would effectively ban devices and software in the United States that the manufacturer cannot retrieve data from. Here is a good analysis of the breadth of the proposal and a good analysis of the bill itself. While complying with court orders might sound great in theory, in practice this means these devices and software will be insecure by design. While that s probably reasonably obvious to most normal readers here, don t just take my word for it, take Bruce Schneier s. In my opinion, policy makers (and it s not just in the United States) are suffering from a perception gap about security and how technically hard it is to get right. It seems to me that they are convinced that technologists could just do security right while still allowing some level of extraordinary access for law enforcement if they only wanted to. We ve tried this before and the story never seems to end well. This isn t a complaint from wide eyed radicals that such extraordinary access is morally wrong or inappropriate. It s hard core technologists saying it can t be done. I don t know how to get the message across. Here s President Obama, in my opinion, completely missing the point when he equates a desire for security with fetishizing our phones above every other value. Here are some very smart people trying very hard to be reasonable about some mythical middle ground. As Riana Pfefferkorn s analysis that I linked in the first paragraph discusses, this middle ground doesn t exist and all the arm waving in the world by policy makers won t create it. Coincidentally, this same week, the White House announced a new Commission on Enhancing National Cybersecurity . Cybersecurity is certainly something we could use more of, unfortunately Congress seems to be heading off in the opposite direction and no one from the executive branch has spoken out against it. Security and privacy are important to many people. Given the personal and financial importance of data stored in computers (traditional or mobile), users don t want criminals to get a hold of it. Companies know this, which is why both Apple IOS and Google Android both encrypt their local file systems by default now. If a bill anything like what s been proposed becomes law, users that care about security are going to go elsewhere. That may end up being non-US companies products or US companies may shift operations to localities more friendly to secure design. Either way, the US tech sector loses. A more accurate title would have been Technology Jobs Off-Shoring Act of 2016. EDIT: Fixed a typo.

Since mid-2015 we were using the dep11-generator in Debian to build AppStream metadata about available software components in the distribution. Getting rid of dep11-generator Unfortunately, the old Python-based dep11-generator was hitting some hard limits pretty soon. For example, using multiprocessing with Python was a pain, since it resulted in some very hard-to-track bugs. Also, the multiprocessing approach (as opposed to multithreading) made it impossible to use the underlying LMDB database properly (it was basically closed and reopened in each forked off process, since pickling the Python LMDB object caused some really funny bugs, which usually manifested themselves in the application hanging forever without any information on what was going on). Additionally to that, the Python-based generator forced me to maintain two implementations of the AppStream YAML spec, one in C and one in Python, which consumes quite some time. There were also some other issues (e.g. no unit-tests) in the implementation, which made me think about rewriting the generator. Adventures in Go / Rust / D Since I didn t want to write this new piece of software in C (or basically, writing it in C was my last option

), I explored Go and Rust for this purpose and also did a small prototype in the D programming language, when I was starting to feel really adventurous. And while I never intended to write the new generator in D (I was pretty fixated on Go ), this is what happened. The strong points for D for this particular project were its close relation to C (and ease of using existing C code), its super-flat learning curve for someone who knows and likes C and C++ and its pretty powerful implementations of the concurrent and parallel programming paradigms. That being said, not all is great in D and there are some pretty dark spots too, mainly when it comes to the standard library and compilers. I will dive into my experiences with D in a separate blogpost. What good to expect from appstream-generator? So, what can the new appstream-generator do for you? Basically, the same as the old dep11-generator: It will extract metadata from a distribution s package archive, download and resize screenshots, search for icons and size them properly and generate reports in JSON and HTML of found metadata and issues. LibAppStream-based parsing, generation of YAML or XML, multi-distro support, As opposed to the old generator, the new generator utilizes the metadata parsers and writers of libappstream. This allows it to return the extracted metadata as AppStream YAML (for Debian) or XML (everyone else) It is also written in a distribution-agnostic way, so if someone wants to use it in a different distribution than Debian, this is possible now. It just requires a very small distribution-specific backend to be written, all of the details of the metadata extraction are abstracted away (just two interfaces need to be implemented). While I do not expect anyone except Debian to use this in the near future (most distros have found a solution to generate metadata already), the frontend-backend split is a much cleaner design than what was available in the previous code. It also allows to unit-test the code properly, without providing a Debian archive in the testsuite. Feature Flags, Optipng, The new generator also allows to enable and disable certain sets of features in a standardized way. E.g. Ubuntu uses a language-pack system for translations, which Debian doesn t use. Features like this can be implemented as disableable separate modules in the generator. We use this at time to e.g. allow descriptions from packages to be used as AppStream descriptions, or for running optipng on the generated PNG images and icons. No more Contents file dependency Another issue the old generator had was that it used the Contents file from the Debian archive to find matching icons for an application. We could never be sure whether the contents in the Contents file actually matched the contents of the package we were currently dealing with. What made things worse is that at Ubuntu, the archive software is only updating the Contents file ~~weekly~~ daily (while the generator might run multiple times a day), which has lead to software being ignored in the metadata, because icons could not yet be found. Even on Debian, with its quickly-updated Contents file, we could immediately see the effects of an out-of-date Contents file when updating it failed once. In the new generator, we read the contents of each package ourselves now and store them in a LMDB database, bypassing the Contents file and removing the whole class of problems resulting from missing or wrong contents-data. It can t all be good, right? That is true, there are also some known issues the new generator has: Large amounts of RAM required The better speed of the new generator comes at the cost of holding more stuff in RAM. Much more. When processing data from 5 architectures initially on Debian, the amount of required RAM might lie above 4GB, with the OOM killer sometimes being quicker than the garbage collector That being said, on subsequent runs the amount of required memory is much lower. Still, this is something I am working on to improve. What are symbolic links? To be faster, the appstream-generator will read the md5sum file in .deb packages instead of extracting the payload archive and reading its contents. Since the md5sums file does not list symbolic links, symlinks basically don t exist for the new generator. This is a problem for software symlinking icons or even .desktop files around, like e.g. LibreOffice does. I am still investigating how widespread the use of symlinks for icons and .desktop files is, but it looks like fixing packages (making them not-symlink stuff and rather move the files) might be the better approach than investing additional computing power to find symlinks or even switch back to parsing the Contents file. Input on this is welcome! Deploying asgen I finished the last pieces of the appstream-generator (together with doing lots of other cool things and talking to great people) at the GNOME Software Hackfest in London last week (detailed blogposts about things that happened there will follow many thanks once again for the Ubuntu community for sponsoring my attendance!). Since today, the new generator is running on the Debian infrastructure. If bigger issues are found, we can still roll back to the old code. I decided to deploy this faster, so we can get some good testing done before the Stretch release. Please report any issues you may find!

The first time I was paid to do software development came as something of a surprise to me. I was working as a sysadmin in a computational physics research group when a friend asked me if I'd be willing to talk to her PhD supervisor. I had nothing better to do, so said yes. And that was how I started the evening having dinner with David MacKay, and ended the evening better fed, a little drunker and having agreed in principle to be paid to write free software.

I'd been hired to work on Dasher, an information-efficient text entry system. It had been developed by one of David's students as a practical demonstration of arithmetic encoding after David had realised that presenting a visualisation of an effective compression algorithm allowed you to compose text without having to enter as much information into the system. At first this was merely a neat toy, but it soon became clear that the benefits of Dasher had a great deal of overlap with good accessibility software. It required much less precision of input, it made it easy to correct mistakes (you merely had to reverse direction in order to start zooming back out of the text you had entered) and it worked with a variety of input technologies from mice to eye tracking to breathing. My job was to take this codebase and turn it into a project that would be interesting to external developers.

In the year I worked with David, we turned Dasher from a research project into a well-integrated component of Gnome, improved its support for Windows, accepted code from an external contributor who ported it to OS X (using an OpenGL canvas!) and wrote ports for a range of handheld devices. We added code that allowed Dasher to directly control the UI of other applications, making it possible for people to drive word processors without having to leave Dasher. We taught Dasher to speak. We strove to avoid the mistakes present in so many other pieces of accessibility software, such as configuration that could only be managed by an (expensive!) external consultant. And we visited Dasher users and learned how they used it and what more they needed, then went back home and did what we could to provide that.

Working on Dasher was an incredible opportunity. I was involved in the development of exciting code. I spoke on it at multiple conferences. I became part of the Gnome community. I visited the USA for the first time. I entered people's homes and taught them how to use Dasher and experienced their joy as they realised that they could now communicate up to an order of magnitude more quickly. I wrote software that had a meaningful impact on the lives of other people.

Working with David was certainly not easy. Our weekly design meetings were, charitably, intense. He had an astonishing number of ideas, and my job was to figure out how to implement them while (a) not making the application overly complicated and (b) convincing David that it still did everything he wanted. One memorable meeting involved me gradually arguing him down from wanting five new checkboxes to agreeing that there were only two combinations that actually made sense (and hence a single checkbox) - and then admitting that this was broadly equivalent to an existing UI element, so we could just change the behaviour of that slightly without adding anything. I took the opportunity to delete an additional menu item in the process.

I was already aware of the importance of free software in terms of developers, but working with David made it clear to me how important it was to users as well. A community formed around Dasher, helping us improve it and allowing us to develop support for new use cases that made the difference between someone being able to type at two words per minute and being able to manage twenty. David saw that this collaborative development would be vital to creating something bigger than his original ideas, and it succeeded in ways he couldn't have hoped for.

I spent a year in the group and then went back to biology. David went on to channel his strong feelings about social responsibility into issues such as sustainable energy, writing a freely available book on the topic. He served as chief adviser to the UK Department of Energy and Climate Change for five years. And earlier this year he was awarded a knighthood for his services to scientific outreach.

David died yesterday. It's unlikely that I'll ever come close to what he accomplished, but he provided me with much of the inspiration to try to do so anyway. The world is already a less fascinating place without him.

comments

Search Results: "ron"

26 April 2016

Reproducible builds folks: Reproducible builds: week 52 in Stretch cycle

Steinar H. Gunderson: Full stack

Matthias Klumpp: A GNOME Software Hackfest report

Francois Marier: Using DNSSEC and DNSCrypt in Debian

25 April 2016

Antoine Beaupr : My free software activities, April 2016

Other free software work With that in mind, I obviously didn't stop doing other computing work this month. In fact, I did a lot of work to try to generally fix the internet, that endless and not-quite-gratifying hobby so many of us are destroying our bodies to accomplish.

Norbert Preining: G del and Daemons an excursion into literature

24 April 2016

Bits from Debian: Debian welcomes its 2016 summer interns

22 April 2016

Matthew Garrett: Circumventing Ubuntu Snap confinement

21 April 2016

John Goerzen: Count me as a systemd convert

Alessio Treglia: Corporate Culture in the Transformative Enterprise

20 April 2016

Jonathan Dowland: mount-on-demand backups

Reproducible builds folks: Reproducible builds: week 51 in Stretch cycle

Norbert Preining: GnuPG notes: subkeys, yubikey, gpg1 vs gpg2

19 April 2016

Dariusz Dwornikowski: HAProxy and 503 HTTP errors with AWS ELB as a backend

18 April 2016

Reproducible builds folks: Reproducible builds: week 50 in Stretch cycle

Petter Reinholdtsen: NUUG contests Norwegian police DNS seizure of popcorn-time.no

Dirk Eddelbuettel: RcppCCTZ 0.0.4

16 April 2016

Scott Kitterman: Future of secure systems in the US

Matthias Klumpp: Introducing AppStream-Generator

15 April 2016

Matthew Garrett: David MacKay