Two years ago, I had a look at trusted timestamping options available, and among other things noted a still open bug in the tsget script included in openssl that made it harder than necessary to use openssl as a trusted timestamping client. A few days ago I was told the Norwegian government office DIFI is close to releasing their own trusted timestamp service, and in the process I was happy to learn about a replacement for the tsget script using only curl:

openssl ts -query -data "/etc/shells" -cert -sha256 -no_nonce \
    curl -s -H "Content-Type: application/timestamp-query" \
         --data-binary "@-" http://zeitstempel.dfn.de > etc-shells.tsr
openssl ts -reply -text -in etc-shells.tsr

This produces a binary timestamp file (etc-shells.tsr) which can be used to verify that the content of the file /etc/shell with the calculated sha256 hash existed at the point in time when the request was made. The last command extract the content of the etc-shells.tsr in human readable form. The idea behind such timestamp is to be able to prove using cryptography that the content of a file have not changed since the file was stamped. To verify that the file on disk match the public key signature in the timestamp file, run the following commands. It make sure you have the required certificate for the trusted timestamp service available and use it to compare the file content with the timestamp. In production, one should of course use a better method to verify the service certificate.

wget -O ca-cert.txt https://pki.pca.dfn.de/global-services-ca/pub/cacert/chain.txt
openssl ts -verify -data /etc/shells -in etc-shells.tsr -CAfile ca-cert.txt -text

Wikipedia have a lot more information about trusted Timestamping and linked timestamping, and there are several trusted timestamping services around, both as commercial services and as free and public services. Among the latter is the zeitstempel.dfn.de service mentioned above and freetsa.org service linked to from the wikipedia web site. I believe the DIFI service should show up on https://tsa.difi.no, but it is not available to the public at the moment. I hope this will change when it is into production. The RFC 3161 trusted timestamping protocol standard is even implemented in LibreOffice, Microsoft Office and Adobe Acrobat, making it possible to verify when a document was created. I would find it useful to be able to use such trusted timestamp service to make it possible to verify that my stored syslog files have not been tampered with. This is not a new idea. I found one example implemented on the Endian network appliances where the configuration of such feature was described in 2012. But I could not find any free implementation of such feature when I searched, so I decided to try to build a prototype named syslog-trusted-timestamp. My idea is to generate a timestamp of the old log files after they are rotated, and store the timestamp in the new log file just after rotation. This will form a chain that would make it possible to see if any old log files are tampered with. But syslog is bad at handling kilobytes of binary data, so I decided to base64 encode the timestamp and add an ID and line sequence numbers to the base64 data to make it possible to reassemble the timestamp file again. To use it, simply run it like this:

syslog-trusted-timestamp /path/to/list-of-log-files

This will send a timestamp from one or more timestamp services (not yet decided nor implemented) for each listed file to the syslog using logger(1). To verify the timestamp, the same program is used with the --verify option:

syslog-trusted-timestamp --verify /path/to/log-file /path/to/log-with-timestamp

The verification step is not yet well designed. The current implementation depend on the file path being unique and unchanging, and this is not a solid assumption. It also uses process number as timestamp ID, and this is bound to create ID collisions. I hope to have time to come up with a better way to handle timestamp IDs and verification later. Please check out the prototype for syslog-trusted-timestamp on github and send suggestions and improvement, or let me know if there already exist a similar system for timestamping logs already to allow me to join forces with others with the same interest. As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

I'm not a big fan of April Fools' day, but this year I decided to try one. My modest contribution: a Pull Request for Chocolate Doom titled Bring back the classic Doom "hit enter" behaviour:

When playing PWADs with classic Doom, you always got the "this game has been modified" banner and had to hit enter to continue. If you played a lot of PWADs, you got used to hitting enter regularly. It became a core part of the Doom-playing experience. Even if you used one of those lovely launchers that came on the shovelware CDs which streamlined the PWAD choosing experience, you always had to hit the enter key. This commit restores this classic behaviour and allows those who still have the muscle-memory "hit enter" behaviour permanently burned into their nervous systems to finally be at peace. TODO for a future commit: the coloured top-line of text (red for 1.666, blue for 1.9 iirc)

Some feedback from IRC:

<Linguica> its funny because its nearly indistinguishable from real chocolate doom feature requests <jmickle> i actually first thought "huh, kinda surprised this isn't in already <ryan> best chocolate-doom feature yet!

Debian Long Term Support (LTS) This is my 4th month working on Debian LTS, started by Raphael Hertzog at Freexian. I spent half of the month away on a vacation so little work was done, especially since I tried to tackle rather large uploads like NSS and Xen. I also worked on the frontdesk shift last week.

Frontdesk That work mainly consisted of figuring out how to best help the security team with the last uploads to the Wheezy release. For those who don't know, Debian 7 Wheezy, or "oldstable", is going to be unsupported by the security team starting end of april, and the Debian 6 Squeeze (the previous LTS) is now unsupported. The PGP signatures on the archived release have started yielding expiration errors which can be ignored but that are really a strong reminder that it is really time to upgrade. So the LTS team is now working towards backporting a few security issues from squeeze to wheezy, and this is what I focused on during triage work. I have identified the following high priority packages I will work on after I complete my work on the Xen and NSS packages (detailed below):

• libidn
• icu
• phpmyadmin
• tomcat6
• optipng
• srtp
• dhcpcd
• python-tornado

Updates to NSS and Xen I have spent a lot of time testing and building packages for NSS and Xen. To be fair, Brian May did most of the work on the Xen packages, and I merely did some work to test the packages on Koumbit's infrastructure, something which I will continue doing in the next month. For NSS, wheezy and jessie are in this weird state where patches were provided to the security team all the way back in November yet were never tested. Since then, yet more issues came up and I worked hard to review and port patches for those new security issues to wheezy. I'll followup on both packages in the following month.

Other free software work

Android TL;DR: there's an even longer version of this with the step-by-step procedures and that I will update as time goes on in my wiki. I somehow inherited an Android phone recently, on a loan from a friend because the phone broke one too many times and she got a new one from her provider. This phone is a HTC One S "Ville", which is fairly old, but good enough to play with and give me a mobile computing platform to listen to podcasts, play music, access maps and create GPS traces. I was previously doing this with my N900, but that device is really showing its age: very little development is happening on it, the SDK is closed source and the device itself is fairly big, compared to the "Ville". Plus, the SIM card actually works on the Ville so, even though I do not have an actual contract with a cell phone provider (too expensive, too invasive on my privacy), I can still make emergency phone calls (911)! Plus, since there is good wifi on the machine, I can use it to connect to the phone system with the built-in SIP client, send text messages through SMS (thanks to VoIP.ms SMS support) or Jabber. I have also played around with LibreSignal, the free software replacement for Signal, which uses proprietary google services. Yes, the VoIP.ms SMS app also uses GCM, but hopefully that can be fixed. (While I was writing this, another Debian Developer wrote a good review of Signal so I am happy to skip that step. Go read that.)

See also my apps list for a more complete list of the apps I have installed on the phone. I welcome recommendations on cool free software apps I should use!

I have replaced the stock firmware on the phone with Cyanogenmod 12.1, which was a fairly painful experience, partly because of the difficult ambiance on the #cyanogenmod channel on Freenode, where I had extreme experiences: a brave soul helped me through the first flashing process for around 2 hours, nicely holding my hand at every step. Other times, I have seen flames and obtuse comments from people being vulgar, brutal, obnoxious, if not sometimes downright homophobic and sexist from other users. It is clearly a community that needs to fix their attitude. I have documented everything I could in details in this wiki page, in case others want to resuscitate their old phones, but also because I ended up reinstalling the freaking phone about 4 times, and was getting tired of forgetting how to do it every time. I am somewhat fascinated by Android: here is the Linux-based device that should save us all from the proprietary Apple nightmares of fenced in gardens and censorship. Yet, public Android downloads are hidden behind license agreements, even though the code itself is free, which has led fellow Debian developers to work on making libre rebuilds of Androids to workaround this insanity. But worse: all phones are basically proprietary devices down to the core. You need custom firmware to be loaded on the machine for it to boot at all, from the bootloader all the way down to the GSM baseband and Wifi drivers. It is a minefield of closed source software, and trying to run free software on there is a bit of a delusion, especially since the baseband has so much power over the phone. Still, I think it is really interesting to run free software on those machines, and help people that are stuck with cell phones get familiar with software freedom. It seems especially important to me to make Android developers aware of software freedom considering how many apps are available for free yet on which it is not possible to contribute significantly because the source code is not published at all, or published only on the Google Store, instead of the more open and public F-Droid repository which publishes only free software. So I did contribute. This month, I am happy to announce that I contributed to the following free software projects on Android:

• listmyapps: feature request, patch
• Locker: bug report, added to f-droid
• Sensorium: bug report
• LibreSignal: feature request and testing

I have also reviewed the literature surrounding Telegram, a popular messaging app rival to Signal and Whatsapp. Oddly enough, my contributions to Wikipedia on that subject were promptly reverted which made me bring up the subject on the page's Talk page. This lead to an interesting response from the article's main editors which at least added that the "its security features have been contested by security researchers and cryptography experts". Considering the history of Telegram, I would keep people away from it and direct people to use Signal instead, even though Signal has similar metadata issues, mostly because Telegram's lack of response to the security issues that were outlined by fellow security researchers. Both systems suffer from a lack of federation as well, which is a shame in this era of increasing centralization. I am not sure I will put much more work in developing for Android for now. It seems like a fairly hostile platform to work on, and unless I have specific pain points I want to fix, it feels so much better to work on my own stuff in Debian. Which brings me to my usual plethora of free software projects I got stuck in this month.

IRC projects irssi-plugin-otr had a bunch of idiotic bugs lying around, and I had a patch that I hadn't submitted upstream from the Debian package, which needed a rebuild because the irssi version changed, which is a major annoyance. The version in sid is now a snapshot because upstream needs to make a new release but at least it should fix things for my friends running unstable and testing. Hopefully those silly uploads won't be necessary in the future. That's for the client side. On the server side, I have worked on updating the Atheme-services package to the latest version, which actually failed because the upstream libmowgli is missing release tags, which means the Debian package for it is not up-to-date either. Still, it is nice to have a somewhat newer version, even though it is not the latest and some bugs were fixed. I have also looked at making atheme reproducible but was surprised at the hostility of the upstream. In the end, it looks like they are still interested in patches, but they will be a little harder to deploy than for Charybdis, so this could take some time. Hopefully I will find time in the coming weeks to test the new atheme services daemon on the IRC network I operate.

Syncthing I have also re-discovered Syncthing, a file synchronization software. Amazingly, I was having trouble transferring a single file between two phones. I couldn't use Bluetooth (not sure why), the "Wifi sharing" app was available only on one phone (and is proprietary, and has a limit of 4MB files), and everything else requires an account, the cloud, or cabling. So. Just heading to f-droid, install syncthing, flash a few qr-codes around and voil : files are copied over! Pretty amazing: the files were actually copied over the local network, using IPv6 link-local addresses, encryption and the DHT. Which is a real geeky way of saying it's completely fast, secure and fast. Now, I found a few usability issues, so much that I wrote a whole usability story for the developers, which were really appreciative of my review. Some of the issues were already fixed, others were pretty minor. Syncthing has a great community, and it seems like a great project I encourage everyone to get familiar with.

Battery stats The battery-status project I mentionned previously has been merged with the battery-stats project (yes, the names are almost the same, which is confusing) and so I had to do some work to fix my Python graph script, which was accepted upstream and will be part of Debian officially from now on, which is cool. The previous package was unofficial. I have also noticed that my battery has a significantly than when I wrote the script. Whereas it was basically full back then, it seems now it has lost almost 15% of its capacity in about 6 months. According to the calculations of the script:

this battery will reach end of life (5%) in 935 days, 19:07:58.336480, on 2018-10-23 12:06:07.270290

Which is, to be fair, a good life: it will somewhat work for about three more years.

Playlist, git-annex and MPD in Python On top of my previously mentioned photos-import script, I have worked on two more small programs. One is called get-playlist and is an extension to git-annex to easily copy to the local git-annex repository all files present in a given M3U playlist. This is useful for me because my phone cannot possibly fit my whole MP3 collection, and I use playlists in GMPC to tag certain files, particularly the Favorites list which is populated by the "star" button in the UI. I had a lot of fun writing this script. I started using elpy as an IDE in Emacs. (Notice how Emacs got a new webpage, which is a huge improvement was had been basically unchanged since the original version, now almost 20 years old, and probably written by RMS himself.) I wonder how I managed to stay away from Elpy for so long, as it glues together key components of Emacs in an elegant and functional bundle:

• Company: the "vim-like" completion mode i had been waiting for forever
• Jedi: context-sensitive autocompletion for Python
• Flymake: to outline style and syntax errors (unfortunately not the more modern Flycheck)
• inline documentation...

In short, it's amazing and makes everything so much easier to work with that I wrote another script. The first program wouldn't work very well because some songs in the playlists had been moved, so I made another program, this time to repair playlists which refer to missing files. The script is simply called fix-playlists, and can operate transparently on multiple playlists. It has a bunch of heuristics to find files and uses a MPD server as a directory to search into. It can edit files in place or just act as a filter.

Useful snippets Writing so many scripts, so often, I figured I needed to stop wasting time always writing the same boilerplate stuff on top of every file, so I started publishing Yasnippet-compatible file snippets, in my snippets repository. For example, this report is based on the humble lts snippet. I also have a base license snippet which slaps the AGPLv3 license on top of a Python file. But the most interesting snippet, for me, is this simple script snippet which is a basic scaffolding for a commandline script that includes argument processing, logging and filtering of files, something which I was always copy-pasting around.

Other projects And finally, a list of interesting issues en vrac:

• there's a new Bootstrap 4 theme for Ikiwiki. It was delivering content over CDNs, which is bad for privacy issues, so I filed an issue which was almost immediately fixed by the author!
• I found out about BitHub, a tool to make payments with Bitcoins for patches submitted on OWS projects. It wasn't clear to me where to find the demo, but when I did, I quickly filed a PR to fix the documentation. Given the number of open PRs there and the lack of activity, I wonder if the model is working at all...
• a fellow Debian Developer shared his photos workflow, which was interesting to me because I have my own peculiar script, which I never mentionned here, but which I ended up sharing with the author

Julian has written about their efforts to strengthen security in APT, and shortly before that notified us that Launchpad s signatures on PPAs use weak SHA-1 digests. Unfortunately we hadn t noticed that before; GnuPG s defaults tend to result in weak digests unless carefully tweaked, which is a shame. I started on the necessary fixes for this immediately we heard of the problem, but it s taken a little while to get everything in place, and I thought I d explain why since some of the problems uncovered are interesting in their own right. Firstly, there was the relatively trivial matter of using SHA-512 digests on new signatures. This was mostly a matter of adjusting our configuration, although writing the test was a bit tricky since PyGPGME isn t as helpful as it could be. (Simpler repository implementations that call gpg from the command line should probably just add the --digest-algo SHA512 option instead of imitating this.) After getting that in place, any change to a suite in a PPA will result in it being re-signed with SHA-512, which is good as far as it goes, but we also want to re-sign PPAs that haven t been modified. Launchpad hosts more than 50000 active PPAs, though, a significant percentage of which include packages for sufficiently recent Ubuntu releases that we d want to re-sign them for this. We can t expect everyone to push new uploads, and we need to run this through at least some part of our usual publication machinery rather than just writing a hacky shell script to do the job (which would have no idea which keys to sign with, to start with); but forcing full reprocessing of all those PPAs would take a prohibitively long time, and at the moment we need to interrupt normal PPA publication to do this kind of work. I therefore had to spend some quality time working out how to make things go fast enough. The first couple of changes (1, 2) were to add options to our publisher script to let us run just the one step we need in careful mode: that is, forcibly re-run the Release file processing step even if it thinks nothing has changed, and entirely disable the other steps such as generating Packages and Sources files. Then last week I finally got around to timing things on one of our staging systems so that we could estimate how long a full run would take. It was taking a little over two seconds per archive, which meant that if we were to re-sign all published PPAs then that would take more than 33 hours! Obviously this wasn t viable; even just re-signing xenial would be prohibitively slow. The next question was where all that time was going. I thought perhaps that the actual signing might be slow for some reason, but it was taking about half a second per archive: not great, but not enough to account for most of the slowness. The main part of the delay was in fact when we committed the database transaction after processing each archive, but not in the actual PostgreSQL commit, rather in the ORM invalidate method called to prepare for a commit. Launchpad uses the excellent Storm for all of its database interactions. One property of this ORM (and possibly of others; I ll cheerfully admit to not having spent much time with other ORMs) is that it uses a WeakValueDictionary to keep track of the objects it s populated with database results. Before it commits a transaction, it iterates over all those alive objects to note that if they re used in future then information needs to be reloaded from the database first. Usually this is a very good thing: it saves us from having to think too hard about data consistency at the application layer. But in this case, one of the things we did at the start of the publisher script was: That innocuous-looking filter means that we do all the public/private filtering of PPAs up-front and return a list of all the PPAs we intend to operate on. This means that all those objects are alive as far as Storm is concerned and need to be considered for invalidation on every commit, and the time required for that stacks up when many thousands of objects are involved: this is essentially accidentally quadratic behaviour, because all archives are considered when committing changes to each archive in turn. Normally this isn t too bad because only a few hundred PPAs need to be processed in any given run; but if we re running in a mode where we re processing all PPAs rather than just ones that are pending publication, then suddenly this balloons to the point where it takes a couple of seconds. The fix is very simple, using an iterator instead so that we don t need to keep all the objects alive: After that, I turned to that half a second for signing. A good chunk of that was accounted for by the signContent method taking a fingerprint rather than a key, despite the fact that we normally already had the key in hand; this caused us to have to ask GPGME to reload the key, which requires two subprocess calls. Converting this to take a key rather than a fingerprint gets the per-archive time down to about a quarter of a second on our staging system, about eight times faster than where we started. Using this, we ve now re-signed all xenial Release files in PPAs using SHA-512 digests. On production, this took about 80 minutes to iterate over around 70000 archives, of which 1761 were modified. Most of the time appears to have been spent skipping over unmodified archives; even a few hundredths of a second per archive adds up quickly there. The remaining time comes out to around 0.4 seconds per modified archive. There s certainly still room for speeding this up a bit. We wouldn t want to do this procedure every day, but it s acceptable for occasional tasks like this. I expect that we ll similarly re-sign wily, vivid, and trusty Release files soon in the same way.

[I wrote this as part of BelFOSS but I think it s worth posting here.] My Free Software journey starts with The Cuckoo s Egg. Back in the early 90s a family friend suggested I might enjoy reading it. He was right; I was fascinated by the world of interconnected machines it introduced me to. That helped start my involvement in FidoNet, but it also got me interested in Unix. So when I saw a Linux book at the Queen s University bookshop (sadly no longer with us) with a Slackware CD in the back I had to have it. The motivation at this point was to have a low cost version of Unix I could run on the PC hardware I already owned. I had no knowledge of the GNU Project before this point, and as I wasn t a C programmer I had no interest in looking at the source code. I spent some time futzing around with it and that partition (I was dual booting with DOS 6.22) fell into disuse. It wasn t until I d learnt some C and turned up to university, which provided me with an internet connection and others who were either already using Linux or interested in doing so, that I started running a Linux box full time. Once I was doing that I became a lot more interested in the Open Source side of the equation. Rather than running a closed operating system that even the API for wasn t properly specified (or I wouldn t have needed my copy of Undocumented DOS) I had the complete source to both the underlying OS and all the utilities that it was using. For someone doing a computer science degree this was invaluable. Minix may have been the OS discussed in the OS Design module I studied, but Linux was a much more feature complete option that I was running on my desktop and could also peer under the hood of. In my professional career I ve always welcomed the opportunities to work with Open Source. A long time ago I experienced a particularly annoying issue when writing a device driver under QNX. The documentation didn t seem to match the observed behaviour of the subsystem I was interfacing with. However due to licensing issues only a small number of people in the organisation were able to actually look at the QNX source. So I ended up wasting a much more senior engineer s time with queries like I think it s actually doing x, y and z instead of a, b and c; can you confirm? . Instances where I can look directly at the source code myself make me much more productive. Commercial development also started to make me more understanding of the Free Software nature of the code I was running. It wasn t just the ability to look at the code which was useful, but also the fact there was no need to reinvent the wheel. Need a base OS to build an appliance on? Debian ensures that the main component is Free for all usage. No need to worry about rolling your own compilers, base libraries etc. From a commercial perspective that allows you to concentrate on the actual product. And when you hit problems, the source is available and you can potentially fix it yourself or at least more easily find out if there s been a fix for that issue released (being able to see code development in version control systems rather than getting a new upstream release with a whole heap on unrelated fixes in it really helps with that). I had thus progressed from using FLOSS because it was free-as-in-beer, to appreciating the benefits of Open Source in my own learning and employment experiences, to a deeper understanding of the free-as-in-speech benefits that could be gained. However at this point I was still thinking very much from a developer mindset. Even my thoughts about how users can benefit from Free Software were in the context of businesses being able to easily switch suppliers or continue to maintain legacy software because they had the source to their systems available. One of the major factors that has helped me to see beyond this is the expansion of the Internet of Things (IoT). With desktop or server software there is by and large a choice about what to use. This is not the case with appliances. While manufacturers will often produce a few revisions of software for their devices, usually eventually there is a newer and shiny model and the old one is abandoned. This is problematic for many reasons. For example, historically TVs have been long lived devices (I had one I bought second hand that happily lasted me 7+ years). However the smart capabilities of the TV I purchased in 2012 are already of limited usefulness, and LG have moved on to their current models. I have no intention of replacing the device any time soon, so have had to accept it is largely acting as a dumb display. More serious is the lack of security updates. For a TV that doesn t require a network connection to function this is not as important, but the IoT is a trickier proposition. For example Matthew Garrett had an awful experience with some intelligent light bulbs, which effectively circumvented any home network security you might have set up. The manufacturer s defence? No longer manufactured or supported. It s cases like these that have slowly led me to a more complete understanding of the freedom that Free Software truly offers to users. It s not just about cost free/low cost software. It s not just about being able to learn from looking at the source to the programs you are running. It s not even about the freedom to be able to modify the programs that we use. It s about giving users true Freedom to use and modify their devices as they see fit. From this viewpoint it is much easier to understand the protections against Tivoization that were introduced with GPLv3, and better appreciate the argument sometimes made that the GPL offers more freedom than BSD style licenses.

Some of you (but though probably not many) will know that I've suffered from depression at times. It has made it very difficult for me to do my job, to keep my home in order, even to have real conversations. Many important tasks seemed to require more energy than I had, and every setback left me feeling hopeless. I have sought treatment on several occasions, with very limited results. Certainly I would get better eventually, but it was hard to tell whether this had anything to do with the medication or therapy I received. And even while I felt better, I still didn't feel particularly happy or hopeful except when going to my favourite social events like DebConf. As I was recovering from the 'flu a weeks ago, I realised that I was feeling enthusiastic and energetic for no particular reason. Obviously I was recovering from physical illness, but that was brief. I had also just started a new project at work, but it isn't that exciting (at this stage) and I'm already aware of plenty of problems with it. So nothing external seemed to have changed to make me feel better, and yet my mood was so high that I worried that my illness could have changed to bipolar disorder and that I was entering a manic phase.

Feeling better mentally than I have in at least 6 months. Unfortunately my head's been buzzing with thoughts that kept me awake all night. Ben Hutchings (@benhutchingsuk) 29 February 2016

Thankfully, I'm not behaving manically. I have been doing a lot more shopping than usual, but almost the first purchase was a whiteboard to maintain the to-do list that I had been keeping rather unreliably in my head, and mostly I've been buying things off that list - that I've intended to get for months or years. It's just been so long since I could feel generally positive and optimistic about my life - outside of any special event - that I didn't immediately recognise this state of mind as normality.

@benhutchingsuk Did I say 6 months? I think it's more like a decade. There's a pretty deep pile of stuff I'm finally dealing with... Ben Hutchings (@benhutchingsuk) 7 March 2016

It's now been two three weeks since my recovery, and I still feel so much better:

• I'm bothered far less by discomforts like feeling a little cold or hungry. I used to become almost paralysed by this, while feeling too short of energy to fix myself some food.
• Pleasure is more intense. Food tastes and smells wonderful. Music feels more immersive. Sex made me cry.
• I can easily have normal conversations with people I know and people I don't. I'm having important conversations I had been putting off.
• I'm also able to write personal things like this, that I've shied away from for years.
• When I see minor problems with the flat, I either fix them straight away or put them on the whiteboard to be fixed soon.
• I'm far more persistent. If I run into an obstacle I'll either try a different approach or add an item to the whiteboard for whatever tools or supplies I'm missing.

As I feel better in myself I also find myself less self-centred and more able to focus on other people's needs, particularly my wife Nattie. Frankly I've been an asshole pretty often while I was depressed, so I apologise to those who've been the brunt of that. I hope you'll find me more pleasant to deal with from now on. There's lots more I could write, but I'll end this entry now and maybe say more at a later date.

Back in 2013 I proposed a way to make paper and PDF invoices easier to process electronically by adding a QR code with the key information about the invoice. I suggested using vCard field definition, to get some standard format for name and address, but any format would work. I did not do anything about the proposal, but hoped someone one day would make something like it. It would make it possible to efficiently send machine readable invoices directly between seller and buyer. This was the background when I came across a proposal and specification from the web based accounting and invoicing supplier Visma in Sweden called UsingQR. Their PDF invoices contain a QR code with the key information of the invoice in JSON format. This is the typical content of a QR code following the UsingQR specification (based on a real world example, some numbers replaced to get a more bogus entry). I've reformatted the JSON to make it easier to read. Normally this is all on one long line:

 
 "vh":500.00,
 "vm":0,
 "vl":0,
 "uqr":1,
 "tp":1,
 "nme":"Din Leverand r",
 "cc":"NO",
 "cid":"997912345 MVA",
 "iref":"12300001",
 "idt":"20151022",
 "ddt":"20151105",
 "due":2500.0000,
 "cur":"NOK",
 "pt":"BBAN",
 "acc":"17202612345",
 "bc":"BIENNOK1",
 "adr":"0313 OSLO"
 

The interpretation of the fields can be found in the format specification (revision 2 from june 2014). The format seem to have most of the information needed to handle accounting and payment of invoices, at least the fields I have needed so far here in Norway. Unfortunately, the site and document do not mention anything about the patent, trademark and copyright status of the format and the specification. Because of this, I asked the people behind it back in November to clarify. Ann-Christine Savlid (ann-christine.savlid (at) visma.com) replied that Visma had not applied for patent or trademark protection for this format, and that there were no copyright based usage limitations for the format. I urged her to make sure this was explicitly written on the web pages and in the specification, but unfortunately this has not happened yet. So I guess if there is submarine patents, hidden trademarks or a will to sue for copyright infringements, those starting to use the UsingQR format might be at risk, but if this happen there is some legal defense in the fact that the people behind the format claimed it was safe to do so. At least with patents, there is always a chance of getting sued... I also asked if they planned to maintain the format in an independent standard organization to give others more confidence that they would participate in the standardization process on equal terms with Visma, but they had no immediate plans for this. Their plan was to work with banks to try to get more users of the format, and evaluate the way forward if the format proved to be popular. I hope they conclude that using an open standard organisation like IETF is the correct place to maintain such specification. Update 2016-03-20: Via Twitter I became aware of some comments about this blog post that had several useful links and references to similar systems. In the Czech republic, the Czech Banking Association standard #26, with short name SPAYD, uses QR codes with payment information. More information is available from the Wikipedia page on Short Payment Descriptor. And in Germany, there is a system named BezahlCode, (specification v1.8 2013-12-05 available as PDF), which uses QR codes with URL-like formatting using "bank:" as the URI schema/protocol to provide the payment information. There is also the ZUGFeRD file format that perhaps could be transfered using QR codes, but I am not sure if it is done already. Last, in Bolivia there are reports that tax information since november 2014 need to be printed in QR format on invoices. I have not been able to track down a specification for this format, because of my limited language skill sets.

I own a motherboard an Intel 8-Series Lynx Point chipset, with an Intel Haswell CPU supporting VT-d. This allow me to use Linux s VFIO features and assign PCIe devices to a KVM-based virtual machine. High-end network controllers goes even further with the Single Root I/O Virtualization (SR-IOV) capabilities, allowing them to be shared between to multiple virtual machines. The Lynx Point chipset provides a total of 14 USB ports arranged in 6 USB 3.0 ports and 8 USB 2.0 ports. It would be nice to be able to assign USB ports to virtual machines. QEMU already allows to assign a USB device to a virtual machine, but it works emulating a USB controller, and the traffic goes through userland. In addition it only works for a specific known device, a random device plugged to a given port is not automatically assigned to the guest (though I guess it can be scripted using the libvirt API). The xHCI specification, the one behind USB 3.0, has been designed to also support SR-IOV, to the best of my knowledege none of them actually support it. We ll see that with some hacks it is possible to actually assign a set of USB ports to a virtual machine, with the restrictions that running ports in SuperSpeed mode is allowed only on one side, host or virtual machine. First let s look at how the USB controllers appears on a Lynx Point chipset using lscpi:
00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 04)
00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 04)
00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 04)
As one can see, three controllers are visible, one xHCI one and two EHCI ones. Let s now look at how the USB ports are arranged using lsusb -t
/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/3p, 480M
__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/3p, 480M
__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/15p, 480M explain EHCI/OHCI/XHCI http://www.intel.com/content/www/us/en/chipsets/8-series-chipset-pch-datasheet.html the kernel in the VM might move back the devices to the xHCI controller. This is always the case for old kernels (like the 3.2 in Debian Wheezy), but for recent kernel it only happens if there is an intel EHCI controller available (either passed through VFIO or emulated by QEMU). add table Add warning

A new version of x13binary is now on CRAN. It updates the underlying X-13ARIMA-SEATS binary to build 1.1.26 which was released recently by the US Census Bureau. Our x13binary package takes the pain out of installing this, and provides it to the seasonal package upon which other packages such as gunsales or ggseas build their functionality. Things just work: Depend on x13binary and on all relevant OSs supported by R, you should have an X-13ARIMA-SEATS binary installed which will be called seamlessly by the higher-level packages. See this announcement blog post describing the initial 0.1.0 release; we had since followed up with a 0.1.1 release tightening up behaviour on two edge case OSs. Courtesy of CRANberries, there is also a diffstat report for the most recent release as well as the preceding bugfix release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

The situation where Agon (the designated organizer of the Chess World Championship, and also the Candidates tournament, the prequalifier to said WC) is trying to claim exclusive rights of the broadcasting of the moves (not just the video) is turning bizarre. First of all, they have readily acknowledged they have no basis in copyright to do so; chess moves, once played, are facts and cannot be limited. They try to jump through some hoops with a New York-specific doctrine (even though the Candidates, unlike the World Championship, is played in Moscow) about hot news , but their main weapon seems to be that they simply will throw out anyone from the hall who tries to report on the moves, and then try to give them only to those that promise not to give them on. This leads to the previously unheard-of situation where you need to register and accept their terms just to get to watch the games in your browser. You have to wonder what they will be doing about the World Championship, which is broadcast unencrypted on Norwegian television (previous editions also with no geoblock). Needless to say, this wasn't practically possible to hold together. All the big sites (like Chessdom, ChessBomb and Chess24) had coverage as if nothing had happened. Move sourcing is a bit of a murky business where nobody really wants to say where they get the moves from (although it's pretty clear that for many tournaments, the tournament organizers will simply come to one or more of the big players with an URL they can poll at will, containing the games in the standard PGN format), and this was no exception ChessBomb went to the unusual move of asking their viewers to download Tor and crowdsource the moves, while Chessdom and Chess24 appeared to do no such thing. In fact, unlike Chessdom and ChessBomb, Chess24 didn't seem to say a thing about the controversy, possibly because they now found themselves on the other side of the fence from Norway Chess 2015, where they themselves had exclusive rights to the PGN in a similar controversy although it would seem from a tweet that they were perfectly okay with people just re-broadcasting from their site if they paid for a (quite expensive) premium membership, and didn't come up with any similar legal acrobatics to try to scare other sites. However, their ToS were less clear on the issue, and they didn't respond to requests for clarification at the time, so I guess all of this just continues to be on some sort of gentleman's agreement among the bigger players. (ChessBomb also provides PGNs for premium members for the tournaments they serve, but they expressly prohibit rebroadcast. They claim that for the tournaments they host, which is a small minority, they provide free PGNs for all.) Agon, predictably, sent out angry letters where they threatened to sue the sites in question, although it's not clear at all to me what exactly they would sue for. Nobody seemed to care, except one entity TWIC, which normally has live PGNs from most tournaments, announced they would not be broadcasting from the Candidats tournament. This isn't that unexpected, as TWIC (which is pretty much a one-man project anyway) mainly is about archival, where they publish weekly dumps of all top-level games played that week. This didn't affect a lot of sites, though, as TWIC's live PGNs are often not what you'd want to base a top-caliber site on (they usually lack clock information, and moves are often delayed by half a minute or so). I run a hobby chess relay/analysis site myself (mainly focusing on the games of Magnus Carlsen), though, so I've used TWIC a fair bit in the past, and if I were to cover the Candidates tournament (I don't plan to do so, given Agon's behavior, although I plan to cover the World Championship itself), I might have been hit by this. So, that was the background. The strange part started when worldchess.com, Agon's broadcasting site, promptly went down during the first round of the Candidates tournament today Agon blamed DDoS, which I'm sure is true, but it's unclear exactly how strong the DDoS was, and if they did anything at all to deal with it other than to simply wait it out. But this lead to the crazy situation where the self-declared monopolist was the only big player not broadcasting the tournament in some form. And now, in the trully bizarre move, World Chess is publishing a detailed rebuttal of Agon's arguments, explaining how it is bad for chess, not juridically sound, and also morally wrong. Yes, you read that right; Agon's broadcast site is carrying an op-ed saying Agon is wrong. You at least have to give them credit for not trying to censor their columinst when he says something they don't agree with. Oh, and if you want those PGNs? I will, at least for the time being, be pushing them out live on http://pgn.sesse.net/. I have not gone into any agreement with Agon, and they're hosted in Norway, far from any New York-specific doctrines. So feel free to relay from them, although I would of course be happy to know if you do.

February 2016 has been a very active month regarding me contributing to the FLOSS world.

• Finalizing MATE uploads to Debian with regards to the Beta 1 Freeze of Ubuntu 16.04
• Work on RDP related packages in Debian
• Work on Debian Edu related packages
• Work on Debian LTS
• Work on nx-libs (NX v3)

Honouring my Sponsors I am happy to share that this month's FLOSS work has been sponsored by various sponsors.

• Work on the packages mate-dock-applet, topmenu-gtk and getting GIR-support back into libwnck has been sponsored by Martin Wimpress, the main driving force (AFAIK) behind Ubuntu MATE [1].
• Work on Debian LTS has been sponsored by the various Debian LTS sponsors proxied through Freexian SARL (thanks to Raphael Hertzog for organizing the paid LTS contributors team via his company) [2].
• Work on nx-libs has been sponsored by the Qindel Group [3].

Thanks to all people and companies sponsoring my work on FLOSS projects. This month's MATE uploads to Debian With regards to the Beta 1 Freeze date of Ubuntu 16.04 LTS (18th Feb 2016), Martin Wimpress, Vangelis Mouhtsis and I performed quite some work on Debian MATE. Uploads to Debian unstable:

• caja 1.12.4-2 [4]
• mate-menu 5.6.8-1 [5]
• mate-panel 1.12.2-1 [6]
• mate-dock-applet 0.67-3 (NEW) [7]
• mate-polkit 1.12.0-3 [8]
• eom 1.12.2-1 [9]
• pluma 1.12.2-1 [10]
• topmenu-gtk 0.2.1+git20151210.8c6108f-2 (NEW) [11]
• mate-tweak 3.5.7-1 (latest upload on 2nd of March) [12]

The Debian MATE Packaging Team also took over maintenance of the GTK-2+ legacy package libwnck [13]. The first upload introducing some major changes and package clean-ups caused a slight wave [14] because of a missing dependency in libwnck-dev (that fell victim to some clean-ups in debian/control). Those issues have been addressed immediately and have now been settled. The main reason for working on a legacy package like libwnck was the need for having gir1.2-wnck-1.0 (back) in Debian. The new MATE dock applet requires the libwnck GIR package to be present at runtime. One of the novelties in Ubuntu MATE 16.04 LTS will be the option to adapt the look and feel of the MATE desktop to how a Unity-based desktop looks like. Martin Wimpress is giving intense work to providing a dock applet and topmenu support as one alternative among the various Ubuntu MATE desktop experiences provided. The alternative desktop layouts can be configured with the MATE Tweak tool. Work on RDP related packages Work on FreeRDP 1.1 as currently in Debian I finally managed to give some priority (and thus time) to fixing various issues in the freerdp package in Debian [15]. Many people had provided patches and solutions to open issues and I tried to honour as many of those, as possible. Please note that I had to disable the GStreamer support in FreeRDP for the recent uploads, as the currently used Git snapshot of FreeRDP only supports GStreamer 0.10's API whereas the security team is in the process of having gstreamer0.10-* packages removed from the Debian stretch/unstable archives. Work on FreeRDP 2.0, coming to Debian soon Furthermore, Bernhard Miklautz is currently working on a freerdp2 package, which will bring the latest Git snapshot of FreeRDP upstream into Debian (and also re-introduce GStreamer support, based on GStreamer 1.0). Bernhard invested a lot of time on pushing the current HEAD of FreeRDP upstream [16] towards a FreeRDP 2.x version. Starting with FreeRDP 2.x it will be possible to install different FreeRDP versions on one system without file naming conflicts. For March 2016, I have doing the final freerdp2 reviewing on my todo list (possibly together with H ctor Or n Mart nez who is highly interested in the RDP backend support in Wayland/Weston), so that we can provide first uploads to Debian experimental sometime the coming month. The packaging progress is continuously discussed on the #freerdp channel on Freenode and can also be viewed on Github [17]. Review of revised XRDP package Recently, Dominik George from Teckids e.V. [18] contacted me about reviewing their effort of updating the Debian package xrdp, which currently is in ITA state [19]. Feedback has been provided and I am waiting for a ping from his side so that I can take some (ideally) final looks at the package and sponsor the upload. Work on Debian Edu related packages This month, I spent a couple of hours of work on several Debian Edu related tasks, some of them induced by problems at local school sites we support.

• Fix the gosa package in Debian unstable [20]
• Document a switch-over to systemd-networkd for Debian Edu jessie notebooks [21]
• Help out in fixing outstanding issues in ldap2bind [22]
• Upload a new shutdown-at-night package version to Debian unstable [23]
• Fix some icon issues in the italc package [24]
• Take a first glance at packaging terminal-quest [25]
• Playing with the Lego Mindstorm (NXT) tools

Work on Debian LTS My 8h-portion of work for the Debian LTS Project, I performed at the very end of February. With the Debian squeeze LTS EOL date on 29th February, I saw to finalizing my personal open todos regarding Debian squeeze LTS, which basically was getting two CVE issues fixed in the lxc package [26]. The rest of the work hours has been spent on helping out the Security Team of Debian with open CVE issues in Debian wheezy packages:

• Providing a .debdiff for gosa in Debian wheezy (addressing CVE-2015-8771 and CVE-2014-9760)
• Investigating an open issue in smarty3 (CVE-2014-8350) and providing a recommendation on how to handle that (which is: pick the latest smarty3 3.1.21-1 from Debian jessie and provide that to users of Debian wheezy)
• Taking a first look at pdns regarding a fix for CVE-2014-7210 in Debian wheezy

The gosa .debdiff has been approved by a member of the Security Team, the upload will happen today. With my LTS frontdesk hat on (during week 9 / 2016) I also spent some time providing help regarding SVN checkout problems and raised a couple of questions on how to coordinate the work phase between the Debian squeeze LTS EOL and the official launch of the Debian wheezy LTS project phase [27]. Work on nx-libs At the end of February, I finally managed to propose a way of dropping the libNX_Xrender.so bundled library from the nx-libs code base. I filed a PR [28] against nx-libs that proposes its removal and provides a patch for using X.Org's libXrender.so version. As a preview for nx-libs work in March 2016... I have started with removing the complete libNX_X11.so library from nx-libs and using X.Org's X11 client library. This will introduce a code removal of around 160.000 lines of code to nx-libs. More to come on this later... light+love,
Mike [1] http://ubuntu-mate.org/
[2] https://www.freexian.com/
[3] http://www.qindel.com/ [4] (caja)
https://lists.debian.org/debian-devel-changes/2016/02/msg00468.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02080.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02086.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02183.html [5] (mate-menu)
https://lists.debian.org/debian-devel-changes/2016/02/msg00469.html [6] (mate-panel)
https://lists.debian.org/debian-devel-changes/2016/02/msg01900.html [7] (mate-dock-applet)
https://lists.debian.org/debian-devel-changes/2016/02/msg01935.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02481.html
https://lists.debian.org/debian-devel-changes/2016/02/msg03097.html [8] (mate-polkit)
https://lists.debian.org/debian-devel-changes/2016/02/msg01936.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02395.html [9] (eom)
https://lists.debian.org/debian-devel-changes/2016/02/msg02073.html [10] (pluma)
https://lists.debian.org/debian-devel-changes/2016/02/msg02128.html [11] (topmenu-gtk)
https://lists.debian.org/debian-devel-changes/2016/02/msg02399.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02501.html [12] (mate-tweak)
https://lists.debian.org/debian-devel-changes/2016/02/msg03086.html [13] (libwnck)
https://lists.debian.org/debian-devel-changes/2016/02/msg01248.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01404.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01825.html [14] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814585
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814588
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814697 [15] (freerdp)
https://lists.debian.org/debian-devel-changes/2016/02/msg02487.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02630.html [16] https://github.com/FreeRDP/FreeRDP
[17] https://github.com/bmiklautz/debian-freerdp2 [18] https://www.teckids.org/ [19] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719624 [20] (gosa)
https://lists.debian.org/debian-devel-changes/2016/02/msg01554.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01954.html [21] https://sunweavers.net/blog/node/34 [22] (ldap2zone)
https://lists.debian.org/debian-devel-changes/2016/02/msg01966.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01967.html [23] (shutdown-at-night)
https://lists.debian.org/debian-devel-changes/2016/02/msg03605.html [24] (italc)
https://lists.debian.org/debian-devel-changes/2016/02/msg01944.html [25] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815948 [26] (lxc, Debian squeeze LTS)
https://lists.debian.org/debian-lts-changes/2016/02/msg00037.html [27] https://lists.debian.org/debian-lts/2016/02/msg00155.html
(The thread continues in March 2016) [28] https://github.com/ArcticaProject/nx-libs/pull/93

As most distributions have switched to systemd as default "init", some people have asked why we actually keep sysvinit around, as it's old and crusty, and systemd can do so much more. My answer is: systemd and sysvinit solve entirely different problems, and neither can ever replace the other fully. The big argument in favour of systemd is integration: when you keep everything in one process, it is easy to make sure that components talk to each other. That is nice, but kind of misses the point: integration could be done via IPC as well, and systemd uses a lot of it to separate privileges anyway. The tighter integration comes from something else: the configuration format. Where sysvinit uses a lot of shell scripts ("imperative" format), systemd uses explicit keywords ("descriptive" format), and this is the important design decision here: descriptive formats cannot do anything unanticipated, neither good nor bad. This is important for UI programmers: if there is a button that says "if I close the lid switch, suspend the laptop", then this needs to be connected to functionality that behaves this way, and this connection needs to work in both directions, so the user will find the active setting there when opening the dialog. So, we need to limit ourselves to a closed set of choices, so the UI people can prepare translations. This is the design tradeoff systemd makes: better integration in exchange for less flexibility. Of course, it is simple to allow "custom" actions everywhere, but that starts to break the "well-integrated" behaviour. There is an old blog post from Joey Hess about how to implement a simple alarm clock, using systemd, and this shows both how powerful and how limited the system is. On one hand, the single line WakeSystem=true takes care of the entire dependency chain attached to making sure the system is actually on -- when the system is going to suspend mode, some program needs to determine that there are scheduled jobs that should wake up the system, determined which of these is the next one, program the wakeup time into the hardware and only then allow the suspend to continue. On the other hand, the descriptive framework already breaks down in this simple example, because the default behaviour of the system is to go to sleep if the lid switch is closed, and the easiest way to fix this is to tell systemd to ignore the lid switch while the program is running. The IPC permissions disallow scheduled user jobs from setting this, so a "custom" action is taken that sets up the "ignore the lid switch" policy, changes to an unprivileged user, and runs the actual job. So, it is possible to get back flexibility, but this is traded for integration: while the job is running, any functionality that the "suspend laptop when the lid is closed" button had, is broken. For something simple as a switch on a personal laptop that breaks if the user configures something that is non-standard, that is acceptable. However, the keywords in the service unit files, the timer files, etc. are also part of a descriptive framework, and the services using them expect that the things they promise are kept by the framework, so there is a limit to how many exceptions you can grant. SystemV init really comes from the other direction here. Nothing is promised to init scripts but a vague notion that they will be run with sufficient privileges. There is no dependency tracking that ensures that certain services will always start before others, just priorities. You can add a dependency system like insserv if you like and make sure that all the requirements for it to work (i.e. services declaring their dependencies) are given, and there is a safe fallback of not parallelizing so we can always play it safe. Because there is so little promised to scripts, writing them becomes a bit tedious -- all of them have a large case statement where they parse the command line, all of them need to reimplement dropping privileges and finding the PID of a running process. There are lots of helpers for the more common things, like the LSB standard functions for colorful and matching console output, and Debian's start-stop-daemon for privilege and PID file handling, but in the end it remains each script's responsibility. Systemd will never be able to provide the same flexibility to admins while at the same time keeping the promises made in the descriptive language, and sysvinit will never reach the same amount of easy integration. I think it is fairly safe to predict both of these things, and I'd even go a step further: if any one of them tried, I'd ask the project managers what they were thinking. The only way to keep systems as complex as these running is to limit the scope of the project. With systemd, the complexity is kept in the internals, and it is important for manageability that all possible interactions are well-defined. That is a huge issue when writing adventure games, where you need to define interaction rules for each player action in combination with each object, and each pair of objects that can be combined with the "Use" action. Where adventure games would default to "This doesn't seem to work." or "I cannot use this.", this would not be a helpful error message when we're trying to boot, so we really need to cut down on actions and object types here. Sysvinit, on the other hand, is both more extreme in limiting the scope to very few actions (restart, wait, once) and only one object type (process that can be started and may occasionally terminate) in the main program, and a lot more open in what scripts outside the main program are allowed to do. The prime example for this is that the majority of configuration takes place outside the inittab even -- a script is responsible for the interface that services are registered by dropping init scripts into specific directories, and scripts can also create additional interfaces, but none of this is intrinsic to the init system itself. Which system is the right choice for your system is just that: your choice. You would choose a starting point, and customize what is necessary for your computer to do what you want it to do. Here's the thing: most users will be entirely happy with fully uncustomized systemd. It will suspend your laptop if you close the lid, and even give your download manager veto power. I fully support Debian's decision to use systemd as the default init for new installs, because it makes sense to use the default that is good for the vast majority of users. However, my opening statement still stands: systemd can never support all use cases, because it is constrained by its design. Trying to change that would turn it into an unmaintainable mess, and the authors are right in rejecting this. There are use cases that define the project, and use cases that are out of scope. This is what we need sysvinit for: the bits that the systemd project managers rightfully reject, but that are someone's use case. Telling these users to get lost would be extremely bad style for the "universal operating system." For example, if someone needs a service that asks a database server for a list of virtual machines to start, runs each in its private network namespace that is named after replacing part of the virtual machine name with the last two octets of the host machine's IP address, then binds a virtual Ethernet device into the network namespace and sets up a NAT rule that allows devices to talk to the public Internet and a firewall rule to stop VMs from talking to each other. Such a beast would live outside of systemd's world view. You can easily start it, but systemd would not know how to monitor it (as long as there is some process still running, is that a good sign), not know how to shut down one instance, not know how to map processes to instances and so on. SystemV init has the advantage here that none of these things are defined by the system, and as long as my init script has some code to handle "status", I can easily check up on my processes with the same interface I'd use for everything else. This requires me to have more knowledge of how the init system works, that is true, however I'd also venture that the learning curve for sysvinit is shallower. If you know shell scripting, you can understand what init scripts do, without memorizing special rules that govern how certain keywords interact, and keeping track of changes to these rules as the feature set is expanded. That said, I'm looking at becoming the new maintainer of the sysvinit package so we can continue to ship it and give our users a choice, but I'd definitely need help, as I only have a few systems left that won't run properly with systemd because of its nasty habit to keep lots of code in memory where sysvinit would unload it after use (by letting the shell process end), and these don't make good test systems. The overall plan is to set up a CI system that

• tests sysvinit -> systemd transition
• tests systemd -> sysvinit transition
• tests debootstrap with systemd
• tests debootstrap with sysvinit
• tests whether service start works for a list of services, so we can catch regressions
• scans packages that have systemd unit files for init scripts, and vice versa

Ideally, I'd like to reach a state where we can actually ensure that the default configuration is somewhat similar regardless of which init system we use, and that it is possible to install with either (this is a nasty regression in jessie: bootstrapping a foreign-architecture system broke). This will take time, and step one is probably looking at bugs first -- I'm grateful for any help I can get here, primarily I'd like to hear about regressions via bug reports. tl;dr: as a distribution, we need both systemd and sysvinit. If you suggest that systemd can replace sysvinit, I will probably understand that you think that the systemd maintainers don't know project management. I'm reluctantly looking at becoming the official maintainer because I think it is important to keep around.

Categories:

• Debian-Blog
• Computing
• Technology
• Tools

Keywords:

• power saving
• laptop-mode-tools
• laptop
• battery

Like:

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

• Various improvements to django-slack, a library to easily post messages to the Slack group-messaging utility from projects using the Django web development framework:
  • Added explicit support to send messages asynchronously via the Celery distributed task queue. (#29)
  • Worked with Patrick Clope to add an escapeslack template filter to ensure the correct characters for Slack's API are escaped using Django's built-in safe is too invasive. (#36)
  • Corrected an issue where custom endpoints and channel names were incompatible. (#27)
  • Overhaul of the field/option/block parsing. (49ff3c4)
  • Moved away from using a django.template.Context instance, preventing a deprecation warning. (#31)
• Added a create-folders subcommand to my tickle-me-email Getting Things Done (GTD) email toolbox to create numbered folders for the rotate "tickler" functionality. (#3)
• Wrote and released a Django template tag to collapse multiple whitespace/newline characters in order to correctly format, for example, plaintext emails that make extensive use of for-loops. These would normally require careful and fragile placement of the % for .. % and % endfor % tags. (Repo)
• Pushed a number of updates to my Strava Enhancement Suite, a Chrome extension that improves and fixes annoyances in the web interface of the Strava cycling and running tracker:
  • Fix unit conversion where the element was wrapped in an another HTML element. (#48)
  • Hide premium indicator from top-level navigation. (#45)
  • Correct matching of the "shop" link in top-level navigation and footer module. (#47 & #46)
  • Remove "premium" pages on Club pages. (#44)
• Added a security-oriented warning to Ansible's documentation regarding the behaviour if an UFW firewall application profile is added and subsequently removed. (#1740)
• Added support to my django-staticfiles-dotd "staticfiles" library which concatenates Javascript and CSS files from .d-style directories to support an arbitrary file rendering method. This allows the use of media pre-processors such as SASS within such directories. (#1)
• Corrected my Chrome extension for the FastMail web interface to correctly hook into the internal "send" mechanism. (#2)
• Moved my stravabot IRC bot to use dh-virtualenv and systemd, improving security and reliability. (1c48709)
• Updated django-pedantic-http-methods a tool to raise an exception during development when attempting to perform side effects in GET and HEAD HTTP methods to support the latest version of Django. (#1)

---

This is a bit of an odd posting since it's about something I've done but is also here to help me explain why I did it and thus perhaps encourage some discussion around the topic within the Kicad community...

Recently (as you will know if you follow this blog anywhere it is syndicated) I have started playing with Kicad for the development of some hardware projects I've had a desire for. In addition, some of you may be aware that I used to work for a hardware/software consultancy called Simtec, and there I got to play for a while with an EDA tool called Mentor Designview. Mentor was an expensive, slow, clunky, old-school EDA tool, but I grew to understand and like the workflow. I spent time looking at gEDA and Eagle when I wanted to get back into hardware hacking for my own ends; but neither did I really click with. On the other hand, a mere 10 minutes with Kicad and I knew I had found the tool I wanted to work with long-term. I designed the beer'o'meter project (a flow meter for the pub we are somehow intimately involved with) and then started on my first personal surface-mount project -- SamDAC which is a DAC designed to work with our HiFi in our study at home. As I worked on the SamDAC project, I realised that I was missing a very particular thing from Mentor, something which I had low-level been annoyed by while looking at other EDA tools -- Kicad lacks a mechanism to mark a wire as being linked to somewhere else on the same sheet. Almost all of the EDA tools I've looked at seem to lack this nicety, and honestly I miss it greatly, so I figured it was time to see if I could successfully hack on Kicad. Kicad is written in C++, and it has been mumble mumble years since I last did any C++, either for personal hacking or professionally, so it took a little while for that part of my brain to kick back in enough for me to grok the codebase. Kicad is not a small project, taking around ten minutes to build on my not-inconsiderable computer. And while it beavered away building, I spent time looking around the source code, particularly the schematic editor eeschema. To skip ahead a bit, after a couple of days of hacking around, I had a proof-of-concept for the intra-sheet links which I had been missing from my days with Mentor, and some ERC (electrical rules checking) to go alongside that to help produce schematics without unwanted "sharp corners". In total, I added:

• an "intra-sheet link" schematic element, including:
  • the schematic element itself
  • loading and saving of the element (which required a bump in schematic file format versioning)
  • UI for adding intra-sheet links to your schematic
  • netlist build support to ensure that the links found their way into the internal netlist content.

• an ERC tickybox "Strict wiring checks" and behind that I hid:
  • checks for wires with dangling ends (dangles are bad)
  • junctions which have no point in being present (pointless junctions are confusing to readers)
  • checks that intra-sheet links exist in groups of two or more on any given net, since individual links are pointless.

I forked the Kicad mirror on Github and pushed my own branch with this work to my Kicad fork. All of this is meant to allow schematic capture engineers to more clearly state their intentions regarding what they are drawing. The intra-sheet link could be thought of like a no-connect element, except instead of saying "this explicitly goes nowhere" we're saying "this explicitly goes somewhere else on this sheet, you can go look for it". Obviously, people who dislike (or simply don't want to use) such intra-sheet link elements can just disable that ERC tickybox and not be bothered by them in the least (well except for the toolbar button and menu item I suppose). Whether this work gets accepted into Kicad, or festers and dies on the vine, it was good fun developing it and I'd like to illustrate how it could help you, and why I wrote it in the first place:

A contrived story

Note, while this story is meant to be taken seriously, it is somewhat contrived, the examples are likely electrical madness, but please just think about the purpose of the checks etc.

To help to illustrate the feature and why it exists, I'd like to tell you a somewhat contrived story about Fred. Fred is a schematic capture engineer and his main job is to review schematics generated by his colleagues. Fred and his colleagues work with Kicad (hurrah) but of late they've been having a few issues with being able to cleanly review schematics. Fred's colleagues are not the neatest of engineers. In particular they tend to be quite lazy when it comes to running busses, which are not (for example) address and data busses, around their designs and they tend to simply have wires which end in mid-space and pick up somewhere else on the sheet. All this is perfectly reasonable of course, and Kicad handles it with aplomb. Sadly it seems quite error prone for Fred's workplace. As an example, Fred's colleague Ben has been designing the power supply for a particular board. As with most power supplies, plenty of capacitors are needed to stabilise the regulators and smooth the output. In the example below, the intent is that all of the capacitors are on the FOO net. Sadly there's a missing junction and/or slightly misplaced label in the upper section which means that C2 and C3 simply don't join to the FOO net. This could easily be missed, but the ERC can't spot it at all since there's more than one thing on each net, so the pins of the capacitors are connected to something. Fred is very sad that this kind of problem can sometimes escape notice by the schematic designer Ben, Fred himself, and the layout engineer, resulting in boards which simply do not work. Fred takes it upon himself to request that the strict wiring checks ERC is made mandatory for all designs, and that the design engineers be required to use intra-sheet link symbols when they have signals which wander off to other parts of the sheet like FOO does in the example. Without any further schematic changes, strict wiring checks enabled gives the following points of ERC concern for Ben to think about: As you can see, the ERC is pointing at the wire ends and the warnings are simply that the wires are dangling and that this is not acceptable. This warning is very like the pin-not-connected warnings which can be silenced with an explicit no-connect schematic element. Ben, being a well behaved and gentle soul, obeys the design edicts from Fred and seeks out the intra-sheet link symbols, clearing off the ERC markers and then adding intra-sheet links to his design: This silences the dangling end ERC check, which is good, however it results in another ERC warning: This time, the warning for Ben to deal with is that the intra-sheet links are pointless. Each exists without a companion to link to because of the net name hiccough in the top section. It takes Ben a moment to realise that the mistake which has been made is that a junction is missing in the top section. He adds the junction and bingo the ERC is clean once more: Now, this might not seem like much gain for so much effort, but Ben can now be more confident that the FOO net is properly linked across his design and Fred can know, when he looks at the top part of the design, that Ben intended for the FOO net to go somewhere else on the sheet and he can look for it.

Why do this at all? Okay, dropping out of our story now, let's discuss why these ERC checks are worthwhile and why the intra-sheet link schematic element is needed.

Note: This bit is here to remind me of why I did the work, and to hopefully explain a little more about why I think it's worth adding to Kicad...

Designers are (one assumes) human beings. As humans we (and I count myself here too) are prone to mistakes. Sadly mistakes are often subtle and could easily be thought of as deliberate if the right thought processes are not followed carefully when reviewing. Anyone who has ever done code review, proofread a document, or performed any such activity, will be quite familiar with the problems which can be introduced by a syntactically and semantically valid construct which simply turns out to be wrong in the greater context. When drawing designs, I often end up with bits of wire sticking out of schematic sections which are not yet complete. Sadly if I sleep between design sessions, I often lose track of whether such a dangling wire is meant to be attached to more stuff, or is simply left because the net is picked up elsewhere on the sheet. With intra-sheet link elements available, if I had intended the latter, I'd have just dropped such an element on the end of the wire before I stopped for the day. Also, when drawing designs, I sometimes forget to label a wire, especially if it has just passed through a filter or current-limiting resistor or similar. As such, even with intra-sheet link elements to show me when I mean for a net to go bimbling off across the sheet, I can sometimes end up with unnamed nets whose capacitors end up not used for anything useful. This is where the ERC comes in. By having the ERC complain if a wire dangles -- the design engineer won't forget to add links (or check more explicitly if the wire is meant to be attached to something else). By having junctions which don't actually link anything warned about, the engineer can't just slap a junction blob down on the end of a wire to silence that warning, since that doesn't mean anything to a reviewer later down the line. By having the ERC warn if a net has exactly one intra-sheet link attached to it, missing net names and errors such as that shown in my contrived example above can be spotted and corrected. Ultimately this entire piece of work is about ensuring that the intent of the design engineer is captured clearly in the schematic. If the design engineer meant to leave that wire dangling because it's joining to another bit of wire elsewhere on the sheet, they can put the intra-sheet links in to show this. The associated ERC checks are there purely to ensure that the validation of this intent is not bypassed accidentally, or deliberately, in order to make the use of this more worthwhile and to increase the usefulness of the ERC on designs where signals jump around on sheets where wiring them up directly would just create a mess.

Categories:

• Debian-Blog
• Computing
• Technology

Keywords:

• linux
• memory
• block
• cpu
• contention

Like:

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved. On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now. Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills. All handled issues:

• #11980 Fix #11979 DECLARE not accepted as valid SQL
• #11609 Errors when Analyzer fails
• #11974 debug event in phpmyadmin for day and month (error)
• #11975 unrecognized keyword left in where clause
• #11973 Tracker won't track some queries executed from SQL tab
• #11907 Update query shows twice
• #11850 CSV import to new table broken
• #11958 Bug Fix #11907
• #11470 Impossible to cancel "Drop files here" overlay
• #6146 Syslog authentication requests / brute-force protection
• #6261 All-Languages version via composer/packagist.org
• #11947 Notice in sql.lib
• #11669 Can't edit rows if table name has one upper case char (OS X)
• #11968 Syntax-errored ALTER TABLE is exported
• #11967 Warning on loading corrupted XML
• #11959 Uncaught ReferenceError: isStorageSupported is not defined
• #22 Make INI settings configurable via environment variables

Filed under: English phpMyAdmin 2 comments

The US Government is attempting to force Apple to build a signed image that can be flashed onto an iPhone used by one of the San Bernardino shooters. To their credit, Apple have pushed back against this - there's an explanation of why doing so would be dangerous here. But what's noteworthy is that Apple are arguing that they shouldn't do this, not that they can't do this - Apple (and many other phone manufacturers) have designed their phones such that they can replace the firmware with anything they want.

In order to prevent unauthorised firmware being installed on a device, Apple (and most other vendors) verify that any firmware updates are signed with a trusted key. The FBI don't have access to Apple's firmware signing keys, and as a result they're unable to simply replace the software themselves. That's why they're asking Apple to build a new firmware image, sign it with their private key and provide it to the FBI.

But what do we mean by "unauthorised firmware"? In this case, it's "not authorised by Apple" - Apple can sign whatever they want, and your iPhone will happily accept that update. As owner of the device, there's no way for you to reconfigure it such that it will accept your updates. And, perhaps worse, there's no way to reconfigure it such that it will reject Apple's.

I've previously written about how it's possible to reconfigure a subset of Android devices so that they trust your images and nobody else's. Any attempt to update the phone using the Google-provided image will fail - instead, they must be re-signed using the keys that were installed in the device. No matter what legal mechanisms were used against them, Google would be unable to produce a signed firmware image that could be installed on the device without your consent. The mechanism I proposed is complicated and annoying, but this could be integrated into the standard vendor update process such that you simply type a password to unlock a key for re-signing.

Why's this important? Sure, in this case the government is attempting to obtain the contents of a phone that belonged to an actual terrorist. But not all cases governments bring will be as legitimate, and not all manufacturers are Apple. Governments will request that manufacturers build new firmware that allows them to monitor the behaviour of activists. They'll attempt to obtain signing keys and use them directly to build backdoors that let them obtain messages sent to journalists. They'll be able to reflash phones to plant evidence to discredit opposition politicians.

We can't rely on Apple to fight every case - if it becomes politically or financially expedient for them to do so, they may well change their policy. And we can't rely on the US government only seeking to obtain this kind of backdoor in clear-cut cases - there's a risk that these techniques will be used against innocent people. The only way for Apple (and all other phone manufacturers) to protect users is to allow users to remove Apple's validation keys and substitute their own. If Apple genuinely value user privacy over Apple's control of a device, it shouldn't be a difficult decision to make.

comments

more than 50000 binary packages in sid main amd64 So in the last weeks every morning I looked at this graph and yesterday for the first time I finally saw it exceed 50k binary packages: The graph shows the piuparts results for the unstable/sid suite since 2009. In not really related news, in our reproducible builds test setup 87% of the source packages in testing/stretch build on amd64 were reproducible today, which is the highest percentage we've seen so far. Because of some more build failures and due to some more variations added, testing/stretch on armhf is a bit behind and "only" reached a very good 83.6% so far. One of the next statistic quesions is, whether we'll reach 24k source packages in unstable first, or 23k source packages in testing And hopefully we'll reach 20k reproducible packages (in our current test framework) on testing/amd64 before that (We already reachred 20k reproducible packages on unstable/amd64.) And this is all because of you awesome people! I'm very thankful we enable each other to grow like we do and I look forward to go further!

Debugging output of GUI apps I have been hit by #643726 again. I was happily working on a terminal while an apt upgrade was running on another, and suddenly kbuildsycoca4 started vomiting on my command line because I had run okular once on that terminal last tuesday. Among the possible workarounds, one can run kdebugdialog from package kde-runtime and check the "Disable all debug output" button. This is the output of kdebugdialog when run on a terminal after enabling "Disable all debug output":

$ kdebugdialog
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.
QDBusConnection: session D-Bus connection created before QCoreApplication. Application may misbehave.

I see it as debugging output turned into an art performance, and I have an appreciation for this kind of things. kdebugdialog, my hat's off to you.