The binary code, although necessary for major social and technological developments, is annihilating the Homo Technologicus, stifling his innate freedom for analog connections.

The first widespread use of the binary code was the Morse code alphabet. For decades, this communication system allowed to transmit information over long distances, between ships in the ocean and the mainland, between one continent and another, and today its use is still active in emergency situations. In its disarming simplicity (connect or interrupt two electrical wires), this system proved to be the shortest way to transcribe an alphabet.

Then, the Boolean algebra (binary) represented the language of communication between humans and computers, enabling the translation of instructions and commands towards the processing unit (microprocessor / CPU). This determined the Big Bang of the computers and the technological era

<Read More...>

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved. On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now. Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills. All handled issues:

• #11980 Fix #11979 DECLARE not accepted as valid SQL
• #11609 Errors when Analyzer fails
• #11974 debug event in phpmyadmin for day and month (error)
• #11975 unrecognized keyword left in where clause
• #11973 Tracker won't track some queries executed from SQL tab
• #11907 Update query shows twice
• #11850 CSV import to new table broken
• #11958 Bug Fix #11907
• #11470 Impossible to cancel "Drop files here" overlay
• #6146 Syslog authentication requests / brute-force protection
• #6261 All-Languages version via composer/packagist.org
• #11947 Notice in sql.lib
• #11669 Can't edit rows if table name has one upper case char (OS X)
• #11968 Syntax-errored ALTER TABLE is exported
• #11967 Warning on loading corrupted XML
• #11959 Uncaught ReferenceError: isStorageSupported is not defined
• #22 Make INI settings configurable via environment variables

Filed under: English phpMyAdmin 2 comments

The US Government is attempting to force Apple to build a signed image that can be flashed onto an iPhone used by one of the San Bernardino shooters. To their credit, Apple have pushed back against this - there's an explanation of why doing so would be dangerous here. But what's noteworthy is that Apple are arguing that they shouldn't do this, not that they can't do this - Apple (and many other phone manufacturers) have designed their phones such that they can replace the firmware with anything they want.

In order to prevent unauthorised firmware being installed on a device, Apple (and most other vendors) verify that any firmware updates are signed with a trusted key. The FBI don't have access to Apple's firmware signing keys, and as a result they're unable to simply replace the software themselves. That's why they're asking Apple to build a new firmware image, sign it with their private key and provide it to the FBI.

But what do we mean by "unauthorised firmware"? In this case, it's "not authorised by Apple" - Apple can sign whatever they want, and your iPhone will happily accept that update. As owner of the device, there's no way for you to reconfigure it such that it will accept your updates. And, perhaps worse, there's no way to reconfigure it such that it will reject Apple's.

I've previously written about how it's possible to reconfigure a subset of Android devices so that they trust your images and nobody else's. Any attempt to update the phone using the Google-provided image will fail - instead, they must be re-signed using the keys that were installed in the device. No matter what legal mechanisms were used against them, Google would be unable to produce a signed firmware image that could be installed on the device without your consent. The mechanism I proposed is complicated and annoying, but this could be integrated into the standard vendor update process such that you simply type a password to unlock a key for re-signing.

Why's this important? Sure, in this case the government is attempting to obtain the contents of a phone that belonged to an actual terrorist. But not all cases governments bring will be as legitimate, and not all manufacturers are Apple. Governments will request that manufacturers build new firmware that allows them to monitor the behaviour of activists. They'll attempt to obtain signing keys and use them directly to build backdoors that let them obtain messages sent to journalists. They'll be able to reflash phones to plant evidence to discredit opposition politicians.

We can't rely on Apple to fight every case - if it becomes politically or financially expedient for them to do so, they may well change their policy. And we can't rely on the US government only seeking to obtain this kind of backdoor in clear-cut cases - there's a risk that these techniques will be used against innocent people. The only way for Apple (and all other phone manufacturers) to protect users is to allow users to remove Apple's validation keys and substitute their own. If Apple genuinely value user privacy over Apple's control of a device, it shouldn't be a difficult decision to make.

comments

As you may be aware I have a bit of a problem with Single Board Computers in that I have a lot of them. Keeping them organised has turned into a bit of a problem.

I designed clip cases for many of these systems giving me a higher storage density on my rack shelves and built a power supply to reduce the cabling complexity. These helped but I still ended up with a cluttered shelf full of SBC.

I decided I would make a rack enclosure to hold the SBC, I was limited to material I could easily CNC machine which limited me to acrylic plastics or wood.

Initially I started with the idea of housing the individual boards in a toast rack arrangement. This would mean that the enclosure would have to be at least 2U high to fit the boards all the existing cases would have to be discarded. This approach was dropped when the drawbacks of having no flexibility and only being able to fit the units that were selected at design time became apparent (connector cutouts and mounting hole placement.

Instead I changed course to try and incorporate the existing cases which already solved the differing connector and mounting placement problem and gave me a uniform size to consider. Once I had this approach the design came pretty quickly. I used a tube girder construction 1U in height to get as much strength as possible from the 3mm acrylic plastic I would use.

The design was simply laser cut from sheet stock and fastened together with M3 nut and bolts. Once I corrected the initial design errors (I managed to get almost every important dimension wrong on the first attempt) the result was a success.

The prototype is a variety of colours because makespace ran out of suitably sized clear acrylic stock but the colouring has no effect on the result other than aesthetical. The structure gives a great deal of rigidity and there is no sagging or warping, indeed testing on the prototype got to almost 50Kg loading without a failure (one end clamped and the other end loaded at 350mm distance)

I added some simple rotating latches at the front which keep the modules held in place and allow units to be removed quickly if necessary.

Overall this project was successful and I can now pack five SBC per U neatly. It does limit me to using systems cased in my "slimline" designs (68x30x97mm) which currently means the Raspberry Pi B+ style and the Orange Pi PC.

Once small drawback is access to I/O and power connectors. These need to be right angled and must be unplugged before unit removal which can be a little fiddly. Perhaps a toast rack design of cases would have given easier connector access but I am happy with this trade off of space for density.

As usual the design files are freely available, perhaps they could be useful as a basis for other laser cut rack enclosure designs.

Canonical appear to require that you remove all trademarks entirely even if using them wouldn't be a violation of trademark law.

-- Matthew Garrett Each time Matthew brings this up, and as evidence continues to mount that Canonical either actually intends their IP policy to be read that way, or is intentionally keeping the situation unclear to FUD derivatives, I start wondering about references to Ubuntu in my software. Should such references be removed, or obscured, like "U*NIX" in software of old, to prevent exposing users to this trademark nonsense?

joey@darkstar:~/src/git-annex>git grep -i ubuntu  wc -l
457
joey@darkstar:~/src/ikiwiki>git grep -i ubuntu  wc -l
80
joey@darkstar:~/src/etckeeper>git grep -i ubuntu  wc -l
14

Most of the code in git-annex, ikiwiki, and etckeeper is licensed under the GPL or AGPL, and so Canonical's IP policy probably does not require that anyone basing a distribution on Ubuntu strip all references to "Ubuntu" from them. But then, there's Propellor:

joey@darkstar:~/src/propellor>git grep -i ubuntu  wc -l
10

Propellor is BSD licensed. It's in Ubuntu universe. It not only references Ubuntu in documentation, but contains code that uses that trademark:

data Distribution
        = Debian DebianSuite
          Ubuntu Release

So, if an Ubuntu-derived distribution has to remove "Ubuntu" from Propellor, they'd end up with a Propellor that either differs from upstream, or that can't be used to manage Ubuntu systems. Neither choice is good for users. Probably most small derived distributions would not have expertise to patch data types in a Haskell program and would have to skip including Propellor. That's not good for Propellor getting wide distribution either. I think I've convinced myself it would be for the best to remove all references to "Ubuntu" from Propellor.
done Similarly, Debconf is BSD licensed. I originally wrote it, but it's now maintained by Colin Watson, who works for Canonical. If I were still maintaining Debconf, I'd be looking at removing all instances of "Ubuntu" from it and preventing that and other Canonical trademarks from slipping back in later. Alternatively, I'd be happy to re-license all Debconf code that I wrote under the AGPL-3+. Update: Another package that comes to mind is Debootstrap, which is also BSD licensed. Of course it contains "Ubuntu" in lots of places, since it is how Ubuntu systems are built. I'm no longer an active developer of Debootstrap, but I hope its current developers carefully consider how this trademark nonsense affects it. PS: Shall we use "*buntu" as the, erm, canonical trademark-free spelling of "Ubuntu"? Seems most reasonable, unless Canonical has trademarked that too.

This is the story of libspotify, as experienced by a Spotify customer and libspotify developer for six years.

Step 1: Support your API for years Since April 2009, libspotify has been a mostly nice although proprietary C library for integrating with the Spotify music streaming service, providing both music metadata and full playback capabilities. Language wrappers have been written for a plenitude of programming languages (including my own pyspotify). libspotify is integrated into numerous open source projects (including my own Mopidy), networked AV receivers, and rumour has it: even cars. In addition to being closed source, there was another catch: in order to use it, you need the Spotify premium subscription. Speaking from my own experience supporting a project integrating with Spotify for the last six years: lots of end users upgraded to premium in order to use the projects built on top of libspotify.

Step 2: Stop accepting bug reports and stop releasing updates In May 2012, Spotify released libspotify 12.1.51 for all supported architectures, which now even included Android. After months of endless requests from the quickly growing hoards of Raspberry Pi users, Spotify released libspotify 12.1.103 for hardfloat armv6 on January 22 2013. The release included minor API additions, like the addition of explicit lyrics and means for accepting Spotify s Terms of Service. The additions in 12.1.103 never reached the other supported architectures and 12.1.51 remains the last release for those. There was no clear communication about this being the end of libspotify, releases have just ceased. libspotify has not been mentioned in a single developer blog post or developer email newsletter since. That s now more than three years ago.

Step 3: Create new APIs Over the next couple of years Spotify released a new Web API and iOS and Android SDKs. Through their developer blog, they communicated quite a bit both about these and about the deprecation of the apps inside the Spotify desktop client. Here are some pieces from their developer blog to illustrate:

• 2014-02-03: Deprecation of Spotify Apps API 0.x
• 2014-03-06: First Spotify iOS SDK beta release
• 2014-03-24: No longer accepting new Spotify Apps
• 2014-06-17: First Spotify Web API release
• 2014-06-25: First Spotify Android SDK beta release

The Web API grew steadily, as documented by the Web APIs changelog. Over time, its feature list improved, supporting anonymous endpoints for generic music metadata as well as OAuth protected endpoints for managing your own music collection and playlists. The only major part missing that prevented developers replacing libspotify was music playback, as the Web API only provides 30 second previews. From what I recall of 2014 IRC discussions with Spotify employees, they had built a new much smaller library for streaming music. The iOS and Android SDKs were simply platform specific wrappers around this new playback library and the new Web API. My impression is that, after a number of beta releases, the iOS and Android SDKs seamlessly replaced libspotify on those platforms. The plan was apparently to release the new playback library for desktop too, so that it, in combination with the Web API, could replace libspotify there as well. That was all back in 2014.

Step 4: Deprecate old APIs At some point, Spotify deprecated their old web API, the Metadata API. There was no blog post about it and it was not mentioned in any Spotify Developer News email I ve received in recent years. I guess they only published this fact as a note on the Metadata API web page; it is therefore hard to track down exactly when it was deprecated and how the deprecation was communicated. Anyway, it was deprecated, and the natural upgrade path was to the new Web API. Spotify even has a migration guide mapping the concepts of the Metadata API to the new Web API. All good. In May 2015, after 28 months without a single release or any news, libspotify is officially deprecated. The communication channel of choice? A note on the libspotify web page. No post on the developer blog. No Spotify Developer News email. To be able to use libspotify, all developers have registered and requested a personal application key. To my knowledge, none of these registered library users were notified directly.

Step 5: Shut down an old API On January 20 2016 Spotify end-of-lifed their Metadata API. I don t know if this date was communicated on the Metadata API web page well in advance of this shutdown. There s no blog post on the topic. There s no news email. There s no tweet from @SpotifyPlatform or @SpotifyStatus. I can t find anything except users commenting on various Spotify documentation pages that the text is outdated because the API is no longer merely deprecated, but entirely shut down. Hopefully they did publish the shutdown date ahead of time. It had a full replacement API and a migration guide. Fair and square. Nothing much to complain about here. It s time to get porting your code to the Web API! But frankly, I don t really care about this Metadata API because I don t use the Metadata API, do I?

Step 6: Break your API without warning February 3 2016, Mopidy users start reporting General transient error when searching Spotify. This is a libspotify error message known as SP_ERROR_OTHER_TRANSIENT with error code 16. This error usually means that there is some unspecified issue with the Spotify service. We ve actually seen it before and like its name suggests, it is indeed transient and soon goes away. According to a series of tweets between my fellow Mopidy developer Nick Steel and @SpotifyCares it appears that the libspotify search functionality was backed by the Metadata API s search functionality. Who knew? I m not aware of any relation between these APIs ever being mentioned publicly. No libspotify user could possibly have known that the Metadata API shutdown would affect them. Let s summarize: libspotify hasn t had any releases for three years and it was officially deprecated in May last year. But, there have been no warnings about any shutdown. The libspotify project is likely no longer staffed at all. Did someone just forget that libspotify depended on the Metadata API they were shutting down? Many newer networked AV receivers use the Spotify Connect libraries that are exclusive to Spotify s commercial partners. But some receivers probably still use libspotify. If so, did Spotify just break hardware sold with Spotify support as a major feature? Something that people are paying a monthly fee to use? It seems they did! In the Spotify Community forums there are numerous reports of broken devices and software that all have two things in common: the date the problem started occurring, and that search is the only feature that is broken.

• Denon receivers, including models: CEOL N8
• Logitech Squeezebox media players
• Marantz receivers, including models: M-CR510, M-CR610, NR1504, NR1603, SR5007, SR6008
• Onkyo receivers, including CR-N755, TX-NR509, TX-NR609, TX-8050
• Roku media players

Surely, Spotify will quickly explain themselves and fix this?

Step 7: Stay quiet Since the API shutdown, Spotify has been very quiet. No posts on the developer blog. No email newsletter. No tweets from @SpotifyPlatform since December 1 last year. Some users at The Spotify Community forum are quoting responses from Spotify. Most responses seem to link to the Metadata API migration docs and state that application developers must migrate. In other cases the responses point the finger at the commercial partner which only adds to the confusion. Some of these devices may use the Metadata API directly, and in those cases the application maintainers are probably to blame. However, I m quite certain that the libspotify search issue is responsible for a fair share of these complaints. libspotify, in contrast to the Metadata API, has never been officially shut down. The responsibility for any breakage caused by libspotify is Spotify s alone.

What I want from Spotify

1. Apologize to your paying customers and supporters.
2. Get substantially better at communicating deprecations and shutdowns. We want all news, not just good news.
3. Give us a new supported C library for audio playback. Now! Not later this year. I ve heard that promise for three consecutive years now. We can live with pre-release software quality initially. Getting the library out sooner rather than later shows us that there is a road ahead for all the projects that stream music from Spotify. Let us get started making new language wrappers.
4. Release Spotify Connect specifications and libraries. We want to make music players that can be controlled by the Spotify mobile app. We want to make music controllers that can play music on all the Spotify Connect software and hardware players we ve invested in. You ve created this technology, now let us use it.

Thanks to Nick Steel for reviewing this blog post.

Debian Long Term Support (LTS) This is my third month working on Debian LTS, started by Raphael Hertzog at Freexian. This month was my first month working on the frontdesk duty and did a good bunch of triage. I also performed one upload and reviewed a few security issues.

Frontdesk duties I spent some time trying to get familiar with the frontdesk duty. I still need to document a bit of what I learned, which did involve asking around for parts of the process. The following issues were triaged:

• roundcube in squeeze was happily not vulnerable to CVE-2015-8794 and CVE-2015-8793, as the code affected was not present. roundcube is also not shipped with jessie but the backport is vulnerable
• the php-openid vulnerability was actually just a code sample, a bug report comment clarified all of CVE-2016-2049
• ffmpeg issues were closed, as it is not supported in squeeze
• libxml2 was marked as needing work (CVE-2016-2073)
• asterisk was triaged for all distros before i found out it is also unsupported in squeeze (CVEs coming up, AST-2016-001, AST-2016-001, AST-2016-001)
• libebml and libmatroska were marked as unsupported, although an upload of debian-security-support will be necessary to complete that work (bug #814557 filed)

Uploads and reviews I only ended up doing one upload, of the chrony package (CVE-2016-1567), thanks to the maintainer which provided the patch. I tried my best trying to sort through the issues with tiff (CVE-2015-8668 and CVE-2015-7554), which didn't have obvious fixes available. OpenSUSE seem to have patches, but it is really hard to find them through their issue trackers, which were timing out on me. Hopefully someone else can pick that one up. I also tried and failed to reproduce the cpio issue (CVE-2016-2037), which, at the time, didn't have a patch for a fix. This ended up being solved and Santiago took up the upload. I finally spent some time trying to untangle the mess that is libraw, or more precisely, all the packages that embed dcraw code instead of linking against libraw. Now I really feel even more strongly for the Debian policy section 4.13 which states that Debian packages should not ship with copies of other packages code. It made it really hard to figure out which packages were vulnerable, especially because it was hard to figure out which versions of libraw/dcraw were actually vulnerable to the bug, but also just plain figure out which packages were copying code from libraw. I wish I had found out about secure-testing/data/embedded-code-copies earlier... Still, it was interesting to get familiar with codesearch.debian.net to try to find copies of the vulnerable code, which was not working so well. Kudos to darktable 2.0 for getting rid of their embedded copy of libraw, by the way - it made it completely not vulnerable to the issue, the versions in stretch and sid not having the code at all and older versions having non-vulnerable copies of the code.

Issues with VMs again I still had problems running a squeeze VM - not wanting to use virtualbox because of the overhead, I got lost for a bit trying to use libvirt and KVM. A bunch of issues crept up: using virt-manager would just fail on startup with an error saying interface mtu value is improper, which is a very unhelpful error message (what is a proper value??) - and, for the record, the MTU on eth0 and wlan0 is the fairly standard 1500, while lo is at 65536 bytes, nothing unusual there as far as I know. Then the next problem was actually running a VM - I still somewhat expected to be able to boot off a chroot, something I should definitely forget about it seems like (boot loader missing? not sure). I ended up calling virt-install with the live ISO image I was previously using:

virt-install --virt-type kvm --name squeeze-amd64 --memory 512 --cdrom ~/iso/Debian/cdimage.debian.org_mirror_cdimage_archive_6.0.10_live_amd64_iso_hybrid_debian_live_6.0.10_amd64_gnome_desktop.iso --disk size=4 --os-variant debiansqueeze

At least now I have an installed squeeze VM, something I didn't get to do in Virtualbox (mostly because I didn't want to wait through the install, because it was so slow). Finally, I still have trouble getting a commandline console on the VM: somehow, running virtsh console squeeze-amd64 doesn't give me a login terminal, and worse, it actually freezes the terminal that I can actually get on virt-viewer squeeze-amd64, which definitely sounds like a bug. I documented a bit more of that setup in the Debian wiki KVM page so hopefully this will be useful for others.

Other free software work I continued my work on improving timetracking with ledger in my ledger-timetracking git repository, which now got a place on the new plaintextaccounting.org website, which acts as a portal for ledger-like software projects and documentation.

Darktable 2.0 I had the pleasure of trying the new Darktable 2.0 release, which only recently entered Debian. I built a backport for jessie, which works beautifully: much faster thumbnail rendering, no dropping of history when switching views... The new features are great, but I also appreciate how they are being very conservative in their approach. Darktable is great software: I may have trouble approaching the results other are having with lightroom and snapseed, but those are proprietary software that I can't use anyways. I also suspect that I just don't have enough of a clue of what I'm doing to get the results I need in Darktable. Maybe with hand-holding, one day, I will surpass the results I get with the JPEGs from my Canon camera. Until then, I turned off RAW exports in my camera to try and control the explosion of disk use I saw since I got that camera:

41M     2004
363M    2005
937M    2006
2,2G    2007
894M    2008
800M    2009
1,8G    2010
1,4G    2011
9,8G    2012
31G     2013
26G     2014
9,8G    2015

The drop in 2015 is mostly due to me taking less pictures in the last year, for some reason...

Markdown mode hacks I ended up writing some elisp for the markdown mode. It seems I am always writing links like [text](link) which seems more natural at first, but then the formatting looks messier, as paragraph wrapping is all off because of the long URLs. So I always ended up converting those links, which was a painful series of keystrokes. So I made a macro, and while I'm a it, why not rewrite it as a lisp function. Twice. Then I was told by the markdown-mode.el developers that they had already fixed that (in the 2.1 version, not in Debian jessie) and that the C-c C-a r key binding actually recognized existing links and conveniently converted them. I documented my adventures in bug #94, but it seems I wrote this code for nothing else than re-learning Emacs lisp, which was actually quite fun.

More emacs hacking Another thing I always wasted time doing by and is "rename file and buffer". Often, you visit a file but it's named wrong. My most common case is a .txt file that i rename to .mdwn. I would then have to do:

M-x rename-file <ret> newfile
M-x rename-buffer <ret> newfile
C-x C-s <ret> newfile

Really annoying. Turns out that set-visited-file-name actually does most of the job, but doesn't actually rename the file, which is really silly. So I wrote this small function instead:

(defun rename-file-and-buffer (newfname)
  "combine rename-file and rename-buffer
set-visited-file-name does most of the job, but unfortunately
doesn't actually rename the file. rename-file does that, but
doesn't rename the buffer. rename-buffer only renames the buffer,
which is pretty pointless.
only operates on current buffer because set-visited-file-name
also does so and we don't bother doing excursions around.
"
  (interactive "GRename file and bufer: ")
  (let ((oldfname (buffer-file-name)))
    (set-visited-file-name newfname nil t)
    (rename-file oldfname newfname)
    )
  )

Not bound to any key, really trivial, but doing this without that function is really non-trivial, especially since set-visited-file-name needs special arguments to not mark the file as modified.

IRC packages updates I updated the Sopel IRC bot package to the latest release, 6.3.0. They have finally switched to Requests, but apart from that, no change was necessary. I am glad to finally see SNI support working everywhere in the bot! I also update the Charydbis IRC server package to the latest 3.5.0 stable release. This release is great news, as I was able to remove 5 of the 7 patches I was dragging along the Debian package. The previous Charybdis stable release was over 3 years old, as 3.4.2 was released in (December) 2012! I spend a good chunk of time making the package reproducible. I filed a bug upstream and eventually made a patch to make it possible to hardcode a build timestamp, which seems to have been the only detectable change in the reproducible build infrastructure. Charybdis had been FTBS for a while in sid now, and the upload should fix that as well. Unfortunately, Charybdis still doesn't build with hardening flags - but hopefully a future update of the package should fix that. It is probably because CFLAGS are not passed around properly. There's really interesting stuff going on in the IRC world. Even though IRC is one of the oldest protocols still in operation (1988, even before the Web, but after SMTP and the even more venerable FTP), it is still being actively developed, with a working group drafting multiple extensions to the IRCv3 protocol defined in RFC 1459. For example, IRCv3.3 includes a Strict Transport Security extension, which tries to ensure users use encrypted channels as much as possible, through warnings and STARTTLS support. Charybdis goes even further by proposing a reversal of the +S ("secure channel" flag) where all channels are secure by default, and you need to deliberately mark a channel as insecure with the +U flag if you actually want to allow users on an clear-text connection to join the channel. A transition mechanism is also proposed.

Miscellaneous bug reports En vrac... I fell face-first in this amazing game that is endless-sky. I made a small pull request on the documentation, a bug report and a feature request. I forwarded a bug report, originally filed against monkeysign, to the pyqrencode maintainers. I filed a usability bug against tails-installer, which just entered Debian, mostly usability issues. I discovered the fim image viewer, which re-entered Debian recently. It seemed perfect to adjust my photos-import workflow, so I added it to my script, to be able to review photos prior to importing them into Darktable and git-annex.

To state the obvious: my personal preference is to run Debian GNU/Linux. My current workplace is a CentOS shop and usually I'm the first to claim that it doesn't matter at all, and distribution specific implementation details are irrelevant for what we do (running a JVM). Let's take a short detour to the RedHat network-scripts. Two weeks ago we found some systems we originally installed in a different network segment, with different DNS servers and different search domains, came back up after a reboot with a rewritten 'resolv.conf'. Later on cfengine replaced the generated 'resolv.conf' with the intended one, so it wasn't that obvious to spot in the first place. A colleague found the origin of the rewritten 'resolv.conf' in a device specific configuration file that defined the 'DNS 1,2 ' variables with the installation time DNS server IPs from the other segment. I expected to experience the same behaviour (resolv.conf rewritten during startup and replaced by cfengine later on) in other locations, but assumed we just didn't notice it because the main difference would be a slightly different list of search domains. And I was wrong. I checked the timestamps of several 'resolv.conf' files and their cfengine backup file. None were recently created or related somehow to a reboot. grep-ing through parts of the network-scripts I found the following conditional in '/etc/sysconfig/network-scripts/ifup'

if [ "$PEERDNS" != "no" ]  [ -n "$RESOLV_MODS" -a "$RESOLV_MODS" != "no" ]; then
   [ -n "$MS_DNS1" ] && DNS1=$MS_DNS1
   [ -n "$MS_DNS2" ] && DNS2=$MS_DNS2
      if [ -n "$DNS1" ] && ! grep -q "^nameserver $DNS1" /etc/resolv.conf &&
         tr=$(mktemp /tmp/XXXXXX) ; then
 ...

So if you adjust only the second nameserver IP you stored in "DNS2" in your configuration you end up without an update to your 'resolv.conf'. Now knowing that, I'd say this is relevant distribution specific knowledge, and I'm wondering how many of such subtle behaviours we've hidden in Debian specific solutions? Maybe knowledge about distribution specific implementation details even matters after all. Regardig the 'resolv.conf' issue itself the fault is on us. We as the responsible team did not read the documentation properly and thus deployed a configuration that later on let to some unexpected consequences. I try to remember those issues as an example for the next NetworkManager/systemd-networkd vs old-school-network-scripts argument.

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December, 113.50 work hours have been dispatched among 9 paid contributors. Their reports are available:

• Antoine Beaupr did 11.5h.
• Ben Hutchings did 15 hours (out of 15 hours allocated + 10 extra hours remaining, meaning that he still has 10 extra hours to do over February).
• Chris Lamb did 18 hours.
• Guido G nther did 13 hours (out of 12 hours allocated + 1 remaining).
• Mike Gabriel did 16 hours (8 hours allocated + 8h remaining).
• Santiago Ruano Rinc n did 18 hours (out of 12h allocated + 8.50 remaining, thus keeping 2.50 extra hours for February).
• Scott Kitterman did 8 hours.
• Thorsten Alteholz did 30 hours.

Evolution of the situation As expected, we had a small drop in the amount of hours sponsored. New sponsors (re-)joined but others stopped too (Gree this time) mostly balancing the result. We only lost 2 hours of sponsored work. It would be nice if we could invert that curve and actually start again to get closer to our objective of funding the equivalent of a full time position. Let s hope that the switch to wheezy as the version supported by the LTS team will motivate many companies relying on Debian 7 in their IT system. In terms of security updates waiting to be handled, the situation is close to last month(17 packages in dla-needed.txt, 27 in the list of CVE). It looks like that having about 20 packages needing an update is the normal situation and that we can t really get further down given the time required to process some updates (sometimes we wait until the upstream authors provides a patch, and so on). Thanks to our sponsors New sponsors are in bold.

• Platinum sponsors:
• TOSHIBA (for 4 months)
• Gold sponsors:
• The Positive Internet (for 20 months)
• Blablacar (for 19 months)
• Linode LLC (for 9 months)
• Silver sponsors:
• Domeneshop AS (for 19 months)
• Universit Lille 3 (for 19 months)
• Trollweb Solutions (for 17 months)
• Nantes M tropole (for 13 months, renewal after a small break)
• University of Luxembourg (for 11 months)
• Dalenys (for 9 months)
• Univention GmbH (for 5 months)
• Universit Jean Monnet de St Etienne (for 5 months)
• Bronze sponsors:
• David Ayers IntarS Austria (for 20 months)
• Offensive Security (for 20 months)
• Seznam.cz, a.s. (for 20 months)
• Evolix (for 19 months)
• Freeside Internet Service (for 19 months)
• MyTux (for 19 months)
• Linuxhotel GmbH (for 17 months)
• Intevation GmbH (for 16 months)
• Daevel SARL (for 15 months)
• Bitfolk LTD (for 14 months)
• Megaspace Internet Services GmbH (for 14 months)
• Greenbone Networks GmbH (for 13 months)
• NUMLOG (for 13 months)
• WinGo AG (for 12 months)
• Ecole Centrale de Nantes LHEEA (for 8 months)
• Sig-I/O (for 6 months)
• Entr ouvert (for 4 months)
• Adfinis SyGroup AG

No comment Liked this article? Click here. My blog is Flattr-enabled.

One of the difficulties of being a kernel maintainer for a busy subsystem is that you will often end up getting a lot of mail that requires reading and handling which in turn requires sending a lot of mail out in reply. Some of that requires thought and careful consideration but a lot of it is quite routine and (perhaps surprisingly) there is often more challenge in doing a good job of handling these routine messages. For a long time I used to hand write every reply I sent but the problem with doing that is that sending the same message a lot of times tends to result in the messages getting more and more brief as the message becomes routine and practised. Your words become more optimised and if you ve stopped thinking about the message before you ve finished typing it then there s a desire to finish the typing and get on to the next thing. This is I think a lot of the reputation that kernel maintainers have for being terse and unhelpful comes from messages that are very practised for someone sending them all the time aren t always going to be obvious or helpful for someone who s not so intimately familiar with what s going on. The good part of it is that everyone is getting a personalised response and it s easy to insert a comment about that specific situation when you re already replying but it s not clear that the tradeoff is a good one. What I ve started doing instead for most things is keeping a set of pre-written paragraphs for common cases that I can just insert into a mail and edit as needed. Hopefully it s working well for people, it means the replies are that bit more verbose than they might otherwise be (mainly adding an explanation of why a given thing is being asked for) but can easily be adapted as needed. The one exception is the Applied, thanks mails I used to send when I apply a patch (literally just saying that). Those are now automatically generated by the script I use to sync my local git repository with kernel.org and very much more verbose:

From: Mark Brown <broonie@kernel.org>
To: $ CCS 
Cc: $ LIST 
Subject: $ SUBJECT 
In-Reply-To: $ MSGID 
The patch
   $ TITLE 
has been applied to the $ REPO  tree at
   $ URL  $ BRANCH 
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.

(unfortunately this bit seems to be something that it s worth pointing out)

If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark

(the script does try to CC relevant lists). As well as giving people more information this also means that the mails only get sent out when things actually get published to my public repositories which avoids some confusion that used to happen sometimes with people getting my replies before I d pushed, especially when I d been working with poor connectivity as often happens when travelling. On the down side it s very much an obvious form letter which some people don t like and which can make people glaze over. My hope with this is to make things easier on average for patch submitters and easier for me, feedback on the scripted e-mails appears to be good thus far and the goal with the pasted in content is that it should be less obvious that it s happening so I d expect less feedback there.

It's that time of the year where I go to FOSDEM (Free and Open Source Software Developers' European Meeting). The keynotes and the maintracks are very good, with good presentations and contents. It's very dificult to choose what talks to see, what talks to see later in video and what talks to loose. What I leave here is my selection of talks. This selection is representative of my tastes, not of the quality of the presentations. I will give links for material that is available now. I will do periodic updates when the new material is available: video or slides.

• Welcome to FOSDEM 2016

• systemd and Where We Want to Take the Basic Linux Userspace in 2016

• Ian Murdock - In Memoriam

• Putting 8 Million People on the Map

• Closing FOSDEM 2016

• Re-thinking Linux Distributions

• Beyond reproducible builds

• What Do Code Reviews at Microsoft and in Open Source Projects Have in Common?

• Rearchitecting Linux I/O towards Petascale Storage

• Digital Hardware Design: Why is it still so hard?

• Libreboot - free your BIOS today!

• How to design a Linux kernel API

• Scaling and Securing LibreOffice Online

• How containers work in Linux

• Micro-datacenter with Raspberry Pi and Kubernetes

• Live Migration of Virtual Machines From the Bottom Up

• Interesting Things You Can Do With ZFS

• ElectroBSD - Getting a reproducible BSD out of the door

• Reproducible builds in FreeBSD packages

• An introduction to LXD

• Filesystem defragmentation strategies

• CGroups: resources management inside and outside of a container

• Powering Twitter's infrastructure with containers

• Make backups and versions available easy.

This article describes how to use systemd-networkd on Debian Edu 8.x (aka jessie) notebooks. What we have to deal with? At the schools we support we have several notebooks running Debian Edu 8.x (aka jessie) in the field. For school notebooks (classroom sets) we install the Debian Edu Workstation Profile. Those machines are mostly used over wireless network. We know that Debian Edu also offers a Roaming Workstation Profile at installation time, but with that profile chosen, user logins create local user accounts and local home directories on the notebooks (package: libpam-mklocaluser). For our customers, we do not want that. People using the school notebooks shall always work on their NFS home directories. School notebooks shall not be usable outside of the school network. Our woes... The default setup on Debian Edu jessie workstations regarding networking is this:

• systemd runs as PID 1
• ifupdown manages static network interfaces (eth0, etc.)
• NetworkManager manages wireless network interfaces
• for our customers we configured NetworkManager with a system-wide WiFi (WPA2-PSK) profile

We have observed various problems with that setup:

• By default, network interface eth0 is managed by ifupdown (via /etc/network/interfaces):
  

  auto eth0
  iface eth0 inet dhcp
  

  Woe no. 1: In combination with systemd, this results in a 120sec delay at system startup.

read more

Note to myself so I don't have to search for it the next time I've to answer security audit questions. If you're lucky and you're running Debian you can install pgpdump and use

gpg --export-options export-minimal --export $KEYID   pgpdump

to retrieve a human friendly output. If you're unlucky you have to use

gpg --export-options export-minimal --export $KEYID   gpg --list-packets

and match the CIPHER_ALGO_ and DIGEST_ALGO_ numbers with those in include/cipher.h. Found the information in this thread. Update: anarcat suggested to take a look at the tools contained in hopenpgp-tools.

Johannes Bechberger, while working on his Bachelor s thesis supervised by my colleague Andreas Zwinkau, has developed a performance benchmark runner and results visualizer called temci , and used GHC as a guinea pig. You can read his elaborate analysis on his blog. This is particularly interesting given recent discussions about GHC itself becoming slower and slower, as for example observed by Johannes Waldmann and Anthony Cowley. Johannes Bechberger s take-away is that, at least for the programs at hand (which were taken from the The Computer Language Benchmarks Game, there are hardly any changes worth mentioning, as most of the observed effects are less than a standard deviation and hence insignificant. He tries hard to distill some useful conclusions from the data; the one he finds are:

• Compile time does not vary significantly.
• The compiler flag -O2 indeed results in faster code than -O.
• With -O (but not -O2), GHC 8.0.1 is better than GHC 7.0.1. Maybe some optimizations were promoted to -O?

If you are interested, please head over to Johannes s post and look at the gory details of the analysis and give him feedback on that. Also, maybe his tool temci is something you want to try out? Personally, I find it dissatisfying to learn so little from so much work, but as he writes: It s so easy to lie with statistics. , and I might add lie to yourself , e.g. by ignoring good advise about standard deviations and significance. I m sure my tool gipeda (which powers perf.haskell.org) is guilty of that sin. Maybe a different selection of test programs would yield more insight; the benchmark s games programs are too small and hand-optimized, the nofib programs are plain old and the fibon collection has bitrotted. I would love to see a curated, collection of real-world programs, bundled with all dependencies and frozen to allow meaningful comparisons, but updated to a new, clearly marked revision, on a maybe bi-yearly basis maybe Haskell-SPEC-2016 if that were not a trademark infringement.

What happened in the reproducible builds effort this week:

Toolchain fixes After remarks from Guillem Jover, Lunar updated his patch adding generation of .buildinfo files in dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dracut, ent, gdcm, guilt, lazarus, magit, matita, resource-agents, rurple-ng, shadow, shorewall-doc, udiskie. The following packages became reproducible after getting fixed:

• disque/1.0~rc1-5 by Chris Lamb, noticed by Reiner Herrmann.
• dlm/4.0.4-2 by Ferenc W gner.
• drbd-utils/8.9.6-1 by Apollon Oikonomopoulos.
• java-common/0.54 by by Emmanuel Bourg.
• libjibx1.2-java/1.2.6-1 by Emmanuel Bourg.
• libzstd/0.4.7-1 by Kevin Murray.
• python-releases/1.0.0-1 by Jan Dittberner.
• redis/2:3.0.7-2 by Chris Lamb, noticed by Reiner Herrmann.
• tetex-brev/4.22.github.20140417-3 by Petter Reinholdtsen.

Some uploads fixed some reproducibility issues, but not all of them:

• anarchism/14.0-4 by Holger Levsen.
• hhvm/3.11.1+dfsg-1 by Faidon Liambotis.
• netty/1:4.0.34-1 by Emmanuel Bourg.

Patches submitted which have not made their way to the archive yet:

• #813309 on lapack by Reiner Herrmann: removes the test log and sorts the files packed into the static library locale-independently.
• #813345 on elastix by akira: suggest to use the $datetime placeholder in Doxygen footer.
• #813892 on dietlibc by Reiner Herrmann: remove gzip headers, sort md5sums file, and sort object files linked in static libraries.
• #813912 on git by Reiner Herrmann: remove timestamps from documentation generated with asciidoc, remove gzip headers, and sort md5sums and tclIndex files.

reproducible.debian.net For the first time, we've reached more than 20,000 packages with reproducible builds for sid on amd64 with our current test framework. Vagrant Cascadian has set up another test system for armhf. Enabling four more builder jobs to be added to Jenkins. (h01ger)

Package reviews 233 reviews have been removed, 111 added and 86 updated in the previous week. 36 new FTBFS bugs were reported by Chris Lamb and Alastair McKinstry. New issue: timestamps_in_manpages_generated_by_yat2m. The description for the blacklisted_on_jenkins issue has been improved. Some packages are also now tagged with blacklisted_on_jenkins_armhf_only.

Misc. Steven Chamberlain gave an update on the status of FreeBSD and variants after the BSD devroom at FOSDEM 16. He also discussed how jails can be used for easier and faster reproducibility tests. The video for h01ger's talk in the main track of FOSDEM 16 about the reproducible ecosystem is now available.

FOSDEM was fun. It was great to see all these open source projects coming together in one place and it was really good to talk to people that were just as enthusiastic about the FOSS activities they do as I am about mine. Thanks go to Sa l Corretg who looked after the real-time communications dev room and made sure everything ran smoothly. I was very pleased to find that I had to stand for a couple of talks as the room was full with people eager to learn more about the world of RTC. I was again pleased on the Sunday when I had such a great audience for my talk in the distributions dev room. Everyone was very welcoming and after the talk I had some corridor discussions with a few people that were really interesting and have given me a few new things to explore in the near future. A few highlights from FOSDEM:

• ReactOS: Since I last looked at this project it has really matured and is getting to be rather stable. It may be possible to start seriously considering replacing Windows XP/Vista machines with ReactOS where the applications being run just cannot be used with later versions of Windows.
• Haiku: I used BeOS a long long time ago on my video/music PC. I can't say that I was using it over a Linux or BSD distribution for any particular reason but it worked well. I saw a talk that discussed how Haiku was keeping up-to-date with drivers and also there was a talk, that I didn't see, that talked about the new Haiku package management system. I think I may check out Haiku again in the near future, even if only for the sake of nostalgia.
• Kolab: Continuing with the theme of things that have matured since I last looked at them, I visited the Kolab stand at FOSDEM and I was impressed with how far it has come. In fact, I was so impressed that I'm looking at using it for my primary email and calendaring in the near future.
• picoTCP: When I did my Honours project at University, I was playing with Contiki. This looks a lot easier to get started with, even if it's perhaps missing parts of the stack that Contiki implements well. If I ever find time for doing some IoT hacking, this will be on the list of things to try out first.

This is just some of the highlights, and I know I'm missing out a lot here. One of the main things that FOSDEM has done for me is open my eyes as to how wide and diverse our community is and it has served as a reminder that there is tons of cool stuff out there if you take a moment to look around. Also, thanks to my trip to FOSDEM, I now have four new t-shirts to add into the rotation: FOSDEM 2016, Debian, XMPP and twiki.org.

I've integrated letsencrypt into propellor today. I'm using the reference letsencrypt client. While I've seen complaints that it has a lot of dependencies and is too complicated, it seemed to only need to pull in a few packages, and use only a few megabytes of disk space, and it has fewer options than ls does. So seems fine. (Although it would be nice to have some alternatives packaged in Debian.) I ended up implementing this:

letsEncrypt :: AgreeTOS -> Domain -> WebRoot -> Property NoInfo

This property just makes the certificate available, it does not configure the web server to use it. This avoids relying on the letsencrypt client's apache config munging, which is probably useful for many people, but not those of us using configuration management systems. And so avoids most of the complicated magic that the letsencrypt client has a reputation for. Instead, any property that wants to use the certificate can just use leteencrypt to get it and set up the server when it makes a change to the certificate:

letsEncrypt (LetsEncrypt.AgreeTOS (Just "me@my.domain")) "example.com" "/var/www"
     onChange  setupthewebserver

(Took me a while to notice I could use onChange like that, and so divorce the cert generation/renewal from the server setup. onChange is awesome! This blog post has been updated accordingly.) In practice, the http site has to be brought up first, and then letsencrypt run, and then the cert installed and the https site brought up using it. That dance is automated by this property:

Apache.httpsVirtualHost "example.com" "/var/www"
    (LetsEncrypt.AgreeTOS (Just "me@my.domain"))

That's about as simple a configuration as I can imagine for such a website!

---

The two parts of letsencrypt that are complicated are not the fault of the client really. Those are renewal and rate limiting. I'm currently rate limited for the next week because I asked letsencrypt for several certificates for a domain, as I was learning how to use it and integrating it into propellor. So I've not quite managed to fully test everything. That's annoying. I also worry that rate limiting could hit at an inopportune time once I'm relying on letsencrypt. It's especially problimatic that it only allows 5 certs for subdomains of a given domain per week. What if I use a lot of subdomains? Renewal is complicated mostly because there's no good way to test it. You set up your cron job, or whatever, and wait three months, and hopefully it worked. Just as likely, you got something wrong, and your website breaks. Maybe letsencrypt could offer certificates that will only last an hour, or a day, for use when testing renewal. Also, what if something goes wrong with renewal? Perhaps letsencrypt.org is not available when your certificate needs to be renewed. What I've done in propellor to handle renewal is, it runs letsencrypt every time, with the --keep-until-expiring option. If this fails, propellor will report a failure. As long as propellor is run periodically by a cron job, this should result in multiple failure reports being sent (for 30 days I think) before a cert expires without getting renewed. But, I have not been able to test this.

With thanks to Iain Learmonth for the hardware, there is now a Debian instance of LAVA available for use and the Debian wiki page has been updated. LAVA is a continuous integration system for deploying operating systems onto physical and virtual hardware for running tests. Tests can be simple boot testing, bootloader testing and system level testing. Extra hardware may be required for some system tests. Results are tracked over time and data can be exported for further analysis. LAVA has a long history of supporting continuous integration of the Linux kernel on ARM devices (ARMv7 & ARMv8). So if you are testing a Linux kernel image on armhf or arm64 devices, you will find a lot of similar tests already running on the other LAVA instances. The Debian LAVA instance seeks to widen that testing in a couple of ways:

• A wider range of tests including use of Debian artifacts as well as mainline Linux builds
• A wider range of devices by allowing developers to offer devices for testing from their own desks.
• Letting developers share local test results easily with the community without losing the benefits of having the board on your desk.

This instance relies on the latest changes in lava-server and lava-dispatcher. The 2016.2 release has now deprecated the old, complex dispatcher and a whole new pipeline design is available. The Debian LAVA instance is running 2015.12 at the moment, I ll upgrade to 2016.2 once the packages migrate into testing in a few days and I can do a backport to jessie. What can LAVA do for Debian? ARMMP kernel testing Unreleased builds, experimental initramfs testing this is the core of what LAVA is already doing behind the scenes of sites like http://kernelci.org/. U-Boot ARM testing This is what fully automated LAVA labs have not been able to deliver in the past, at least without a usable SD Mux What s next LOTS. This post actually got published early (distracted by the rugby) I ll update things more in a later post. Contact me if you want to get involved, I ll provide more information on how to use the instance and how to contribute to the testing in due course.

Last week-end, I was in Brussels, Belgium for the FOSDEM, one of the greatest open source developer conference. I was not sure to go there this year (I already skipped it in 2015), but it turned out I was requested to do a talk in the shared Lua & GNU Guile devroom. As a long time Lua user and developer, and a follower of GNU Guile for several years, the organizer asked me to run a talk that would be a link between the two languages. I've entitled my talk "How awesome ended up with Lua and not Guile" and gave it to a room full of interested users of the awesome window manager . We continued with a panel discussion entitled "The future of small languages Experience of Lua and Guile" composed of Andy Wingo, Christopher Webber, Ludovic Court s, Etiene Dalcol, Hisham Muhammaad and myself. It was a pretty interesting discussion, where both language shared their views on the state of their languages. It was a bit awkward to talk about Lua & Guile whereas most of my knowledge was year old, but it turns out many things didn't change. I hope I was able to provide interesting hindsight to both community. Finally, it was a pretty interesting FOSDEM to me, and it was a long time I didn't give talk here, so I really enjoyed it. See you next year!