Search Results: "magnus"

6 October 2021

Reproducible Builds: Reproducible Builds in September 2021

The goal behind reproducible builds is to ensure that no deliberate flaws have been introduced during compilation processes via promising or mandating that identical results are always generated from a given source. This allowing multiple third-parties to come to an agreement on whether a build was compromised or not by a system of distributed consensus. In these reports we outline the most important things that have been happening in the world of reproducible builds in the past month:
First mentioned in our March 2021 report, Martin Heinz published two blog posts on sigstore, a project that endeavours to offer software signing as a public good, [the] software-signing equivalent to Let s Encrypt . The two posts, the first entitled Sigstore: A Solution to Software Supply Chain Security outlines more about the project and justifies its existence:
Software signing is not a new problem, so there must be some solution already, right? Yes, but signing software and maintaining keys is very difficult especially for non-security folks and UX of existing tools such as PGP leave much to be desired. That s why we need something like sigstore - an easy to use software/toolset for signing software artifacts.
The second post (titled Signing Software The Easy Way with Sigstore and Cosign) goes into some technical details of getting started.
There was an interesting thread in the /r/Signal subreddit that started from the observation that Signal s apk doesn t match with the source code:
Some time ago I checked Signal s reproducibility and it failed. I asked others to test in case I did something wrong, but nobody made any reports. Since then I tried to test the Google Play Store version of the apk against one I compiled myself, and that doesn t match either. was announced this month, which aims to be a repository of Reproducible Build Proofs for Bitcoin Projects :
Most users are not capable of building from source code themselves, but we can at least get them able enough to check signatures and shasums. When reputable people who can tell everyone they were able to reproduce the project s build, others at least have a secondary source of validation.

Distribution work Fr d ric Pierret announced a new testing service at, showing actual rebuilds of binaries distributed by both the Debian and Qubes distributions. In Debian specifically, however, 51 reviews of Debian packages were added, 31 were updated and 31 were removed this month to our database of classified issues. As part of this, Chris Lamb refreshed a number of notes, including the build_path_in_record_file_generated_by_pybuild_flit_plugin issue. Elsewhere in Debian, Roland Clobus posted his Fourth status update about reproducible live-build ISO images in Jenkins to our mailing list, which mentions (amongst other things) that:
  • All major configurations are still built regularly using live-build and bullseye.
  • All major configurations are reproducible now; Jenkins is green.
    • I ve worked around the issue for the Cinnamon image.
    • The patch was accepted and released within a few hours.
  • My main focus for the last month was on the live-build tool itself.
Related to this, there was continuing discussion on how to embed/encode the build metadata for the Debian live images which were being worked on by Roland Clobus.
Ariadne Conill published another detailed blog post related to various security initiatives within the Alpine Linux distribution. After summarising some conventional security work being done (eg. with sudo and the release of OpenSSH version 3.0), Ariadne included another section on reproducible builds: The main blocker [was] determining what to do about storing the build metadata so that a build environment can be recreated precisely . Finally, Bernhard M. Wiedemann posted his monthly reproducible builds status report.

Community news On our website this month, Bernhard M. Wiedemann fixed some broken links [ ] and Holger Levsen made a number of changes to the Who is Involved? page [ ][ ][ ]. On our mailing list, Magnus Ihse Bursie started a thread with the subject Reproducible builds on Java, which begins as follows:
I m working for Oracle in the Build Group for OpenJDK which is primary responsible for creating a built artifact of the OpenJDK source code. [ ] For the last few years, we have worked on a low-effort, background-style project to make the build of OpenJDK itself building reproducible. We ve come far, but there are still issues I d like to address. [ ]

diffoscope diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb prepared and uploaded versions 183, 184 and 185 as well as performed significant triaging of merge requests and other issues in addition to making the following changes:
  • New features:
    • Support a newer format version of the R language s .rds files. [ ]
    • Update tests for OCaml 4.12. [ ]
    • Add a missing format_class import. [ ]
  • Bug fixes:
    • Don t call close_archive when garbage collecting Archive instances, unless open_archive definitely returned successfully. This prevents, for example, an AttributeError where PGPContainer s cleanup routines were rightfully assuming that its temporary directory had actually been created. [ ]
    • Fix (and test) the comparison of R language s .rdb files after refactoring temporary directory handling. [ ]
    • Ensure that RPM archives exists in the Debian package description, regardless of whether python3-rpm is installed or not at build time. [ ]
  • Codebase improvements:
    • Use our assert_diff routine in tests/comparators/ [ ]
    • Move diffoscope.versions to diffoscope.tests.utils.versions. [ ]
    • Reformat a number of modules with Black. [ ][ ]
However, the following changes were also made:
  • Mattia Rizzolo:
    • Fix an autopkgtest caused by the androguard module not being in the (expected) python3-androguard Debian package. [ ]
    • Appease a shellcheck warning in debian/tests/ [ ]
    • Ignore a warning from h5py in our tests that doesn t concern us. [ ]
    • Drop a trailing .1 from the Standards-Version field as it s required. [ ]
  • Zbigniew J drzejewski-Szmek:
    • Stop using the deprecated distutils.spawn.find_executable utility. [ ][ ][ ][ ][ ]
    • Adjust an LLVM-related test for LLVM version 13. [ ]
    • Update invocations of llvm-objdump. [ ]
    • Adjust a test with a one-byte text file for file version 5.40. [ ]
And, finally, Benjamin Peterson added a --diff-context option to control unified diff context size [ ] and Jean-Romain Garnier fixed the Macho comparator for architectures other than x86-64 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Testing framework The Reproducible Builds project runs a testing framework at, to check packages and other artifacts for reproducibility. This month, the following changes were made:
  • Holger Levsen:
    • Drop my package rebuilder prototype as it s not useful anymore. [ ]
    • Schedule old packages in Debian bookworm. [ ]
    • Stop scheduling packages for Debian buster. [ ][ ]
    • Don t include PostgreSQL debug output in package lists. [ ]
    • Detect Python library mismatches during build in the node health check. [ ]
    • Update a note on updating the FreeBSD system. [ ]
  • Mattia Rizzolo:
    • Silence a warning from Git. [ ]
    • Update a setting to reflect that Debian bookworm is the new testing. [ ]
    • Upgrade the PostgreSQL database to version 13. [ ]
  • Roland Clobus (Debian live image generation):
    • Workaround non-reproducible config files in the libxml-sax-perl package. [ ]
    • Use the new DNS for the snapshot service. [ ]
  • Vagrant Cascadian:
    • Also note that the armhf architecture also systematically varies by the kernel. [ ]

Contributing If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

9 August 2016

Reproducible builds folks: Reproducible builds: week 67 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday July 31 and Saturday August 6 2016: Toolchain development and fixes Packages fixed and bugs filed The following 24 packages have become reproducible - in our current test setup - due to changes in their build-dependencies: alglib aspcud boomaga fcl flute haskell-hopenpgp indigo italc kst ktexteditor libgroove libjson-rpc-cpp libqes luminance-hdr openscenegraph palabos petri-foo pgagent sisl srm-ifce vera++ visp x42-plugins zbackup The following packages have become reproducible after being fixed: The following newly-uploaded packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews and QA These are reviews of reproduciblity issues of Debian packages. 276 package reviews have been added, 172 have been updated and 44 have been removed in this week. 7 FTBFS bugs have been reported by Chris Lamb. Reproducibility tools Test infrastructure For testing the impact of allowing variations of the buildpath (which up until now we required to be identical for reproducible rebuilds), Reiner Herrmann contribed a patch which enabled build path variations on testing/i386. This is possible now since dpkg 1.18.10 enables the --fixdebugpath build flag feature by default, which should result in reproducible builds (for C code) even with varying paths. So far we haven't had many results due to disturbances in our build network in the last days, but it seems this would mean roughly between 5-15% additional unreproducible packages - compared to what we see now. We'll keep you updated on the numbers (and problems with compilers and common frameworks) as we find them. lynxis continued work to test LEDE and OpenWrt on two different hosts, to include date variation in the tests. Mattia and Holger worked on the (mass) deployment scripts, so that the - for space reasons - only GIT clone resides in ~jenkins-adm/ and not anymore in Holger's homedir, so that soon Mattia (and possibly others!) will be able to fully maintain this setup, while Holger is doing siesta. Miscellaneous Chris, dkg, h01ger and Ximin attended a Core Infrastricture Initiative summit meeting in New York City, to discuss and promote this Reproducible Builds project. The CII was set up in the wake of the Heartbleed SSL vulnerability to support software projects that are critical to the functioning of the internet. This week's edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

30 April 2016

Stein Magnus Jodal: March and April contributions

The following is a short summary of my open source work in March and April, almost like in previous months, except that I haven t spent as much time as previously on Open Source the last two months.


  • Bugfixes for the upcoming Mopidy 2.0.1 (which should have been released a long time ago): merged PR #1455, created PR #1493.
  • Started on, but didn t finish, fixing the Travis CI setup for Mopidy-GMusic.
  • Upgraded the Mopidy project server from Ubuntu 14.04 LTS to 16.04 LTS. Rebuilt the Discourse/Docker instance.
  • Accepted Lars Kruse as the new maintainer of Mopidy-Beets. Thanks!
  • The extensions still in need of a new maintainer are: If you re a user of any of these and want to contribute, please step up. Instructions can be found in the README of any of these projects.

11 March 2016

Steinar H. Gunderson: Agon and the Candidates tournament

The situation where Agon (the designated organizer of the Chess World Championship, and also the Candidates tournament, the prequalifier to said WC) is trying to claim exclusive rights of the broadcasting of the moves (not just the video) is turning bizarre. First of all, they have readily acknowledged they have no basis in copyright to do so; chess moves, once played, are facts and cannot be limited. They try to jump through some hoops with a New York-specific doctrine (even though the Candidates, unlike the World Championship, is played in Moscow) about hot news , but their main weapon seems to be that they simply will throw out anyone from the hall who tries to report on the moves, and then try to give them only to those that promise not to give them on. This leads to the previously unheard-of situation where you need to register and accept their terms just to get to watch the games in your browser. You have to wonder what they will be doing about the World Championship, which is broadcast unencrypted on Norwegian television (previous editions also with no geoblock). Needless to say, this wasn't practically possible to hold together. All the big sites (like Chessdom, ChessBomb and Chess24) had coverage as if nothing had happened. Move sourcing is a bit of a murky business where nobody really wants to say where they get the moves from (although it's pretty clear that for many tournaments, the tournament organizers will simply come to one or more of the big players with an URL they can poll at will, containing the games in the standard PGN format), and this was no exception ChessBomb went to the unusual move of asking their viewers to download Tor and crowdsource the moves, while Chessdom and Chess24 appeared to do no such thing. In fact, unlike Chessdom and ChessBomb, Chess24 didn't seem to say a thing about the controversy, possibly because they now found themselves on the other side of the fence from Norway Chess 2015, where they themselves had exclusive rights to the PGN in a similar controversy although it would seem from a tweet that they were perfectly okay with people just re-broadcasting from their site if they paid for a (quite expensive) premium membership, and didn't come up with any similar legal acrobatics to try to scare other sites. However, their ToS were less clear on the issue, and they didn't respond to requests for clarification at the time, so I guess all of this just continues to be on some sort of gentleman's agreement among the bigger players. (ChessBomb also provides PGNs for premium members for the tournaments they serve, but they expressly prohibit rebroadcast. They claim that for the tournaments they host, which is a small minority, they provide free PGNs for all.) Agon, predictably, sent out angry letters where they threatened to sue the sites in question, although it's not clear at all to me what exactly they would sue for. Nobody seemed to care, except one entity TWIC, which normally has live PGNs from most tournaments, announced they would not be broadcasting from the Candidats tournament. This isn't that unexpected, as TWIC (which is pretty much a one-man project anyway) mainly is about archival, where they publish weekly dumps of all top-level games played that week. This didn't affect a lot of sites, though, as TWIC's live PGNs are often not what you'd want to base a top-caliber site on (they usually lack clock information, and moves are often delayed by half a minute or so). I run a hobby chess relay/analysis site myself (mainly focusing on the games of Magnus Carlsen), though, so I've used TWIC a fair bit in the past, and if I were to cover the Candidates tournament (I don't plan to do so, given Agon's behavior, although I plan to cover the World Championship itself), I might have been hit by this. So, that was the background. The strange part started when, Agon's broadcasting site, promptly went down during the first round of the Candidates tournament today Agon blamed DDoS, which I'm sure is true, but it's unclear exactly how strong the DDoS was, and if they did anything at all to deal with it other than to simply wait it out. But this lead to the crazy situation where the self-declared monopolist was the only big player not broadcasting the tournament in some form. And now, in the trully bizarre move, World Chess is publishing a detailed rebuttal of Agon's arguments, explaining how it is bad for chess, not juridically sound, and also morally wrong. Yes, you read that right; Agon's broadcast site is carrying an op-ed saying Agon is wrong. You at least have to give them credit for not trying to censor their columinst when he says something they don't agree with. Oh, and if you want those PGNs? I will, at least for the time being, be pushing them out live on I have not gone into any agreement with Agon, and they're hosted in Norway, far from any New York-specific doctrines. So feel free to relay from them, although I would of course be happy to know if you do.

1 March 2016

Stein Magnus Jodal: February contributions

The following is a short summary of my open source work in February, just like in previous months.


  • Released Mopidy-Spotify 2.3.1: Works around search being broken in libspotify by using the Spotify Web API for the search functionality, and nothing else.
  • Wrapped up and released Mopidy 2.0 release. This was a quite large release that ports to GStreamer 1.x and adds support for gapless playback. It was uploaded to Debian in time to be part of Ubuntu 16.04 LTS.
  • Released Mopidy-Spotify 3.0.0: A couple of small compatibility fixes to work with Mopidy 2.x and GStreamer 1.x.
  • Updates to packages not in Debian, only at
    • Uploaded mopidy-spotify-tunigo 1.0.0-0mopidy1. Far down the chain, it depends on libspotify, which is not in Debian.
    • Uploaded mopidy-spotify 2.0.0-0mopidy1.
  • Updated homebrew-mopidy with lots of new releases:
    • Mopidy 2.0.0
    • Mopidy-Dirble 1.3.0
    • Mopidy-SoundCloud 2.0.2
    • Mopidy-Spotify 2.3.0, 2.3.1 and 3.0.0
    • python-backports-abc 0.4
    • python-backports-ssl-match-hostname
    • python-certifi 2015.11.20.1
    • python-cffi 1.5.0 and 1.5.2
    • python-requests 2.9.1
    • python-singledispatch
    • python-six 1.10.0
    • python-tornado 4.3
  • Rebased my feature/py3-compat branch on top of Mopidy 2.0.
  • Planned breaking changes for Mopidy 3.0: unbundling of Mopidy.js, removal of deprecated core APIs, model APIs, audio APIs and removal of the http/static_dir config.
  • Planned breaking changes to modernize Mopidy.js: change default calling convention to by-position-or-by-name, replace When.js with ES6 Promise, replace Bane.js with EventEmitter, and replace Buster.js with Karma+Mocha.

  • Started working on the upgrade to Django 1.8. Most deps are upgraded. The major remaining pieces is to upgrade django-registration from 0.8 to 2.0 and to replace the years-old vendored copy of django-invitation (singular) with django-invitations (plural).

18 February 2016

Stein Magnus Jodal: A guide to poor API management

This is the story of libspotify, as experienced by a Spotify customer and libspotify developer for six years.

Step 1: Support your API for years Since April 2009, libspotify has been a mostly nice although proprietary C library for integrating with the Spotify music streaming service, providing both music metadata and full playback capabilities. Language wrappers have been written for a plenitude of programming languages (including my own pyspotify). libspotify is integrated into numerous open source projects (including my own Mopidy), networked AV receivers, and rumour has it: even cars. In addition to being closed source, there was another catch: in order to use it, you need the Spotify premium subscription. Speaking from my own experience supporting a project integrating with Spotify for the last six years: lots of end users upgraded to premium in order to use the projects built on top of libspotify.

Step 2: Stop accepting bug reports and stop releasing updates In May 2012, Spotify released libspotify 12.1.51 for all supported architectures, which now even included Android. After months of endless requests from the quickly growing hoards of Raspberry Pi users, Spotify released libspotify 12.1.103 for hardfloat armv6 on January 22 2013. The release included minor API additions, like the addition of explicit lyrics and means for accepting Spotify s Terms of Service. The additions in 12.1.103 never reached the other supported architectures and 12.1.51 remains the last release for those. There was no clear communication about this being the end of libspotify, releases have just ceased. libspotify has not been mentioned in a single developer blog post or developer email newsletter since. That s now more than three years ago.

Step 3: Create new APIs Over the next couple of years Spotify released a new Web API and iOS and Android SDKs. Through their developer blog, they communicated quite a bit both about these and about the deprecation of the apps inside the Spotify desktop client. Here are some pieces from their developer blog to illustrate: The Web API grew steadily, as documented by the Web APIs changelog. Over time, its feature list improved, supporting anonymous endpoints for generic music metadata as well as OAuth protected endpoints for managing your own music collection and playlists. The only major part missing that prevented developers replacing libspotify was music playback, as the Web API only provides 30 second previews. From what I recall of 2014 IRC discussions with Spotify employees, they had built a new much smaller library for streaming music. The iOS and Android SDKs were simply platform specific wrappers around this new playback library and the new Web API. My impression is that, after a number of beta releases, the iOS and Android SDKs seamlessly replaced libspotify on those platforms. The plan was apparently to release the new playback library for desktop too, so that it, in combination with the Web API, could replace libspotify there as well. That was all back in 2014.

Step 4: Deprecate old APIs At some point, Spotify deprecated their old web API, the Metadata API. There was no blog post about it and it was not mentioned in any Spotify Developer News email I ve received in recent years. I guess they only published this fact as a note on the Metadata API web page; it is therefore hard to track down exactly when it was deprecated and how the deprecation was communicated. Anyway, it was deprecated, and the natural upgrade path was to the new Web API. Spotify even has a migration guide mapping the concepts of the Metadata API to the new Web API. All good. In May 2015, after 28 months without a single release or any news, libspotify is officially deprecated. The communication channel of choice? A note on the libspotify web page. No post on the developer blog. No Spotify Developer News email. To be able to use libspotify, all developers have registered and requested a personal application key. To my knowledge, none of these registered library users were notified directly.

Step 5: Shut down an old API On January 20 2016 Spotify end-of-lifed their Metadata API. I don t know if this date was communicated on the Metadata API web page well in advance of this shutdown. There s no blog post on the topic. There s no news email. There s no tweet from @SpotifyPlatform or @SpotifyStatus. I can t find anything except users commenting on various Spotify documentation pages that the text is outdated because the API is no longer merely deprecated, but entirely shut down. Hopefully they did publish the shutdown date ahead of time. It had a full replacement API and a migration guide. Fair and square. Nothing much to complain about here. It s time to get porting your code to the Web API! But frankly, I don t really care about this Metadata API because I don t use the Metadata API, do I?

Step 6: Break your API without warning February 3 2016, Mopidy users start reporting General transient error when searching Spotify. This is a libspotify error message known as SP_ERROR_OTHER_TRANSIENT with error code 16. This error usually means that there is some unspecified issue with the Spotify service. We ve actually seen it before and like its name suggests, it is indeed transient and soon goes away. According to a series of tweets between my fellow Mopidy developer Nick Steel and @SpotifyCares it appears that the libspotify search functionality was backed by the Metadata API s search functionality. Who knew? I m not aware of any relation between these APIs ever being mentioned publicly. No libspotify user could possibly have known that the Metadata API shutdown would affect them. Let s summarize: libspotify hasn t had any releases for three years and it was officially deprecated in May last year. But, there have been no warnings about any shutdown. The libspotify project is likely no longer staffed at all. Did someone just forget that libspotify depended on the Metadata API they were shutting down? Many newer networked AV receivers use the Spotify Connect libraries that are exclusive to Spotify s commercial partners. But some receivers probably still use libspotify. If so, did Spotify just break hardware sold with Spotify support as a major feature? Something that people are paying a monthly fee to use? It seems they did! In the Spotify Community forums there are numerous reports of broken devices and software that all have two things in common: the date the problem started occurring, and that search is the only feature that is broken. Surely, Spotify will quickly explain themselves and fix this?

Step 7: Stay quiet Since the API shutdown, Spotify has been very quiet. No posts on the developer blog. No email newsletter. No tweets from @SpotifyPlatform since December 1 last year. Some users at The Spotify Community forum are quoting responses from Spotify. Most responses seem to link to the Metadata API migration docs and state that application developers must migrate. In other cases the responses point the finger at the commercial partner which only adds to the confusion. Some of these devices may use the Metadata API directly, and in those cases the application maintainers are probably to blame. However, I m quite certain that the libspotify search issue is responsible for a fair share of these complaints. libspotify, in contrast to the Metadata API, has never been officially shut down. The responsibility for any breakage caused by libspotify is Spotify s alone.

What I want from Spotify
  1. Apologize to your paying customers and supporters.
  2. Get substantially better at communicating deprecations and shutdowns. We want all news, not just good news.
  3. Give us a new supported C library for audio playback. Now! Not later this year. I ve heard that promise for three consecutive years now. We can live with pre-release software quality initially. Getting the library out sooner rather than later shows us that there is a road ahead for all the projects that stream music from Spotify. Let us get started making new language wrappers.
  4. Release Spotify Connect specifications and libraries. We want to make music players that can be controlled by the Spotify mobile app. We want to make music controllers that can play music on all the Spotify Connect software and hardware players we ve invested in. You ve created this technology, now let us use it.
Thanks to Nick Steel for reviewing this blog post.

1 February 2016

Stein Magnus Jodal: January contributions

The following is a short summary of my open source work in January, just like I did back in November and December.


  • PR #1381: Made lookup() ignore tracks without URI.
  • Updated docs for Raspberry Pi installation to match Raspbian jessie.
  • Rewrote docs for running Mopidy as a service, more focused on systemd than Debian specifics to also cater for Arch users.
  • PR #1397: Added missing MPD volume command.
  • Merged a bunch of contributed fixes and released Mopidy 1.1.2.
  • Updated all extensions hosted under the Mopidy GitHub organization with either the name of the primary maintainer or a call for a new maintainer. The extensions in need of a new maintainer are: If you re a user of any of these and want to contribute, please step up. Instructions can be found in the README of any of these projects.
  • The feature/gst1 branch is complete as far as I know. There are no known regressions from Mopidy 1.1.2. PR #1419 is hopefully the last iteration of the pull request and GStreamer 1.x support will land in Mopidy 1.2.
  • Wrapping up the 1.2 release is now the focus. We might want to include this in Debian/Ubuntu before the Ubuntu 16.04 import freeze February 18, depending on feedback over the next week or two.

  • Added one new crawler.
  • Released comics 2.4.2.
  • Merged one new crawler and a crawler update.

12 January 2016

Bits from Debian: New Debian Developers and Maintainers (November and December 2015)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

1 January 2016

Stein Magnus Jodal: December contributions

The following is a short summary of my open source work in December, following up on my first report in November.


  • The feature/gst1 branch: Finished porting Mopidy from GStreamer 0.10 to PyGI and GStreamer 1.x. Merge of the branch is currently blocked on a single test failure (test_gapless) and issues with transitioning from one track to another with Mopidy-Spotify, which is the only backend using an appsrc for playback. The goal is for this branch to be part of Mopidy 1.2, which I hope to have in Debian/Ubuntu before the Ubuntu 16.04 import freeze February 18.
  • The feature/py3-compat branch: I ve worked quite a bit on this private branch, frequently rebased on top of feature/gst1. Currently Mopidy starts without any crashes on Python 3 and the test suite is down to 262 failed and 1841 passed tests. My current thinking, is that this will become part of a Mopidy 2.0 release, which will support both Python 2.7 and 3.4+. As soon as most of Mopidy s extension ecosystem supports Python 2+3, a new Mopidy major release (3.0?) will drop Python 2 support.
  • Merged a bunch of pull requests, both targeting the 1.1.2 bug fix release and the 1.2 feature release.

30 December 2015

Steve Kemp: I joined the internet of things.

In my old flat I had a couple of simple radio-controlled switches, which allowed me to toggle power to a pair of standing lamps - one at each side of the bed. This was very lazy, but also really handy and I've always been curious about automation.. When it comes to automation there seems to be three main flavours:
The original standard, with stuff produced by many vendors and good Linux support. X10 supports two ways of sending/receiving commands - over the electrical wiring, and over RF.
This is the newcomer, which despite that seems to be well-supported and extensible. It allows "measurements" to be sent/received in addition to the broadcast of events like "switch on", and "switch off".
Other systems - often lighting-centric
There are toy-things like the previously noted power-controlling things, there are also stand-alone devices from people like Philips with their philips hue system, but given how Philips recently crippled their devices to disable third-party bulbs I've no desire to use them. One company caught my eye though, Osram make a smart lightbulb and mini-hub to work with it.
So I bought one of the osram lightify systems, consisting of a magic box and a pair of lightbulbs. The box connects to your wifi, and gets an IP address. The IP address is then used by the application on your mobile phone (i.e. the magic box does the magic, not the bulbs). The phone application can be used to trigger "on", "off", "dim", "brighter", and the various colour-changing commands, as you would expect. You absolutely must use the phone-based application to do the setup, but after that the whole point was that I could automate things. I wanted to be able to setup my desktop computer to schedule events, and started hacking. I've written a simple Perl module to let me discover bulbs, and turn them off and on. No doubt it'll be on CPAN in the near future, once I can pick a suitable name for it:
$ ol --bridge= --list
hall       MAC:8418260000d9c70c RGBW:255,255,255,255 STATE:On
kitchen    MAC:8418260000cb433b RGBW:255,255,255,255 STATE:On
$ ol --bridge= --off=kitchen
$ ol --bridge= --list
hall       MAC:8418260000d9c70c RGBW:255,255,255,255 STATE:On
kitchen    MAC:8418260000cb433b RGBW:255,255,255,255 STATE:Off
The only niggle was the fiddly pairing, and the lack of any decent documentation. The code I wrote was loosely based on the python project python-lightify written by Mikael Magnusson. Also worth noting that the bridge/magic-box only exposes a single port so you can find the device on your VLAN by nmapping for port 4000:
$ nmap -v -p 4000
The device doesn't seem to allow any network setup at all - it only uses DHCP. So you might want to make sure it gets assigned a stable IP. Anyway I'm going to bed. When I do so I'll turn the lights off with my mobile phone. Neat. In the future I will look at more complex automation, and I think Z-wave is the way I'll go. Right now I'm in a rented flat so replacing wall-switches, etc, is something I can't do. But the systems I've looked at seem neat, and this current setup will keep me amused for several months!

30 November 2015

Stein Magnus Jodal: November contributions

The following is a short summary of my open source work in November. My hope is that keeping better track of what I m doing will help me reflect on how I spend my time, and help me to focus my efforts better.


  • Released Mopidy-Spotify 2.2.0: Fixes related to duplicate Starred playlists and albums from year 0.
  • Moved Mopidy s Travis CI testing from Ubuntu 12.04 to Ubuntu 14.04, to prepare for GStreamer 1.x, and eventually testing with Python 3.4. PR #1341
  • Worked on porting Mopidy from GStreamer 0.10 to PyGI and GStreamer 1.x. PR #1339
  • Briefly looked at what remains to get Mopidy running on both Python 2.7 and 3.4+ when we ve landed the port to GStreamer 1.x. Doesn t look too bad, except that ConfigParser doesn t want to work with bytes in Python 3, so there s no easy way to read a config file referring to a path on a non-UTF-8 file system.

  • Fixed two old crawlers. Added two new crawlers.
  • Needs to upgrade to Django 1.8 before the Django 1.7 security support ends this December.

11 November 2015

Bits from Debian: New Debian Developers and Maintainers (September and October 2015)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

4 May 2015

Lunar: Reproducible builds: first week in Stretch cycle

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let's see if we can keep everyone up-to-date on the developments. Before the release of Jessie The story goes back a long way but a formal announcement to the project has only been sent in February 2015. Since then, too much work has happened to make a complete report, but to give some highlights: Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon. This past week It seems changes were pilling behind the curtains given the amount of activity that happened in just one week. Toolchain fixes We also rebased the experimental version of debhelper twice to merge the latest set of changes. Lunar submitted a patch to add a -creation-date to genisoimage. Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc. Juan Picca submitted a patch to add a --use-date flag to texi2html. Packages fixed The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons-math3, devscripts, disruptor, ehcache, ftphs, gtk2hs-buildtools, haskell-abstract-deque, haskell-abstract-par, haskell-acid-state, haskell-adjunctions, haskell-aeson, haskell-aeson-pretty, haskell-alut, haskell-ansi-terminal, haskell-async, haskell-attoparsec, haskell-augeas, haskell-auto-update, haskell-binary-conduit, haskell-hscurses, jsch, ledgersmb, libapache2-mod-auth-mellon, libarchive-tar-wrapper-perl, libbusiness-onlinepayment-payflowpro-perl, libcapture-tiny-perl, libchi-perl, libcommons-codec-java, libconfig-model-itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt-perl, libdatetime-timezone-perl, libdbd-firebird-perl, libdbix-class-resultset-recursiveupdate-perl, libdbix-profile-perl, libdevel-cover-perl, libdevel-ptkdb-perl, libfile-tail-perl, libfinance-quote-perl, libformat-human-bytes-perl, libgtk2-perl, libhibernate-validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl-perl, liblog-any-perl, libmail-imapclient-perl, libmocked-perl, libmodule-build-xsutil-perl, libmodule-extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia-perl, libnet-frame-layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql-abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar-perl, libterm-shellui-perl, libtest-dir-perl, libtest-log4perl-perl, libtext-context-eitherside-perl, libtime-warp-perl, libtree-simple-perl, libwww-shorten-simple-perl, libwx-perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl-tools, quodlibet, r-cran-rjags, r-cran-rjson, r-cran-sn, r-cran-statmod, ruby-nokogiri, sezpoz, skksearch, slurm-llnl, stellarium. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: Improvements to Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content. Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies. Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version. debbindiff development Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster. Documentation update Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc. Package reviews Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build. For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing. 1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West. Summer of code Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let's welcome them! They have until May 25th before coding officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

7 December 2012

Christoph Berg:

So we finally made it, and sent out an official announcement for This new repository hosts packages for all PostgreSQL server versions (at the moment 8.3, 8.4, 9.0, 9.1, 9.2) for several Debian/Ubuntu distributions (squeeze, wheezy, sid, precise) on two architectures (amd64, i386). Now add packages for extension modules on top of all these, and you get a really large amount of binaries from a small number of sources. Right now there's 1670 .deb files and 148 .dsc files, but the .dsc count includes variants that only differ in the version number per distribution (we attach .pgdg60+1 for squeeze packages, .pgdg70+1 for wheezy and so on), so the real number of different sources is rather something like 81, with 38 distinct source package names. Dimitri Fontaine, Magnus Hagander, and I have been working on this since I first presented the idea at PGconf.EU 2011 in Amsterdam. We now have a Jenkins server building all the packages, an archive server with the master repository, and a feed that syncs the repository to the FTP (well, mostly http) server. If you were previously using, that's the same archive as on (one rsync away). Please update your sources.list to point to, I'll shut down the archive at that location at the end of January. Here's the Quickstart instructions from the Wiki page: Import the repository key from
wget -O -   sudo apt-key add -
Edit /etc/apt/sources.list.d/pgdg.list. The distributions are called codename-pgdg. In the example, replace squeeze with the actual distribution you are using:
deb squeeze-pgdg main
Configure apt's package pinning to prefer the PGDG packages over the Debian ones in /etc/apt/preferences.d/pgdg.pref:
Package: *
Pin: release
Pin-Priority: 500
Note: this will replace all your Debian/Ubuntu packages with available packages from the PGDG repository. If you do not want this, skip this step. Update the package lists, and install the pgdg-keyring package to automatically get repository key updates:
apt-get update
apt-get install pgdg-keyring

27 August 2012

Christoph Berg: PostgreSQL in Debian Hackathon

Almost a year has passed since my talk at 2011 in Amsterdam on Connecting the Debian and PostgreSQL worlds, and unfortunately little has happened on that front, mostly due to my limited spare time between family and job. is up and running, but got few updates and is lagging behind on PostgreSQL releases. Luckily, we got the project moving. Dimitri Fontaine and Magnus Hagander suggested to do a face-to-face meeting, so we got together at my house for two days last week and discussed ideas, repository layouts, build scripts, and whatnot to get all of us aligned for pushing the project ahead. My employer sponsored my time off work for that. We almost finished moving the repository to infrastructure, barring some questions of how to hook the repository into the existing mirror infrastructure; this should get resolved this week. The build server running Jenkins is still located on my laptop, but moving this to a proper host will also happen really soon now. We are using Mika Prokop's jenkins-debian-glue scripts for driving the package build from Jenkins. The big plus point about Jenkins is that it makes executing jobs on different distributions and architectures in parallel much easier than a bunch of homemade shell scripts could get us with reasonable effort. Here's a list of random points we discussed: We really aim at using unmodified packages from Debian as much as possible, and in fact this project doesn't mean to replace Debian's PostgreSQL packaging work, but to extend it beyond the number of server versions (and Debian and Ubuntu versions covered) supported. The people behind the Debian and Ubuntu packages, and this repository are mostly the same, so we will claim that "our" packages will be the same quality as the "original" ones. Big thanks go to Martin Pitt for maintaining the postgresql-common testsuite that really covers every aspect of running PostgreSQL servers on Debian/Ubuntu systems. Stay tuned for updates! :)

26 January 2012

Russell Coker: Links January 2012

Cops in Tennessee routinely steal cash from citizens [1]. They are ordered to do so and in some cases their salary is paid from the cash that they take. So they have a good reason to imagine that any large sum of money is drug money and take it. David Frum wrote an insightful article for NY Mag about the problems with the US Republican Party [2]. has an interesting article about eco-friendly features on some modern cruise ships [3]. Dan Walsh describes how to get the RSA SecureID PAM module working on a SE Linux system [4]. It s interesting that RSA was telling everyone to turn off SE Linux and shipping a program that was falsely marked as needing an executable stack and which uses netstat instead of /dev/urandom for entropy. Really the only way RSA could do worse could be to fall victim to an Advanced Persistent Attack :-# The Long Now has an interesting summary of a presentation about [5]. I never realised the range of things that stores, I will have to explore that if I find some spare time! Jonah Lehrer wrote a detailed and informative article about the way that American high school students receive head injuries playing football[6]. He suggests that it might eventually be the end of the game as we know it. Fran ois Marier wrote an informative article about optimising PNG files [7], optipng is apparently the best option at the moment but it doesn t do everything you might want. Helen Keeble wrote an interesting review of Twilight [8]. The most noteworthy thing about it IMHO is that she tries to understand teenage girls who like the books and movies. Trying to understand young people is quite rare. Jon Masters wrote a critique of the concept of citizen journalism and described how he has two subscriptions to the NYT as a way of donating to support quality journalism [9]. The only comment on his post indicates a desire for biased news (such as Fox) which shows the reason why most US media is failing at journalism. Luis von Ahn gave an interesting TED talk about crowd-sourced translation [10]. He starts by describing CAPTCHAs and the way that his company ReCAPTCHA provides the CAPTCHA service while also using people s time to digitise books. Then he describes his online translation service and language education system DuoLingo which allows people to learn a second language for free while translating text between languages [11]. One of the benefits of this is that people don t have to pay to learn a new language and thus poor people can learn other languages great for people in developing countries that want to learn first-world languages! DuoLingo is in a beta phase at the moment but they are taking some volunteers. Cory Doctorow wrote an insightful article for the Publishers Weekly titles Copyrights vs Human Rights [12] which is primarily about SOPA. Naomi Wolf wrote an insightful article for The Guardian about the Occupy movement, among other things the highest levels of the US government are using the DHS as part of the crackdown [13]. Naomi s claim is that the right-wing and government attacks on the Occupy movement are due to the fact that they want to reform the political process and prevent corruption. John Bohannon gave an interesting and entertaining TED talk about using dance as part of a presentation [14]. He gave an example of using dancerts to illustrate some concepts related to physics and then spoke about the waste of PowerPoint. Joe Sabia gave an amusing and inspiring TED talk about the technology of storytelling [15]. He gave the presentation with live actions on his iPad to match his words, a difficult task to perform successfully. Thomas Koch wrote an informative post about some of the issues related to binary distribution of software [16]. I think the problem is evenm worse than Thomas describes. Related posts:
  1. Links January 2011 Halla Tomasdottir gave an interesting TED talk about her financial...
  2. Links January 2010 Magnus Larsson gave an interesting TED talk about using bacteria...
  3. Links January 2009 Jennifer 8 Lee gave an interesting TED talk about the...

2 October 2011

Gregor Herrmann: RC bugs 2011/39

another weekly report about RC bugs I've fixed, this time mostly by applying patches from ubuntu:

14 January 2010

Russell Coker: Links January 2010

Magnus Larsson gave an interesting TED talk about using bacteria to transform dunes into architecture [1]. The concept of making a wall across Africa to stop sand dunes from overtaking farm land is obviously a good one, the idea of using bacteria to convert sand into sandstone to do so cheaply is also good. But making that into houses seems a little risky. I wouldn t want to live under shifting sand with only bacteria generated sandstone to protect me. Cory Doctorow gave an interesting speech titled How to Destroy the Book , here is the transcript [2]. He talks about how much he loves books and described his opposition to the DRM people who want to destroy the book culture. Sendmail has a DKIM Wizard for generating ADSP (Domain Signing Policy) records [3]. If I knew that ADSP records were so easy to implement then I would have used them a year ago! Loretta Napoleoni gave an insightful TED talk about the economics of terrorism [4]. Apparently the US dollar used to be THE currency for international crime, when the PATRIOT act was passed it s anti-money-laundering provisions encouraged many shady people to invest in Euros instead and thus led to the devaluation of the US currency. It s also interesting to note that terrorist organisations are driven by economics, if only we could prevent them from making money Ryan Lobo gave an interesting TED talk about his photographic work [5]. The effectiveness of the all-women peace-keeping force is noteworthy. The part about the Liberian war criminal who has become an evangelical Christian and who now tours Liberia begging forgiveness from his victims (and their relatives in the case of the people he murdered). Should someone like that be permitted to remain free if his victims forgive him? Charles Stross has an appealing vision for how Apple and Google can destroy the current mobile telephony market [6]. I can t wait for the mobile phone market to be entirely replaced by mobile VOIP devices! James Geary gave an interesting TED talk about metaphors [7]. The benefits of metaphors in poetry are well known (particularly in lyrics), but the impact of metaphors in influencing stock market predictions surprised me. Shaffi Mather gave an interesting TED talk about his company that makes money from fighting corruption [8]. Instead of paying a bribe you can pay his company to force the official(s) in question to do the right thing. Apparently the cost of doing so tends to be less than 10% the cost of the bribe if you know what you are doing. His previous company was an ambulance service that charges what the patient can afford is also interesting. John Robb wrote an interesting article about lottery winners and griefers [9]. He suggests that publishing the names, addresses, etc of rich people will be a new trend in Griefing. One thing I ve been wondering about is the value of the HR database at a typical corporation. A single database typically contains the home addresses, phone numbers, and salaries of all the employees. It would be very easy to do an SQL dump and store it on a USB flash device to carry out of the office. Then it could be sold to the highest bidder. They could probably make a market in the private data about rich people in the same way that there is currently a market for credit card data maybe they have already done this but it s kept quiet to stop others from implementing the same idea. Michael Smith wrote an interesting article for the Washington Times about home schooling and socialisation [10]. It seems that people who were home schooled as children tend to be more academically successful and involved in civic life as well as being happier and having career success. Richard Seager wrote an interesting article for American Scientist about ocean currents and heat transfer from the tropics [11]. It seems that when the ocean currents shut down the UK and other parts of northern Europe won t be getting a mini ice-age. Ian Lance Taylor (most known for the gold linker) has written a good summary of the situation in regard to climate change and what must be done about it [12]. The Wrath of the Killdozer article about how one angry man converted a bulldozer into a tank [13]. This wasn t a big bulldozer (every mining company has bigger ones) and he didn t have any serious weapons (only rifles). Imagine what terrorists could do if they started with a mining vehicle and serious weapons Simon Singh has written about being sued for libel by the British Chiropractic Association [14]. The BCA didn t like his article criticising chiropractors for claiming to be able to treat many conditions unrelated to the spine. Remember, chiropractors are not doctors all they can do is alleviate some back problems. See a GP if you have any medical condition that doesn t involve a sore back or neck. Avoid uppity chiropracters who claim to be able to cure all ills. Nicholas D. Kristof wrote an interesting article for the New York times about how happy the people in Costa Rica are [15]. He claims that the Costa Rican government s decision in 1949 to dissolve it s armed forces and invest the money in education is the root cause of the happy population. Maybe if the US government would scale back military spending the US population could be as happy as the Costa Ricans. While there are good arguments for having some sort of military, there are no good arguments for spending more money on the military than the rest of the world combined (as the US does).

15 March 2008

Daniel Burrows: Why should Europeans get all the cool words?

My sister is studying at a German university. Having talked to her about her experiences there means I can understand blog posts like this one instead of just being confused. For non-Europeans: apparently in some parts of continental Europe video projectors are called beamers. Given how much more succinct and easy on the vocal apparatus beamer is than video projector, not to mention that it's plain cool, I think we should adopt the name here in the States. All in favor: refer to the next video projector you see as a beamer. If enough of us do it they'll call it a movement! UPDATE: the original post confused overhead and video projectors. I always have used the terms interchangably, and apparently incorrectly. I'm not sure why: maybe it's partly because when I was a kid the only projectors I encountered were overheads. Anyway, several readers wrote me to say that beamer is quite specifically a term for a video-projector and not at all a term for the overhead type. My bad. Comment by Meike Reichle on 2007-12-13: Hey Daniel! actually we do call an overhead projector[0] simply an "Overhead" (in the former GDR is used to be called a "Polylux"). A "Beamer" is a video projector[1]. Best wishes from Germany, Meike [0] [1] Comment by Magnus Therning on 2007-12-13: No, /video projectors/ are called beamers in some places. Search for "beamer" on wikipedia and you'll find "Video projector, a pseudo-anglicism in a number of languages including German and Dutch." An overhead projector is quite a different thing: /Magnus (wearing his besserwisser hat)

14 June 2007

Marc 'Zugschlus' Haber: Please test exim4 from experimental

I have uploaded exim4 4.67-2 to experimental. Lots of changes and improvements. Quite some changes have gone into the Debconf stuff (for example, the split/unsplit config question is not asked first any more), and into update-exim4.conf (including input sanitazion, transformation of input to lower case, and getting rid of the DEBCONFsomethingDEBCONF stuff in the configuration). I’d like you to test the experimental package before I upload to unstable (probably on sunday). Please report your findings.
exim4 (4.67-2) experimental; urgency=low
  - update-exim4.conf:
    - finally get rid of the DEBCONFfooDEBCONF stuff. That information
      is now passed to the configuration by ue4c by directly setting exim
      macros in the configuration. This has caused both the configuration
      and ue4c to be much shorter.
    - run with -e, -C and -u.
    - convert input read from update-exim4.conf.conf to lower case
    - barf if strange characters are found in ue4cc. Closes: #400294
  - Remove superfluous “x$foo” = “xbar” constructs from scripts
  - Add routers to reject mail to accounts with low UID.
    Closes: #400790.
  - Make daily cron job barf if /usr/bin/mail is not found. Have
    exim4-base recommend mailx. Closes: #427960
  - Have all -daemon packages provide exim4-localscanapi-1.0 and
    exim4-localscanapi-1.1 as requested by Magnus Holmgren while fixing
    #426425. Also include exim4-localscan-plugin-config script with
    exim4-dev. Thanks to Magnus for helping with this. Closes: #428274
  - remove /etc/exim4/email-addresses symlink and document this.
    Thanks to Josip Rodin. Closes: #420578
  - introduce conf.d/250_exim4-config_lowuid which optionally allows
    to reject (or alias away) mail to low-uid accounts that are not
    listed in an exception list. Thanks to Dominic Hargreaves,
    Marc Sherman and Ross Boylan. Closes: #400790, #307768, #331716
  - remove versioned depends on cron, since the version we need is
    well before sarge.
  - Add cron   fcron dependency. Fcron is going to be removed again
    at the first sign of trouble. Closes: #381806
  - remove move_exim3_spool debconf template. Closes: #391762
  - replace openssl gendh with openssl dhparam. Closes: #413235
  - adapt docs, README and manpages
  - have Hilko fix the lynx-dump postprocessing to repair generating
    README.Debian text version. Thanks!
  - increase README.Debian generation robustness. Thanks to Hilko.
  - debconf:
    - Partly apply Christian Perrier’s patch for reviewed
      templates and control file. Closes: #426980
    - Other minor template changes.
    - get rid of “mails” in debconf templates, use “messages” instead.
      Re-word local_interface debconf template. Other minor changes.
      Thanks to Jens Seidel and Christian Perrrier. Closes: #394976
    - re-work exim4-config.config logic to have split/non-split config
      asked last instead of first. This partly addresses #410756.
    - Add exim4-daemon-heavy.templates, exim4-daemon-light.templates
      and exim4.templates to
    - Re-Word dc_other_hostnames debconf template.
      Thanks to Hans G. Ehrbar. Closes: #421860
    - translation updates:
      - French
      - Ukrainian. Closes: #427793
      - Bulgarian.
      - Thai.
      - Galician.
      - Swedish.
      - Punjabi.
      - Indonesian.
      - Italian.
      - Khmer.
      - Traditional Chinese. Closes: #428072, #428069.
      - Portuguese.
      - Simplified Chinese. Closes: #428072, #428069.
      - Marathi
 -- Marc Haber <>  Wed, 13 Jun 2007 14:00:38 +020