Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

• Various improvements to django-slack, a library to easily post messages to the Slack group-messaging utility from projects using the Django web development framework:
  • Added explicit support to send messages asynchronously via the Celery distributed task queue. (#29)
  • Worked with Patrick Clope to add an escapeslack template filter to ensure the correct characters for Slack's API are escaped using Django's built-in safe is too invasive. (#36)
  • Corrected an issue where custom endpoints and channel names were incompatible. (#27)
  • Overhaul of the field/option/block parsing. (49ff3c4)
  • Moved away from using a django.template.Context instance, preventing a deprecation warning. (#31)
• Added a create-folders subcommand to my tickle-me-email Getting Things Done (GTD) email toolbox to create numbered folders for the rotate "tickler" functionality. (#3)
• Wrote and released a Django template tag to collapse multiple whitespace/newline characters in order to correctly format, for example, plaintext emails that make extensive use of for-loops. These would normally require careful and fragile placement of the % for .. % and % endfor % tags. (Repo)
• Pushed a number of updates to my Strava Enhancement Suite, a Chrome extension that improves and fixes annoyances in the web interface of the Strava cycling and running tracker:
  • Fix unit conversion where the element was wrapped in an another HTML element. (#48)
  • Hide premium indicator from top-level navigation. (#45)
  • Correct matching of the "shop" link in top-level navigation and footer module. (#47 & #46)
  • Remove "premium" pages on Club pages. (#44)
• Added a security-oriented warning to Ansible's documentation regarding the behaviour if an UFW firewall application profile is added and subsequently removed. (#1740)
• Added support to my django-staticfiles-dotd "staticfiles" library which concatenates Javascript and CSS files from .d-style directories to support an arbitrary file rendering method. This allows the use of media pre-processors such as SASS within such directories. (#1)
• Corrected my Chrome extension for the FastMail web interface to correctly hook into the internal "send" mechanism. (#2)
• Moved my stravabot IRC bot to use dh-virtualenv and systemd, improving security and reliability. (1c48709)
• Updated django-pedantic-http-methods a tool to raise an exception during development when attempting to perform side effects in GET and HEAD HTTP methods to support the latest version of Django. (#1)

---

---

---

This article starts a series of blogs on my ventures into Android land after having used iOS devices for 6 years. I have never been a strong believer in the Apple ecosystem, and my trials to convince myself to MacBooks and MacPros always ended with installing Linux on them, so I guess I wasn t really the ideal target for iOS. Thus, my iOS devices were also permanently in jailbroken state, otherwise I would feel amputated. But as much as I disliked the lock-in and closed environment of the iOS world, it was from the user perspective surprisingly well done and smooth. So it was with a certain level of tension when I finally switched to Linux^WAndroid. If you don t want to read on, here the preliminary conclusion: Why didn t I do it earlier! But before we go into details, let me start with my background: History of my devices Originally I was a big opponent of smart phones and preferred the Unix-way, one device for one thing. So I had a normal phone and (various) Palm devices (Tungsten X, Tungsten C, and above all my beloved Handera TRGpro). I loved the Palm world and considered it superior to the then smart phone world, until I came to Japan, where the challenge of a proper input method for Japanese and proper Japanese support posed a big hurdle. The Palm devices had a stick and written input fields, but Japanese input was practically impossible and a huge pain. Searching for a word in Japanese was more hurdle then looking it up in a printed dictionary. In addition, I needed a phone in Japan, so I plunged into the smart phone world and got myself a iPhone (3g). What a world did open for me: easy typing of Japanese, dictionaries, on-the-fly translation, woooow! And above all, I discovered my most beloved and till now one of my most important programs: Flashcards Deluxe. Thanks, and I have to say to 80% thanks to this program my Japanese learning speed has accelerated considerably. There is nothing more important for me than getting drilled in a systematic way. But I derail, anyway, having Flashcard Deluxe on the iPhone within rather short time I had about 10000 or more flashcards created, and moving on to a different architecture (Android) was for quite some time practically impossible without loosing years of statistics and learning, so I renewed my contract after 2 years together with a iPhone 4s. Another two years passed, and these years brought an Android version of Flashcards Deluxe, as well as Dropbox syncing, so I had no excuse anymore to remain in iOS land, wouldn t it be for an iPhone 5s that was passed to me near the end of my forth year, so I again extended the contract for two years. Finally, after 6 years of iPhone devices, this January I finally decided it is time to switch to Android. After lots of thinking, comparing, and requesting advice from good friends with more experience in the smart phone market I went for a Google Nexus 6p. Google Nexus 6p I will not repeat the specs of this phone as they are widely available on the net. My original plan was a Samsung S6, but after consultation with an expert I decided for an original Google phone for better security support. That left me with the option between a Nexus 5x and 6p, and due to prize differences (prizes of mobiles are ridiculously strange in Japan) I went for the 6p instead of the 5x. One point that made the decision for this slight too big device easy was the fact that it uses a great AMOLED display. Moving the data Since I was using Google Calendar and Google Contacts already on the iPhone, moving to the Android phone was far less a hassle than I thought. My contacts and events showed up without a hiccup. Most of the usual apps are nowadays available on both iOS and Android, so the most difficult thing was remembering all the passwords to log into the applications again (SNS like G+, FB, Twitter etc). The same is more or less true for messengers of all kinds (Line, WhatsApp, Threema, etc), but here one is advised to check with the respective web site first to make sure one does not loose all of the important data. Line for example is a stupid ***** that deletes all previous chats on the old phone and does not make them available on the new one. WhatsApp can be converted with a special conversion program. Threema, too allows for transfer of ids. Move of applications After that came the hunt for replacement applications for those that are not available as is on Android: Mail At first like probably everyone I used the shipped GMail program. It might be good for Google Mail accounts, but for anything else it is just a real pain. Thus, I have searched a bit and finally settled (for now) for K-9 Mail: it is open source, open development, feature rich, and more a hackers type email program, perfectly suited to me. There is a commercial variant called K@-Mail that says that it improves the user interface and some usability items as well as features, but I didn t see much of an advantage over the original version (which is completely free) and in fact some of my accounts didn t work at all. So I remain with K9 Mail and I think this is a good decision. Calendar Managing Calendars is one of the most important task for me. I have been a fervent supporter of DateBk4, DateBk 5, and DateBk 6 on the original Palm series, and when I left the Palm World it was with great pain that I had to loose DateBk. Not only because it was a simply fantastic calendar program that allowed me to keep track of all my climbing routes, festivities, in a much more advanced way than any other Calendering application, but also because the programmer of the DateBk series is running the Dewar Wildlife Trust, a Gorilla rescue group and a lot of the money he makes from the app sales is going to rescue Gorillas. With the switch to iOS this option was gone, and I first used the built-in calendar application (which is so weak) and later and for long time Pocket Informant Pro. This is a very good program and probably the only one that can compete with DateBk with respect to functionality and usefulness. During the time of me being locked in in iOS I realized that the world has moved on and a new version of DateBk series for Java was developed, called Pimlical. First only available on Windows, it became later available also on Android and Linux, too. The following screen shot puts Pimlical on the left, and Pocket Informant on the right. I will write a more detailed comparison in future, in short: PInformant is more streamlined and polished, Pimlical has more configuration option. Practically everything can be adjusted to one s need, and in addition there is also a Desktop application that sync either with Google and the phone, or only with the phone if you want to live off the grid. So nowadays on Android I have both Pocket Informant Pro as well as Pimlical, but after a short time I have now switched practically exclusively to Pimlical. Notes Here there is pain HUGE PAIN!!! iOS has an excellent applications for notes, called simply Notebooks. This little pearl was my work horse for everything (more or less) memorable. From poems and song texts to bus time tables, from PDF to GIFs, from MarkDown to HTML, everything could be saved into Notebooks, displayed, edited, ordered. And above all it had automatic background sync with Dropbox. So I could drop new files into the respective sub-folder of my Dropbox folder and could be sure I have the files available on my phone when I leave for a trip. And there is a huge bag of features that I haven t even tapped into! Android is unfortunately not on the list of supported architectures of Notebooks. So I searched far and wide, and without any success. There are all kind of notes, flash colors, overly simple, fast and slow, stylish and plain, but none of them did even provide half of the features of Notebooks. None, not even half. I still hope I might find the ultimate notes application, or even better would be an Android version of the original Notebooks application (but this is not high on the developers todo list), but for now I am in despair The Rest As I said, most apps are nowadays available on both platforms, so there is not much more to do than download the respective Android app and log in again. That worked very nice across practically all apps. Things I don t like (i.e., which are broken!) on Android Although a very convenient system and perfectly made to fit my taste, there are some things that are a huge pain (and a big shame on Google to not being able to fix that for long time!): Japanese fonts when the device is in English interface language In case you are a foreigner living in Japan and want your Android phone in English, but still read emails, news, etc in Japanese, then Android provides you with the worst, namely Chinese fonts: This is a well known problem and I have blogged about fixing the very same problem on Linux (Debian), and the solution is a simple reshuffling in the fontconfig configuration files. There is even an application for it in the Google store, Kanji Fix, but it needs a rooted device (which I haven t done till now my failure!). I can only hope that Google fixes this completely stupid problem in a future version. The Me problem Another of these beasty problems: The Android Contacts application has an entry for Me , which unfortunately, no idea why, cannot be linked with my normal me in the list of contacts. There are reports all over the Internet, strange suggestions, and no real solution. Again, a simple thing that should work but doesn t. Invisible Images folder in MTP mode A more annoying problem is that the camera folder under Photos does not show up when connecting the device in MTP mode to my computer, and as consequence me being unable to copy photos from the device to my computer. The solution I am using at the moment is moving the photos with a file manager to a new folder which is visible during MTP communcation, and copy the photos from there. But this, too, should be something trivial, but alas, despite a lot of posts on the internet I couldn t find a proper solution. Google Music As written somewhere else, Google Music has switched from 5 star system to up/down system, which is a huge pain and PITA. Things I do like (or I discovered) on Android There are some things I haven t been used/tried on iOS they might be possible which I really like: Yubikey Neo support I will write about this in a different blog, but nowadays I have my GPG keys on an hardware token (Yubikey Neo) and the application OpenKeychain on Android works nicely with both K9 Mail and via NFC with my Yubikey. That is a great tool! Bluetooth streaming Bluetooth on iOS devices was always a bit broken for me, so connecting my phone to my old car radio I needed radio transmitter that was connected to the cable port of the iPhone. With Android I use a Bluetooth Radio device (receives data via bluetooth, and sends music out via radio waves for a car stereo to receive them). Now if my monthly data limit wouldn t be that low Debian on Android Yes, you can have a full Debian system running in your terminal on Android. There are several applications providing this feature, and I am rather surprised how smooth it works. Conclusion My preliminary conclusion is that the switch to Android at this time was perfectly timed, and from the technological side I should have done much earlier. In future blogs I will discuss particular instances of this transition in more details. If you have any suggestion for me, in particular for a good notes taking application, please let me know!

This is the story of libspotify, as experienced by a Spotify customer and libspotify developer for six years.

Step 1: Support your API for years Since April 2009, libspotify has been a mostly nice although proprietary C library for integrating with the Spotify music streaming service, providing both music metadata and full playback capabilities. Language wrappers have been written for a plenitude of programming languages (including my own pyspotify). libspotify is integrated into numerous open source projects (including my own Mopidy), networked AV receivers, and rumour has it: even cars. In addition to being closed source, there was another catch: in order to use it, you need the Spotify premium subscription. Speaking from my own experience supporting a project integrating with Spotify for the last six years: lots of end users upgraded to premium in order to use the projects built on top of libspotify.

Step 2: Stop accepting bug reports and stop releasing updates In May 2012, Spotify released libspotify 12.1.51 for all supported architectures, which now even included Android. After months of endless requests from the quickly growing hoards of Raspberry Pi users, Spotify released libspotify 12.1.103 for hardfloat armv6 on January 22 2013. The release included minor API additions, like the addition of explicit lyrics and means for accepting Spotify s Terms of Service. The additions in 12.1.103 never reached the other supported architectures and 12.1.51 remains the last release for those. There was no clear communication about this being the end of libspotify, releases have just ceased. libspotify has not been mentioned in a single developer blog post or developer email newsletter since. That s now more than three years ago.

Step 3: Create new APIs Over the next couple of years Spotify released a new Web API and iOS and Android SDKs. Through their developer blog, they communicated quite a bit both about these and about the deprecation of the apps inside the Spotify desktop client. Here are some pieces from their developer blog to illustrate:

• 2014-02-03: Deprecation of Spotify Apps API 0.x
• 2014-03-06: First Spotify iOS SDK beta release
• 2014-03-24: No longer accepting new Spotify Apps
• 2014-06-17: First Spotify Web API release
• 2014-06-25: First Spotify Android SDK beta release

The Web API grew steadily, as documented by the Web APIs changelog. Over time, its feature list improved, supporting anonymous endpoints for generic music metadata as well as OAuth protected endpoints for managing your own music collection and playlists. The only major part missing that prevented developers replacing libspotify was music playback, as the Web API only provides 30 second previews. From what I recall of 2014 IRC discussions with Spotify employees, they had built a new much smaller library for streaming music. The iOS and Android SDKs were simply platform specific wrappers around this new playback library and the new Web API. My impression is that, after a number of beta releases, the iOS and Android SDKs seamlessly replaced libspotify on those platforms. The plan was apparently to release the new playback library for desktop too, so that it, in combination with the Web API, could replace libspotify there as well. That was all back in 2014.

Step 4: Deprecate old APIs At some point, Spotify deprecated their old web API, the Metadata API. There was no blog post about it and it was not mentioned in any Spotify Developer News email I ve received in recent years. I guess they only published this fact as a note on the Metadata API web page; it is therefore hard to track down exactly when it was deprecated and how the deprecation was communicated. Anyway, it was deprecated, and the natural upgrade path was to the new Web API. Spotify even has a migration guide mapping the concepts of the Metadata API to the new Web API. All good. In May 2015, after 28 months without a single release or any news, libspotify is officially deprecated. The communication channel of choice? A note on the libspotify web page. No post on the developer blog. No Spotify Developer News email. To be able to use libspotify, all developers have registered and requested a personal application key. To my knowledge, none of these registered library users were notified directly.

Step 5: Shut down an old API On January 20 2016 Spotify end-of-lifed their Metadata API. I don t know if this date was communicated on the Metadata API web page well in advance of this shutdown. There s no blog post on the topic. There s no news email. There s no tweet from @SpotifyPlatform or @SpotifyStatus. I can t find anything except users commenting on various Spotify documentation pages that the text is outdated because the API is no longer merely deprecated, but entirely shut down. Hopefully they did publish the shutdown date ahead of time. It had a full replacement API and a migration guide. Fair and square. Nothing much to complain about here. It s time to get porting your code to the Web API! But frankly, I don t really care about this Metadata API because I don t use the Metadata API, do I?

Step 6: Break your API without warning February 3 2016, Mopidy users start reporting General transient error when searching Spotify. This is a libspotify error message known as SP_ERROR_OTHER_TRANSIENT with error code 16. This error usually means that there is some unspecified issue with the Spotify service. We ve actually seen it before and like its name suggests, it is indeed transient and soon goes away. According to a series of tweets between my fellow Mopidy developer Nick Steel and @SpotifyCares it appears that the libspotify search functionality was backed by the Metadata API s search functionality. Who knew? I m not aware of any relation between these APIs ever being mentioned publicly. No libspotify user could possibly have known that the Metadata API shutdown would affect them. Let s summarize: libspotify hasn t had any releases for three years and it was officially deprecated in May last year. But, there have been no warnings about any shutdown. The libspotify project is likely no longer staffed at all. Did someone just forget that libspotify depended on the Metadata API they were shutting down? Many newer networked AV receivers use the Spotify Connect libraries that are exclusive to Spotify s commercial partners. But some receivers probably still use libspotify. If so, did Spotify just break hardware sold with Spotify support as a major feature? Something that people are paying a monthly fee to use? It seems they did! In the Spotify Community forums there are numerous reports of broken devices and software that all have two things in common: the date the problem started occurring, and that search is the only feature that is broken.

• Denon receivers, including models: CEOL N8
• Logitech Squeezebox media players
• Marantz receivers, including models: M-CR510, M-CR610, NR1504, NR1603, SR5007, SR6008
• Onkyo receivers, including CR-N755, TX-NR509, TX-NR609, TX-8050
• Roku media players

Surely, Spotify will quickly explain themselves and fix this?

Step 7: Stay quiet Since the API shutdown, Spotify has been very quiet. No posts on the developer blog. No email newsletter. No tweets from @SpotifyPlatform since December 1 last year. Some users at The Spotify Community forum are quoting responses from Spotify. Most responses seem to link to the Metadata API migration docs and state that application developers must migrate. In other cases the responses point the finger at the commercial partner which only adds to the confusion. Some of these devices may use the Metadata API directly, and in those cases the application maintainers are probably to blame. However, I m quite certain that the libspotify search issue is responsible for a fair share of these complaints. libspotify, in contrast to the Metadata API, has never been officially shut down. The responsibility for any breakage caused by libspotify is Spotify s alone.

What I want from Spotify

1. Apologize to your paying customers and supporters.
2. Get substantially better at communicating deprecations and shutdowns. We want all news, not just good news.
3. Give us a new supported C library for audio playback. Now! Not later this year. I ve heard that promise for three consecutive years now. We can live with pre-release software quality initially. Getting the library out sooner rather than later shows us that there is a road ahead for all the projects that stream music from Spotify. Let us get started making new language wrappers.
4. Release Spotify Connect specifications and libraries. We want to make music players that can be controlled by the Spotify mobile app. We want to make music controllers that can play music on all the Spotify Connect software and hardware players we ve invested in. You ve created this technology, now let us use it.

Thanks to Nick Steel for reviewing this blog post.

During the last days I've completed quite long time challenge - make Weblate compatible with Python 3. It's probably still not in shape you would use it on production server, but it runs fine and the testsuite completes without errors. As usual, the major obstacle are dependencies. The translate-toolkit doesn't support Python 3 in any released version, so you need latest Git snapshot. Another problematic bit is Mercurial, which will not support Python 3 in near future. But in case you need it, you can still install Python 2 version and use it from Weblate (it uses command line interface). The porting was mostly dealing with encoding mess - in Python 2 you really didn't have to distinguish between binary and text data, while Python 3 quite enforces that. This turned out to be especially tricky with CSV files where csv module expects text streams in Python 3, while translate-toolkit CSV parser expects binary ones. All this has one downside as well - the test matrix got bigger again, what makes the testsuite run longer with 5 workers available. While doing these changes, I've also decided to drop support for Django 1.7, this will allow us to get rid of lot of deprecation warnings in Django 1.9 and prepare codebase for Django 1.10.

Filed under: English Weblate 0 comments

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In December, 113.50 work hours have been dispatched among 9 paid contributors. Their reports are available:

• Antoine Beaupr did 11.5h.
• Ben Hutchings did 15 hours (out of 15 hours allocated + 10 extra hours remaining, meaning that he still has 10 extra hours to do over February).
• Chris Lamb did 18 hours.
• Guido G nther did 13 hours (out of 12 hours allocated + 1 remaining).
• Mike Gabriel did 16 hours (8 hours allocated + 8h remaining).
• Santiago Ruano Rinc n did 18 hours (out of 12h allocated + 8.50 remaining, thus keeping 2.50 extra hours for February).
• Scott Kitterman did 8 hours.
• Thorsten Alteholz did 30 hours.

Evolution of the situation As expected, we had a small drop in the amount of hours sponsored. New sponsors (re-)joined but others stopped too (Gree this time) mostly balancing the result. We only lost 2 hours of sponsored work. It would be nice if we could invert that curve and actually start again to get closer to our objective of funding the equivalent of a full time position. Let s hope that the switch to wheezy as the version supported by the LTS team will motivate many companies relying on Debian 7 in their IT system. In terms of security updates waiting to be handled, the situation is close to last month(17 packages in dla-needed.txt, 27 in the list of CVE). It looks like that having about 20 packages needing an update is the normal situation and that we can t really get further down given the time required to process some updates (sometimes we wait until the upstream authors provides a patch, and so on). Thanks to our sponsors New sponsors are in bold.

• Platinum sponsors:
• TOSHIBA (for 4 months)
• Gold sponsors:
• The Positive Internet (for 20 months)
• Blablacar (for 19 months)
• Linode LLC (for 9 months)
• Silver sponsors:
• Domeneshop AS (for 19 months)
• Universit Lille 3 (for 19 months)
• Trollweb Solutions (for 17 months)
• Nantes M tropole (for 13 months, renewal after a small break)
• University of Luxembourg (for 11 months)
• Dalenys (for 9 months)
• Univention GmbH (for 5 months)
• Universit Jean Monnet de St Etienne (for 5 months)
• Bronze sponsors:
• David Ayers IntarS Austria (for 20 months)
• Offensive Security (for 20 months)
• Seznam.cz, a.s. (for 20 months)
• Evolix (for 19 months)
• Freeside Internet Service (for 19 months)
• MyTux (for 19 months)
• Linuxhotel GmbH (for 17 months)
• Intevation GmbH (for 16 months)
• Daevel SARL (for 15 months)
• Bitfolk LTD (for 14 months)
• Megaspace Internet Services GmbH (for 14 months)
• Greenbone Networks GmbH (for 13 months)
• NUMLOG (for 13 months)
• WinGo AG (for 12 months)
• Ecole Centrale de Nantes LHEEA (for 8 months)
• Sig-I/O (for 6 months)
• Entr ouvert (for 4 months)
• Adfinis SyGroup AG

No comment Liked this article? Click here. My blog is Flattr-enabled.

This was my ninth month as a Freexian sponsored LTS contributor. I was assigned 8 hours for the month of January. My time this month was spent preparing updates for clamav and the associated libclamunrar for squeeze and wheezy. For wheezy, I ve only helped a little, mostly I worked on squeeze. This update is more complex than usual because with clamav 0.99 upstream bumped soname and so in addition to the normal case of transitions in unstable, we ve needed transitions for stable, oldstable, and squeeze-lts. We also try to be careful and maintain higher versions in newer releases, so stable needed to wait for 0.99 in testing, oldstable needed to wait for stable, etc. Currently 0.99 is in stable proposed updates and I ve requested that the update for wheezy (oldstable) go forward. Once that s done, I ve got squeeze-lts ready to go.

Debian LTS January was the ninth month I contributed to Debian LTS under the Freexian umbrella. In total I spent 13 hours working on:

• LTS Frontdesk duties like the triaging of 34 CVEs. That was about twice as much CVEs coming as during December's frontdesk work.
• I looked into what needs to be done DLA wise when we move from Squeeze to Wheezy. For that I added a script do find discrepancies between Squeeze LTS and Wheezy.
• I uploaded giflib to squeeze-lts(DLA-389-1), wheezy(#812363) and jessie(#812362) proposed updates.
• I forward ported the fix for CVE-2015-5291 for polarssl to Wheezy adding autopkgtests on the way (#812420) (forward port to Jessie happend in February)
• I forward ported a patch for freetype fixing CVE-2014-9674 to wheezy(DSA-3461-1) - Jessie being not affected.
• Finally I added some basic autopkgtests to icu (#813338).

There was no progress on using the same nss in all suites. This will continue in February as does the Squeeze-lts Wheezy forward porting. Other Debian stuff

• I uploaded libvirt 1.3.1~rc1 and 1.3.1~rc2 to experimental and 1.3.1 final to unstable.
• Released git-buildpackage 0.7.2

Johannes Bechberger, while working on his Bachelor s thesis supervised by my colleague Andreas Zwinkau, has developed a performance benchmark runner and results visualizer called temci , and used GHC as a guinea pig. You can read his elaborate analysis on his blog. This is particularly interesting given recent discussions about GHC itself becoming slower and slower, as for example observed by Johannes Waldmann and Anthony Cowley. Johannes Bechberger s take-away is that, at least for the programs at hand (which were taken from the The Computer Language Benchmarks Game, there are hardly any changes worth mentioning, as most of the observed effects are less than a standard deviation and hence insignificant. He tries hard to distill some useful conclusions from the data; the one he finds are:

• Compile time does not vary significantly.
• The compiler flag -O2 indeed results in faster code than -O.
• With -O (but not -O2), GHC 8.0.1 is better than GHC 7.0.1. Maybe some optimizations were promoted to -O?

If you are interested, please head over to Johannes s post and look at the gory details of the analysis and give him feedback on that. Also, maybe his tool temci is something you want to try out? Personally, I find it dissatisfying to learn so little from so much work, but as he writes: It s so easy to lie with statistics. , and I might add lie to yourself , e.g. by ignoring good advise about standard deviations and significance. I m sure my tool gipeda (which powers perf.haskell.org) is guilty of that sin. Maybe a different selection of test programs would yield more insight; the benchmark s games programs are too small and hand-optimized, the nofib programs are plain old and the fibon collection has bitrotted. I would love to see a curated, collection of real-world programs, bundled with all dependencies and frozen to allow meaningful comparisons, but updated to a new, clearly marked revision, on a maybe bi-yearly basis maybe Haskell-SPEC-2016 if that were not a trademark infringement.

In the last week of January, the project IT-Zukunft Schule (Logo EDV-Systeme GmbH and DAS-NETZWERKTEAM) had visitors from Norway: Klaus Ade Johnstad and Linnea Skogtvedt from LinuxAvdelingen [1] came by for exchanging insights, knowledge, technology, stories regarding IT services at school in Norway and Nothern Germany. This was our schedule... Tuesday

• 3pm Arrival of Klaus Ade and Linnea, meet up at LOGO with coffee and cake
• 4pm Planning the workshop, coming up with an agenda for the next two days (Klaus Ade, Andreas, Mike)
• 5pm Preparing OPSI demo sites (Mike, Linnea)
• 8pm Gr nkohl and Rotkohl and ... at Traum GmbH, Kiel (Klaus Ade, Linnea, Andreas, Mike)

Wednesday

• 8.30am more work on the OPSI demo site (Mike, Linnea)
• 10am pfSense (esp. captive portal functionality), backup solutions (Klaus Ade, all)
• 11am ITZkS overlay packages, basic principles of Debian packaging (Mike, special guests: Torsten, Lucian, Benni)
• 12am-2pm lunch break
• 2pm OPSI demonstration, discussion, foundation of the OpsiPackages project [2] (Mike)
• 4pm Puppet (Linnea)
• 7pm dinner time (eating in, Thai fast food :-) )
• 20pm Sepiida hacking (Mike, Linnea), customer care (Andreas, Klaus Ade)
• 22:30pm zZzZzZ time...

Thursday

read more

What happened in the reproducible builds effort this week:

Toolchain fixes After remarks from Guillem Jover, Lunar updated his patch adding generation of .buildinfo files in dpkg.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dracut, ent, gdcm, guilt, lazarus, magit, matita, resource-agents, rurple-ng, shadow, shorewall-doc, udiskie. The following packages became reproducible after getting fixed:

• disque/1.0~rc1-5 by Chris Lamb, noticed by Reiner Herrmann.
• dlm/4.0.4-2 by Ferenc W gner.
• drbd-utils/8.9.6-1 by Apollon Oikonomopoulos.
• java-common/0.54 by by Emmanuel Bourg.
• libjibx1.2-java/1.2.6-1 by Emmanuel Bourg.
• libzstd/0.4.7-1 by Kevin Murray.
• python-releases/1.0.0-1 by Jan Dittberner.
• redis/2:3.0.7-2 by Chris Lamb, noticed by Reiner Herrmann.
• tetex-brev/4.22.github.20140417-3 by Petter Reinholdtsen.

Some uploads fixed some reproducibility issues, but not all of them:

• anarchism/14.0-4 by Holger Levsen.
• hhvm/3.11.1+dfsg-1 by Faidon Liambotis.
• netty/1:4.0.34-1 by Emmanuel Bourg.

Patches submitted which have not made their way to the archive yet:

• #813309 on lapack by Reiner Herrmann: removes the test log and sorts the files packed into the static library locale-independently.
• #813345 on elastix by akira: suggest to use the $datetime placeholder in Doxygen footer.
• #813892 on dietlibc by Reiner Herrmann: remove gzip headers, sort md5sums file, and sort object files linked in static libraries.
• #813912 on git by Reiner Herrmann: remove timestamps from documentation generated with asciidoc, remove gzip headers, and sort md5sums and tclIndex files.

reproducible.debian.net For the first time, we've reached more than 20,000 packages with reproducible builds for sid on amd64 with our current test framework. Vagrant Cascadian has set up another test system for armhf. Enabling four more builder jobs to be added to Jenkins. (h01ger)

Package reviews 233 reviews have been removed, 111 added and 86 updated in the previous week. 36 new FTBFS bugs were reported by Chris Lamb and Alastair McKinstry. New issue: timestamps_in_manpages_generated_by_yat2m. The description for the blacklisted_on_jenkins issue has been improved. Some packages are also now tagged with blacklisted_on_jenkins_armhf_only.

Misc. Steven Chamberlain gave an update on the status of FreeBSD and variants after the BSD devroom at FOSDEM 16. He also discussed how jails can be used for easier and faster reproducibility tests. The video for h01ger's talk in the main track of FOSDEM 16 about the reproducible ecosystem is now available.

TL;DR All Debian license and patches are belong to us. Discover them here and here. In case you hadn't already stumbled upon sources.debian.net in the past, Debsources is a simple web application that allows to publish an unpacked Debian source mirror on the Web. On the live instance you can browse the contents of Debian source packages with syntax highlighting, search files matching a SHA-256 hash or a ctag, query its API, highlight lines, view accurate statistics and graphs. It was initially developed at IRILL by Stefano Zacchiroli and Matthieu Caneill. During GSOC 2015 I helped introduce two new features. License Tracker Since Debsources has all the debian/copyright files and that many of them adopted the DEP-5 suggestion (machine readable copyright files) it was interesting to exploit them for end users. You may find interesting the following features:

• an API that allows users to find the license of file "foo" or the licenses for a bunch of packages, using filenames or SHA-256 hashes
• a better looking interface for debian/copyright files

Have a look at the documentation to discover more! Patch tracker The old patch tracker unfortunately died a while ago. Since Debsources stores all the patches it was, probably, natural for it to be able to exploit them and present them over the web. You can navigate through packages by prefix or by searching them here. Among the use cases:

• a summary which contains all the patches of a package together with their diffs and summaries/subjects
• links to view and download (quilt-3.0) patches.

Read more about the API! Coming ...

• In the future these informations will be added in the DB. This will allow:
  • the license tracker to provide interesting statistics and graphs about licensing trends (What do Pythonistas usually choose as a license, how many GPL-3 files are in Jessie etc). Those are going to be quite accurate since they will take into account each file in a given package and not just the "general" license of the package.
  • the patch tracker to produce a list of packages that contain patches - this will enable providing links from PTS to the patch tracker.
• Not far in the horizon there is also an initial work for exporting debian/copyright files into SPDX documents. You can have a look at a beta / testing version on debsources-dev. (Example)

I hope you find these new features useful. Don't hesitate to report any bugs or suggestions you come accross.

I haven t yet blogged about this (as normally I don t blog often), but I joined Collabora in June last year. Since then, I had an opportunity to work with OpenEmbedded again, write a kernel patch, learn lots of things about systemd (in particular, how to stop worrying about it taking over the world and so on), and do lots of other things. As one would expect when working for a free software consultancy, our customers do understand the value of the community and contributing back to it, and so does the customer for the project I m working on. In fact, our customer insists we keep the number of locally applied patches to, for example, Linux kernel, to minimum, submitting as much as possible upstream. However, apart from the upstreaming work which may be done for the customer, Collabora encourages us, the engineers, to spend up to two hours weekly for upstreaming on top of what customers need, and up to five days yearly as paid Community days. These community days may be spent working on the code or doing volunteering at free software events or even speaking at conferences. Even though on this project I have already been paid for contributing to the free software project which I maintained in my free time previously (ifupdown), paid community time is a great opportunity to contribute to the projects I m interested in, and if the projects I m interested in coincide with the projects I m working with, I effectively can spend even more time on them. A bit unfortunately for me, I haven t spent enough time last year to plan my community days, so I used most of them in the last weeks of the calendar year, and I used them (and some of my upstreaming hours) on something that benefitted both free software community and Collabora. I m talking about SparkleShare, a cross-platform Git-based file synchronisation solution written in C#. SparkleShare provides an easy to use interface for Git, or, actually, it makes it possible to not use any Git interface at all, as it monitors the working directory using inotify and commits stuff right after it changes. It automatically handles conflicts even for binary files, even though I have to admit its handling could still be improved. At Collabora, we use SparkleShare to store all sorts of internal documents, and it s being used by users not familiar with command line interfaces too. Unfortunately, the version we recently had in Debian had a couple of very annoying bugs, making it a great pain to use it: it would not notice edits in local files, or not notice new commits being pushed to the server, and that led to individual users edits being lost sometimes. Not cool, especially when the document has to be sent to the customer in a couple of minutes. The new versions, 1.4 (and recently released 1.5) was reported as being much better and also fixing some crashes, but it also used GTK+ 3 and some libraries not yet packaged for Debian. Thanh Tung Nguyen packaged these packages (and a newer SparkleShare) for Ubuntu and published them in his PPA, but they required some work to be fit for Debian. I have never touched Mono packages before in my life, so I had to learn a lot. Some time was spent talking to upstream about fixing their copyright statements (they had none in the code, and only one author was mentioned in configure.ac, and nowhere else in the source), a bit more time went into adjusting and updating the patches to the current source code version. Then, of course, waiting the packages to go through NEW. Fixing parallel build issues, waiting for buildds to all build dependencies for at least one architecture But then, finally, on 19^th of January I had the updated SparkleShare in Debian. As you may have already guessed, this blog post has been sponsored by Collabora, the first of my employers to encourage require me to work on free software in my paid time :)

In January I carried over 10 hours from December and was assigned another 15 hours of work by Freexian's Debian LTS initiative. I worked a total of 15 hours. I had a few days on 'front desk' at the start of the month, as my week in that role spanned the new year. I fixed a regression in the kernel that was introduced to all stable suites in December. I uploaded this along with some minor security fixes, and issued DLA 378-1. I finished backporting and testing fixes to sudo for CVE-2015-5602. I uploaded an update and issued DLA 382-1, which was followed by DSA 3440-1 for wheezy and jessie. I finished backporting and testing fixes to Claws Mail for CVE-2015-8614 and CVE-2015-8708. I uploaded an update and issued DLA 383-1. This was followed by DSA 3452-1 for wheezy and jessie, although the issues are less serious there. I also apent a little time on InspIRCd, though this isn't a package that Freexian's customers care about and it seems to have been broken in squeeze for several years due to a latent bug in the build system. I had already backported the security fix by the time I discovered this, so I went ahead with an update fixing that regression as well, and issued DLA 384-1. Finally, I diagnosed the regression in the update to isc-dhcp in DLA 385-1.

I had some pile of hosting requests in queue since half of January and my recent talk on FOSDEM had some impact on requests for hosting translations as well, so it's about time to process them. New kids on the block are:

• DSC Auto Rename
• Openbmap / radiocells.org
• Lexica
• Transistor

Second round includes:

• OpenBikeSharing
• Binary Tick Trade App

Unfortunately I had to reject some projects as well mostly due to lack of file format support. This is still the same topic - when translating project, please stick with some standard format. Preferably what is usual on your platform. If you like this service, you can support it on Bountysource salt or Gratipay. There is also option for hosting translations of commercial products.

Filed under: English Weblate 0 comments

What happened in the reproducible builds effort between January 24th and January 30th:

Media coverage Holger Levsen was interviewed by the FOSDEM team to introduce his talk on Sunday 31st.

Toolchain fixes Jonas Smedegaard uploaded d-shlibs/0.63 which makes the order of dependencies generated by d-devlibdeps stable accross locales. Original patch by Reiner Herrmann.

Packages fixed The following 53 packages have become reproducible due to changes in their build dependencies: appstream-glib, aptitude, arbtt, btrfs-tools, cinnamon-settings-daemon, cppcheck, debian-security-support, easytag, gitit, gnash, gnome-control-center, gnome-keyring, gnome-shell, gnome-software, graphite2, gtk+2.0, gupnp, gvfs, gyp, hgview, htmlcxx, i3status, imms, irker, jmapviewer, katarakt, kmod, lastpass-cli, libaccounts-glib, libam7xxx, libldm, libopenobex, libsecret, linthesia, mate-session-manager, mpris-remote, network-manager, paprefs, php-opencloud, pisa, pyacidobasic, python-pymzml, python-pyscss, qtquick1-opensource-src, rdkit, ruby-rails-html-sanitizer, shellex, slony1-2, spacezero, spamprobe, sugar-toolkit-gtk3, tachyon, tgt. The following packages became reproducible after getting fixed:

• angband-doc/3.0.3.6 by Manoj Srivastava, obsolete patch by Chris Lamb.
• atdgen/1.7.2-1 by St phane Glondu.
• bibtool/2.63+ds-1 by Jerome Benoit.
• cglib/3.2.0-1 by Emmanuel Bourg.
• cmst/2016.01.28-1 by Alf Gaida.
• coreutils/8.25-1 uploded by Michael Stone, fixed upstream.
• doc-base/0.10.7 uploaded by Robert Luberda, original patch by Dhole.
• fpc/3.0.0+dfsg-1 uploaded by Paul Gevers.
• libaqbanking/5.6.4beta-1 by Micha Lenk.
• libgcrypt20/1.6.4-5 uploaded by Andreas Metzler, original patch by Lunar.
• libgwenhywfar/4.15.2beta-1 by Micha Lenk.
• libxdmcp/1:1.1.2-1.1 by Helmut Grohne (report).
• lpe/1.2.8-2 by Adam Majer, obsolete patches (#778197, #793697 by Chris Lamb and akira.
• mariadb-10.0/10.0.23-2 by Otto Kek l inen.
• mixxx/2.0.0~dfsg-1 by Sebastian Ramacher.
• pd-lua/0.7.3-1 by IOhannes m zm lnig.
• pd-zexy/2.2.6-2 by IOhannes m zm lnig.
• polymake/3.0-1 uploaded by David Bremner, fixed upstream.
• prometheus/0.16.2+ds-1 by Mart n Ferrari.
• screengrab/1.95+20160128-1 uploaded by Alf Gaida (report).
• spykeviewer/0.4.4-1 by Robert Pr pper, original patch by Reiner Herrmann.
• testdisk/7.0-1 uploaded by Roland Stigge, upstream patch reported by Mattia Rizzolo.
• xorg/1:7.7+13 uploaded by Timo Aaltonen, original patch by Dhole, merged by Andreas Boll.

Some uploads fixed some reproducibility issues, but not all of them:

• gnubg/1.05.000-4 by Russ Allbery.
• grcompiler/4.2-6 by Hideki Yamane.
• sdlgfx/2.0.25-5 fix by Felix Geyer, uploaded by Gianfranco Costamagna.

Patches submitted which have not made their way to the archive yet:

• #812876 on glib2.0 by Lunar: ensure that functions are sorted using the C locale when giotypefuncs.c is generated.

diffoscope development diffoscope 48 was released on January 26th. It fixes several issues introduced by the retrieval of extra symbols from Debian debug packages. It also restores compatibility with older versions of binutils which does not support readelf --decompress.

strip-nondeterminism development strip-nondeterminism 0.015-1 was uploaded on January 27th. It fixes handling of signed JAR files which are now going to be ignored to keep the signatures intact.

Package reviews 54 reviews have been removed, 36 added and 17 updated in the previous week. 30 new FTBFS bugs have been submitted by Chris Lamb, Michael Tautschnig, Mattia Rizzolo, Tobias Frost.

Misc. Alexander Couzens and Bryan Newbold have been busy fixing more issues in OpenWrt. Version 1.6.3 of FreeBSD's package manager pkg(8) now supports SOURCE_DATE_EPOCH. Ross Karchner did a lightning talk about reproducible builds at his work place and shared the slides.

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS I did not ask for any paid hours this month and won t be requesting paid hours for the next 5 months as I have a big project to handle with a deadline in June. That said I still did a few LTS related tasks:

• I uploaded a new version of debian-security-support (2016.01.07) to officialize that virtualbox-ose is no longer supported in Squeeze and that redmine was not really supportable ever since we dropped support for rails.
• Made a summary of the discussion about what to support in wheezy and started a new round of discussions with some open questions. I invited contributors to try to pickup one topic, study it and bring the discussion to some conclusion.
• I wrote a blog post to recruit new paid contributors. Brian May, Markus Koschany and Damyan Ivanov candidated and will do their first paid hours over February.

Distro Tracker Due to many nights spent on playing Splatoon (I m at level 33, rank B+, anyone else playing it?), I did not do much work on Distro Tracker. After having received the bug report #809211, I investigated the reasons why SQLite was no longer working satisfactorily in Django 1.9 and I opened the upstream ticket 26063 and I had a long discussion with two upstream developers to find out the best fix. The next point release (1.9.2) will fix that annoying regression. I also merged a couple of contributions (two patches from Christophe Siraut, one adding descriptions to keywords, cf #754413, one making it more obvious that chevrons in action items are actionable to show more data, a patch from Balasankar C in #810226 fixing a bad URL in an action item). I fixed a small bug in the unsubscribe command of the mail bot, it was not properly recognizing source packages. I updated the task notifying of new upstream versions to use the data generated by UDD (instead of the data generated by Christoph Berg s mole-based implementation which was suffering from a few bugs). Debian Packaging Testing experimental sbuild. While following the work of Johannes Schauer on sbuild, I installed the version from experimental to support his work and give him some feedback. In the process I uncovered #810248. Python sponsorship. I reviewed and uploaded many packages for Daniel Stender who keeps doing great work maintaining prospector and all its recursive dependencies: pylint-common, python-requirements-detector, sphinx-argparse, pylint-django, prospector. He also prepared an upload of python-bcrypt which I requested last month for Django. Django packaging. I uploaded Django 1.8.8 to jessie-backports.
My stable updates for Django 1.7.11 was not handled before the release of Debian 8.3 even though it was filed more than 1.5 months before. Misc stuff. My stable update for debian-handbook has been accepted fairly shortly after my last monthly report (thank you Adam!) so I uploaded the package once acked by a release manager. I also sponsor a backports upload of zim prepared by Joerg Desch. Kali related work Kernel work. The switch to Linux 4.3 in Kali resulted in a few bug reports that I investigated with the help of #debian-kernel and where I reported my findings back so that the Debian kernel could also benefit from the fixes I uploaded to Kali: first we included a patch for a regression in the vmwgfx video driver used by VMWare virtual machines (which broke the gdm login screen), then we fixed the input-modules udeb to fix support of some Logitech keyboards in debian-installer (see #796096). Misc work. I made a non-maintainer upload of python-maxminddb to fix #805689 which had been removed from stretch and that we needed in Kali. I also had to NMU libmaxminddb since it was no longer available on armel and we actually support armel in Kali. During that NMU, it occurred to me that dh-exec could offer a feature of optional install , that is installing a file that exists but not failing if it doesn t exist. I filed this as #811064 and it stirred up quite some debate. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.