One of my clients has a NAS device. Last week they tried to do what should have been a routine RAID operation, they added a new larger disk as a hot-spare and told the RAID array to replace one of the active disks with the hot-spare. The aim was to replace the disks one at a time to grow the array. But one of the other disks had an error during the rebuild and things fell apart. I was called in after the NAS had been rebooted when it was refusing to recognise the RAID. The first thing that occurred to me is that maybe RAID-5 isn t a good choice for the RAID. While it s theoretically possible for a RAID rebuild to not fail in such a situation (the data that couldn t be read from the disk with an error could have been regenerated from the disk that was being replaced) it seems that the RAID implementation in question couldn t do it. As the NAS is running Linux I presume that at least older versions of Linux have the same problem. Of course if you have a RAID array that has 7 disks running RAID-6 with a hot-spare then you only get the capacity of 4 disks. But RAID-6 with no hot-spare should be at least as reliable as RAID-5 with a hot-spare. Whenever you recover from disk problems the first thing you want to do is to make a read-only copy of the data. Then you can t make things worse. This is a problem when you are dealing with 7 disks, fortunately they were only 3TB disks and only each had 2TB in use. So I found some space on a ZFS pool and bought a few 6TB disks which I formatted as BTRFS filesystems. For this task I only wanted filesystems that support snapshots so I could work on snapshots not on the original copy. I expect that at some future time I will be called in when an array of 6+ disks of the largest available size fails. This will be a more difficult problem to solve as I don t own any system that can handle so many disks. I copied a few of the disks to a ZFS filesystem on a Dell PowerEdge T110 running kernel 3.2.68. Unfortunately that system seems to have a problem with USB, when copying from 4 disks at once each disk was reading about 10MB/s and when copying from 3 disks each disk was reading about 13MB/s. It seems that the system has an aggregate USB bandwidth of 40MB/s slightly greater than USB 2.0 speed. This made the process take longer than expected. One of the disks had a read error, this was presumably the cause of the original RAID failure. dd has the option conv=noerror to make it continue after a read error. This initially seemed good but the resulting file was smaller than the source partition. It seems that conv=noerror doesn t seek the output file to maintain input and output alignment. If I had a hard drive filled with plain ASCII that MIGHT even be useful, but for a filesystem image it s worse than useless. The only option was to repeatedly run dd with matching skip and seek options incrementing by 1K until it had passed the section with errors. for n in /dev/loop[0-6] ; do echo $n ; mdadm examine -v -v scan $n grep Events ; done Once I had all the images I had to assemble them. The Linux Software RAID didn t like the array because not all the devices had the same event count. The way Linux Software RAID (and probably most RAID implementations) work is that each member of the array has an event counter that is incremented when disks are added, removed, and when data is written. If there is an error then after a reboot only disks with matching event counts will be used. The above command shows the Events count for all the disks. Fortunately different event numbers aren t going to stop us. After assembling the array (which failed to run) I ran mdadm -R /dev/md1 which kicked some members out. I then added them back manually and forced the array to run. Unfortunately attempts to write to the array failed (presumably due to mismatched event counts). Now my next problem is that I can make a 10TB degraded RAID-5 array which is read-only but I can t mount the XFS filesystem because XFS wants to replay the journal. So my next step is to buy another 2*6TB disks to make a RAID-0 array to contain an image of that XFS filesystem. Finally backups are a really good thing

Thorsten Glaser reported Debian bug #780000 on Saturday March 7th 2015, against the gcc-4.9 package. Bug #770000 was reported as of November 18th so there have been 10,000 bugs in about 3.5 months, which was significantly slower than earlier. Matthew Vernon reported Debian bug #790000 on Friday June 26th 2015, against the pcre3 package. Thus, there have been 10,000 bugs in 3.5 months again. It seems that the bug report rate stabilized again. Sorry for missing bug #780000 annoucement. I'm doing this since....November 2007 for bug #450000 and it seems that this lack of attention is somehow significant wrt my involvment in Debian. Still, this involvment is still here and I'll try to "survive" in the project until we reach bug #1000000...:-) See you for bug #800000 annoucement and the result of the bets we placed on the date it would happen.

Here we go, I just uploaded 15 packages to the Debian archive that brings TeX Live in Debian up to the 2015 release (and a bit newer)! Uploaded packages are asymptote, biber, context, context-modules, jadetex, musixtex, pmx, tex-common, texinfo, texinfo-doc-nonfree, texlive-base, texlive-bin, texlive-extra, texlive-lang, xmltex. The packages are basically what has been in experimental for quite some time, plus a checkout of tlnet from yesterday. For details on the changes and the new packaging, please consult this post. So, now let the flood of bug reports begin, but in the mean time, enjoy!

TeX Live 2015 has been released, and normal operation with daily updates has started. During the freeze time and afterwards I have made a few changes to the TeX Live Manager (tlmgr) that I want to highlight here. The main changes are better error and return code handling (which should be hardly visible for the users), and more more informative output of the tlmgr info action, incorporating more data from the TeX Catalogue. Error handling With a program that started as an experiment that has grown into the central configuration and management program, there are lots of old code pieces that did not do proper error signaling via return values. That meant that the return value of a tlmgr run didn t have any meaning, mostly because it was 0 (success) most of the times. I have now tried to do proper return code handling throughout the tlmgr code base, that is the tlmgr.pl and the necessary Perl modules. While this should not be a user visible changes, it turned out that the MacOS TeX Live Utility by Adam Maxwell (btw, a great program, it would be nice to have something similar written for Unix replacing the bit clumsy tlmgr gui), got broken for paper configuration, due to forgotten return value fixes in the TLPaper.pm module. That is fixed now in our repository. All in all we do hope that the return value of a tlmgr run now gives proper information about success or error. I might add a bit more semantics by returning bit-values in case of errors, but this is in early stages of thinking. TeX Catalogue data in tlmgr info Since more or less the very beginning we incorporated information from the TeX Catalogue into our database. In particular did we carry over the license information, version, CTAN directory, and date of last change of information in the Catalogue. Recently (or not so recently, I actually don t know), CTAN has enriched their package view with more information, in particular a list of topics, and a list of related packages. Take for example the Asana-math package. It s CTAN page now displays besides the previously available information also a list of topics and a list of related packages. The topic index can also be browsed directly when searching for a specific package. I have now added functionality in the TeX Live Manager that tlmgr info also prints out the topic names and related packages. In the case of Asana Math fonts, that would look like: GUIs could use the topic names and related packages to link directly to the CTAN page. At the moment the related packages are named according to CTAN standards, which are a bit different from what we use in TeX Live. I am not sure whether I will change that, or ship out both names. We will see.

---

The changes are currently in testing, see section about Test version here, and will be pushed out in due time, probably in the next week. As usual, in case of any problems or bugs, please contact us at the TeX Live mailing list. Enjoy.

My first mobile phone lasted for days between charges. I never really found out how long it s battery would last because there was no way that I could use it to deplete the charge in any time that I could spend awake. Even if I had managed to run the battery out the phone was designed to accept 4*AA batteries (it s rechargeable battery pack was exactly that size) so I could buy spare batteries at any store. Modern phones are quite different in physical phone design (phones that weigh less than 4*AA batteries aren t uncommon), functionality (fast CPUs and big screens suck power), and use (games really drain your phone battery). This requires much more effective chargers, when some phones are intensively used (EG playing an action game with Wifi enabled) they can t be charged as they use more power than the plug-pack supplies. I ve previously blogged some calculations about resistance and thickness of wires for phone chargers [1], it s obvious that there are some technical limitations to phone charging based on the decision to use a long cable at ~5V. My calculations about phone charge rate were based on the theoretical resistance of wires based on their estimated cross-sectional area. One problem with such analysis is that it s difficult to determine how thick the insulation is without destroying the wire. Another problem is that after repeated use of a charging cable some conductors break due to excessive bending. This can significantly increase the resistance and therefore increase the charging time. Recently a charging cable that used to be really good suddenly became almost useless. My Galaxy Note 2 would claim that it was being charged even though the reported level of charge in the battery was not increasing, it seems that the cable only supplied enough power to keep the phone running not enough to actually charge the battery. I recently bought a USB current measurement device which is really useful. I have used it to diagnose power supplies and USB cables that didn t work correctly. But one significant way in which it fails is in the case of problems with the USB connector. Sometimes a cable performs differently when connected via the USB current measurement device. The CurrentWidget program [2] on my Galaxy Note 2 told me that all of the dedicated USB chargers (the 12V one in my car and all the mains powered ones) supply 1698mA (including the ones rated at 1A) while a PC USB port supplies ~400mA. I don t think that the Note 2 measurement is particularly reliable. On my Galaxy Note 3 it always says 0mA, I guess that feature isn t implemented. An old Galaxy S3 reports 999mA of charging even when the USB current measurement device says ~500mA. It seems to me that method the CurrentWidget uses to get the current isn t accurate if it even works at all. Android 5 on the Nexus 4/5 phones will tell the amount of time until the phone is charged in some situations (on the Nexus 4 and Nexus 5 that I used for testing it didn t always display it and I don t know why). This is an useful but it s still not good enough. I think that what we need is to have the phone measure the current that s being supplied and report it to the user. Then when a phone charges slowly because apps are using some power that won t be mistaken for a phone charging slowly due to a defective cable or connector.

• [1] http://etbe.coker.com.au/2013/01/24/power-supplies-wires/
• [2] https://play.google.com/store/apps/details?id=com.manor.currentwidget

say you use CFEngine to install Debian packages on your server, so it's likely you'll have a bundle looking like this:

bundle agent agentname

vars:

"packages" slist =>
"pkg1",
"pkg2",
"pkg3"
;

packages:

"$(packages)"
package_policy => "addupdate",
package_method => apt_get;



this works great to guarantee those packages are installed, but if a newer version is available in the repositories, that wont be installed. If you want CFEngine to do that too, then the web suggests this trick:

packages:

"$(packages)"
package_policy => "addupdate",
package_version => "999999999",
package_method => apt_get;

which tweak the install system declaring that you want to install version 999999999 of each package, so if you have available a higher version than the one installed, CFEngine will happily upgrade it for you. It works great.. but sometimes it doesn't. why oh why?

That's because Debian versions can have a epoch: every plain version (like 1.0-1) has an implicit epoch of 0, and same goes for the 999999999 above, that means if any of the installed packages has an epoch, that version will sort higher than 999999999 and the package wont be upgraded. If you want to be sure to upgrade every package, then the right solution is:

packages:

"$(packages)"
package_policy => "addupdate",
package_version => "9:999999999",
package_method => apt_get;

The Core Infrastructure Initiative announced today that they will support two Debian Developers, Holger Levsen and J r my Bobbio, with $200,000 to advance their Debian work in reproducible builds and to collaborate more closely with other distributions such as Fedora, Ubuntu, OpenWrt to benefit from this effort. The Core Infrastructure Initiative (CII) was established in 2014 to fortify the security of key open source projects. This initiative is funded by more than 20 companies and managed by The Linux Foundation. The reproducible builds initiative aims to enable anyone to reproduce bit by bit identical binary packages from a given source, thus enabling anyone to independently verify that a binary matches the source code from which it was said it was derived. For example, this allow the users of Debian to rebuild packages and obtain exactly identical packages to the ones provided by the Debian repositories.

You may have noticed that Jessie no longer includes the useful rescue flavour of live image, formerly included in Wheezy and earlier releases, and neither will Stretch unless you take action. This is my second public call for help this year to revive it. So if you care about rescue, here s how you can help:

• First, try a self-built image, based on the old live-image-rescue configuration. While Jessie still contains the live-image-rescue configuration for live-build as a starting point, to successfully build this image for yourself, you need to edit the package lists to drop or substitute any packages that aren t in the archive. As of writing, this includes libphash0, mii-diag, denyhosts, hal and emacs23-nox. (Tip: for the latter, substitute emacs24-nox.)
• Join or form a team to maintain the rescue metapackages in the long term. All of the official Debian Live images are based on metapackages that are looked after by various other teams, (principally the desktop teams,) with rescue being the sole exception. The old package lists include some forensics packages, so you may wish to contact Debian Forensics, but I don t want to presume they ll take it on.
• Have your team decide on what a rescue system should include. You might start with the old lists, spruced up a bit just to make the image build, or you might take an entirely different tack. This is your project, so it s up to you.
• File a bug on tasksel, preferably with patch, to include a task-forensics and/or task-rescue task (or whatever you decide the task or tasks should be called).
• File a bug on the live-images package to include your work.

If you have any questions not answered in this post, please feel free to leave a comment on this blog, talk to the Debian Live team on irc I m SynrG, and hang out with the team at #debian-live @ irc.oftc.net) or drop us an email at debian-live@lists.debian.org.

I was asked for advice on whether children should have access to smart phones, it s an issue that many people are discussing and seems worthy of a blog post. Claimed Problems with Smart Phones The first thing that I think people should read is this XKCD post with quotes about the demise of letter writing from 99+ years ago [1]. Given the lack of evidence cited by people who oppose phone use I think we should consider to what extent the current concerns about smart phone use are just reactions to changes in society. I ve done some web searching for reasons that people give for opposing smart phone use by kids and addressed the issues below. Some people claim that children shouldn t get a phone when they are so young that it will just be a toy. That s interesting given the dramatic increase in the amount of money spent on toys for children in recent times. It s particularly interesting when parents buy game consoles for their children but refuse mobile phone toys (I know someone who did this). I think this is more of a social issue regarding what is a suitable toy than any real objection to phones used as toys. Obviously the educational potential of a mobile phone is much greater than that of a game console. It s often claimed that kids should spend their time reading books instead of using phones. When visiting libraries I ve observed kids using phones to store lists of books that they want to read, this seems to discredit that theory. Also some libraries have Android and iOS apps for searching their catalogs. There are a variety of apps for reading eBooks, some of which have access to many free books but I don t expect many people to read novels on a phone. Cyber-bullying is the subject of a lot of anxiety in the media. At least with cyber-bullying there s an electronic trail, anyone who suspects that their child is being cyber-bullied can check that while old-fashioned bullying is more difficult to track down. Also while cyber-bullying can happen faster on smart phones the victim can also be harassed on a PC. I don t think that waiting to use a PC and learn what nasty thing people are saying about you is going to be much better than getting an instant notification on a smart phone. It seems to me that the main disadvantage of smart phones in regard to cyber-bullying is that it s easier for a child to participate in bullying if they have such a device. As most parents don t seem concerned that their child might be a bully (unfortunately many parents think it s a good thing) this doesn t seem like a logical objection. Fear of missing out (FOMO) is claimed to be a problem, apparently if a child has a phone then they will want to take it to bed with them and that would be a bad thing. But parents could have a policy about when phones may be used and insist that a phone not be taken into the bedroom. If it s impossible for a child to own a phone without taking it to bed then the parents are probably dealing with other problems. I m not convinced that a phone in bed is necessarily a bad thing anyway, a phone can be used as an alarm clock and instant-message notifications can be turned off at night. When I was young I used to wait until my parents were asleep before getting out of bed to use my PC, so if smart-phones were available when I was young it wouldn t have changed my night-time computer use. Some people complain that kids might use phones to play games too much or talk to their friends too much. What do people expect kids to do? In recent times the fear of abduction has led to children doing playing outside a lot less, it used to be that 6yos would play with other kids in their street and 9yos would be allowed to walk to the local park. Now people aren t allowing 14yo kids walk to the nearest park alone. Playing games and socialising with other kids has to be done over the Internet because kids aren t often allowed out of the house. Play and socialising are important learning experiences that have to happen online if they can t happen offline. Apps can be expensive. But it s optional to sign up for a credit card with the Google Play store and the range of free apps is really good. Also the default configuration of the app store is to require a password entry before every purchase. Finally it is possible to give kids pre-paid credit cards and let them pay for their own stuff, such pre-paid cards are sold at Australian post offices and I m sure that most first-world countries have similar facilities. Electronic communication is claimed to be somehow different and lesser than old-fashioned communication. I presume that people made the same claims about the telephone when it first became popular. The only real difference between email and posted letters is that email tends to be shorter because the reply time is smaller, you can reply to any questions in the same day not wait a week for a response so it makes sense to expect questions rather than covering all possibilities in the first email. If it s a good thing to have longer forms of communication then a smart phone with a big screen would be a better option than a feature phone , and if face to face communication is preferred then a smart phone with video-call access would be the way to go (better even than old fashioned telephony). Real Problems with Smart Phones The majority opinion among everyone who matters (parents, teachers, and police) seems to be that crime at school isn t important. Many crimes that would result in jail sentences if committed by adults receive either no punishment or something trivial (such as lunchtime detention) if committed by school kids. Introducing items that are both intrinsically valuable and which have personal value due to the data storage into a typical school environment is probably going to increase the amount of crime. The best options to deal with this problem are to prevent kids from taking phones to school or to home-school kids. Fixing the crime problem at typical schools isn t a viable option. Bills can potentially be unexpectedly large due to kids inability to restrain their usage and telcos deliberately making their plans tricky to profit from excess usage fees. The solution is to only use pre-paid plans, fortunately many companies offer good deals for pre-paid use. In Australia Aldi sells pre-paid credit in $15 increments that lasts a year [2]. So it s possible to pay $15 per year for a child s phone use, have them use Wifi for data access and pay from their own money if they make excessive calls. For older kids who need data access when they aren t at home or near their parents there are other pre-paid phone companies that offer good deals, I ve previously compared prices of telcos in Australia, some of those telcos should do [3]. It s expensive to buy phones. The solution to this is to not buy new phones for kids, give them an old phone that was used by an older relative or buy an old phone on ebay. Also let kids petition wealthy relatives for a phone as a birthday present. If grandparents want to buy the latest smart-phone for a 7yo then there s no reason to stop them IMHO (this isn t a hypothetical situation). Kids can be irresponsible and lose or break their phone. But the way kids learn to act responsibly is by practice. If they break a good phone and get a lesser phone as a replacement or have to keep using a broken phone then it s a learning experience. A friend s son head-butted his phone and cracked the screen he used it for 6 months after that, I think he learned from that experience. I think that kids should learn to be responsible with a phone several years before they are allowed to get a learner s permit to drive a car on public roads, which means that they should have their own phone when they are 12. I ve seen an article about a school finding that tablets didn t work as well as laptops which was touted as news. Laptops or desktop PCs obviously work best for typing. Tablets are for situations where a laptop isn t convenient and when the usage involves mostly reading/watching, I ve seen school kids using tablets on excursions which seems like a good use of them. Phones are even less suited to writing than tablets. This isn t a problem for phone use, you just need to use the right device for each task. Phones vs Tablets Some people think that a tablet is somehow different from a phone. I ve just read an article by a parent who proudly described their policy of buying feature phones for their children and tablets for them to do homework etc. Really a phone is just a smaller tablet, once you have decided to buy a tablet the choice to buy a smart phone is just about whether you want a smaller version of what you have already got. The iPad doesn t appear to be able to make phone calls (but it supports many different VOIP and video-conferencing apps) so that could technically be described as a difference. AFAIK all Android tablets that support 3G networking also support making and receiving phone calls if you have a SIM installed. It is awkward to use a tablet to make phone calls but most usage of a modern phone is as an ultra portable computer not as a telephone. The phone vs tablet issue doesn t seem to be about the capabilities of the device. It s about how portable the device should be and the image of the device. I think that if a tablet is good then a more portable computing device can only be better (at least when you need greater portability). Recently I ve been carrying a 10 tablet around a lot for work, sometimes a tablet will do for emergency work when a phone is too small and a laptop is too heavy. Even though tablets are thin and light it s still inconvenient to carry, the issue of size and weight is a greater problem for kids. 7 tablets are a lot smaller and lighter, but that s getting close to a 5 phone. Benefits of Smart Phones Using a smart phone is good for teaching children dexterity. It can also be used for teaching art in situations where more traditional art forms such as finger painting aren t possible (I have met a professional artist who has used a Samsung Galaxy Note phone for creating art work). There is a huge range of educational apps for smart phones. The Wikireader (that I reviewed 4 years ago) [4] has obvious educational benefits. But a phone with Internet access (either 3G or Wifi) gives Wikipedia access including all pictures and is a better fit for most pockets. There are lots of educational web sites and random web sites that can be used for education (Googling the answer to random questions). When it comes to preparing kids for the real world or the work environment people often claim that kids need to use Microsoft software because most companies do (regardless of the fact that most companies will be using radically different versions of MS software by the time current school kids graduate from university). In my typical work environment I m expected to be able to find the answer to all sorts of random work-related questions at any time and I think that many careers have similar expectations. Being able to quickly look things up on a phone is a real work skill, and a skill that s going to last a lot longer than knowing today s version of MS-Office. There are a variety of apps for tracking phones. There are non-creepy ways of using such apps for monitoring kids. Also with two-way monitoring kids will know when their parents are about to collect them from an event and can stay inside until their parents are in the area. This combined with the phone/SMS functionality that is available on feature-phones provides some benefits for child safety. iOS vs Android Rumour has it that iOS is better than Android for kids diagnosed with Low Functioning Autism. There are apparently apps that help non-verbal kids communicate with icons and for arranging schedules for kids who have difficulty with changes to plans. I don t know anyone who has a LFA child so I haven t had any reason to investigate such things. Anyone can visit an Apple store and a Samsung Experience store as they have phones and tablets you can use to test out the apps (at least the ones with free versions). As an aside the money the Australian government provides to assist Autistic children can be used to purchase a phone or tablet if a registered therapist signs a document declaring that it has a therapeutic benefit. I think that Android devices are generally better for educational purposes than iOS devices because Android is a less restrictive platform. On an Android device you can install apps downloaded from a web site or from a 3rd party app download service. Even if you stick to the Google Play store there s a wider range of apps to choose from because Google is apparently less restrictive. Android devices usually allow installation of a replacement OS. The Nexus devices are always unlocked and have a wide range of alternate OS images and the other commonly used devices can usually have an alternate OS installed. This allows kids who have the interest and technical skill to extensively customise their device and learn all about it s operation. iOS devices are designed to be sealed against the user. Admittedly there probably aren t many kids with the skill and desire to replace the OS on their phone, but I think it s good to have option. Android phones have a range of sizes and features while Apple only makes a few devices at any time and there s usually only a couple of different phones on sale. iPhones are also a lot smaller than most Android phones, according to my previous estimates of hand size the iPhone 5 would be a good tablet for a 3yo or good for side-grasp phone use for a 10yo [5]. The main benefits of a phone are for things other than making phone calls so generally the biggest phone that will fit in a pocket is the best choice. The tiny iPhones don t seem very suitable. Also buying one of each is a viable option. Conclusion I think that mobile phone ownership is good for almost all kids even from a very young age (there are many reports of kids learning to use phones and tablets before they learn to read). There are no real down-sides that I can find. I think that Android devices are generally a better option than iOS devices. But in the case of special needs kids there may be advantages to iOS.

• [1] http://www.xkcd.com/1227/
• [2] https://www.aldimobile.com.au/plans
• [3] http://etbe.coker.com.au/2014/04/01/comparing-telcos-again/
• [4] http://etbe.coker.com.au/2010/11/25/the-wikireader/
• [5] http://etbe.coker.com.au/2013/02/11/phone-tablet-sizes/

The first 4 weeks have been really great. I had the chance to work on the new webapp, the copyright tracker. Here are my first 4 reports for the weeks:

• week #1
• week #2
• week #3
• week #4

Summary During these weeks i completed the navigation (by prefix, list of versions etc) for the copyright tracker, the license rendering and the API for searching by checksum and by package/version/filename. Still under review the batch API (request many files at once) and the the respective views for the API. In the following weeks I ll work on the database and the plugins to mine license information on update time. Experience The experience i accumulate every week is immense. During the review process of my code i receive great feedback from zack, matthieu, and jpleau. It is a great chance for everyone to improve his/hers coding skills. At the university we barely get any feedback on our code, let alone communicate directly with professors to find the best solution for a given problem. I think it gets development in another level. For example at the university we use git/mercurial for our projects, although we don't really pay attention at commit messages and we marginally make use of these tools besides 'commit pull push'. During the last weeks i had the chance to use far more commands in order to squash amend rebase commits to have a beautiful and useful log history! And this is just a minor example. All in all I think it is a great introduction to the free/open source community for somebody, like me, who has been for years only a user and couldn't find the opportunity to contribute! And don't get me wrong, gsoc is not only about coding skills. At least for myself having the chance to get just a small taste of how the Debian community works, gets organised, get things done (!) is somehow priceless! TDD I have finally understood why test driven development is so powerful. As i am advancing in the project i am sometimes obliged to retouch some areas, refactor pieces of code. Having a great test suite that keeps you on track about the changes, and possible outages in advance is pretty powerful. What is coming I feel the project is on the right path. Both clemux (fellow gsoc student working on debsources) and me are advancing in a productive pace. I hope that soon enough the copyright Blueprint will be completed (most notable pending functionalities are the plugins-hooks to mine the licenses at update time, introduction of at least another license oracle possible ninka, statistics) so that everyone can take profit of the new tools provided by DebSources. Although we haven't talked much about it the patch tracker should also come into the play sooner rather than later. Stay tuned!

The first 4 weeks have been really great. I had the chance to work on the new webapp, the copyright tracker. Here are my first 4 reports for the weeks:

• week #1
• week #2
• week #3
• week #4

Summary During these weeks i completed the navigation (by prefix, list of versions etc) for the copyright tracker, the license rendering and the API for searching by checksum and by package/version/filename. Still under review the batch API (request many files at once) and the the respective views for the API. In the following weeks I ll work on the database and the plugins to mine license information on update time. Experience The experience i accumulate every week is immense. During the review process of my code i receive great feedback from zack, matthieu, and jpleau. It is a great chance for everyone to improve his/hers coding skills. At the university we barely get any feedback on our code, let alone communicate directly with professors to find the best solution for a given problem. I think it gets development in another level. For example at the university we use git/mercurial for our projects, although we don't really pay attention at commit messages and we marginally make use of these tools besides 'commit pull push'. During the last weeks i had the chance to use far more commands in order to squash amend rebase commits to have a beautiful and useful log history! And this is just a minor example. All in all I think it is a great introduction to the free/open source community for somebody, like me, who has been for years only a user and couldn't find the opportunity to contribute! And don't get me wrong, gsoc is not only about coding skills. At least for myself having the chance to get just a small taste of how the Debian community works, gets organised, get things done (!) is somehow priceless! TDD I have finally understood why test driven development is so powerful. As i am advancing in the project i am sometimes obliged to retouch some areas, refactor pieces of code. Having a great test suite that keeps you on track about the changes, and possible outages in advance is pretty powerful. What is coming I feel the project is on the right path. Both clemux (fellow gsoc student working on debsources) and me are advancing in a productive pace. I hope that soon enough the copyright Blueprint will be completed (most notable pending functionalities are the plugins-hooks to mine the licenses at update time, introduction of at least another license oracle possible ninka, statistics) so that everyone can take profit of the new tools provided by DebSources. Although we haven't talked much about it the patch tracker should also come into the play sooner rather than later. Stay tuned!

I've been a proponet of CaCert.org for a long time and I'm still using those certificates in some places, but lately I gave in and searched for something that validates even on iOS. It's not that I strictly need it, it's more a favour to make life for friends and family easier. I turned down startssl.com because I always manage to somehow lose the client certificate for the portal login. Plus I failed to generate several certificates for subdomains within the primary domain. I want to use different keys on purpose so SANs are not helpful, neither are wildcard certs for which you've to pay anyway. Another point against a wildcard cert from startssl is that I'd like to refrain from sending in my scanned papers for verification. On a sidenote I'm also not a fan of random email address extractions from whois to sent validation codes to. I just don't see why the abuse desk of a registrar should be able to authorize on DV certificates for a domain under my control. So I decided to pay the self proclaimed leader of the snakeoil industrie (Comodo) via cheapsslshop.com. That made 12USD for a 3 year Comodo DV certificate. Fair enough for the mailsetup I share with a few friends, and the cheapest one I could find at that time. Actually no hassle with logins or verification. It looks a bit like a scam but the payment is done via 2checkout if I remember correctly and the certificate got issued via a voucher code by Comodo directly. Drawback: credit card payment. Now while we're all waiting for letsencrypt.org I learned about the free offer of wosign.com. The CA is issued by the StartSSL Root CA, so technically we're very close to step one. Beside of that I only had to turn off uBlock origin and the rest of the JavaScript worked fine with Iceweasel once I clicked on the validity time selection checkbox. They offer the certificate for up to 3 years, you can paste your own csr and you can add up to 100 SANs. The only drawback is that it took them about 12 hours to issue the certificate and the mails look a hell lot like spam if you sent them through Spamassassin. That provides now a free and validating certificate for sven.stormbind.net in case you'd like to check out the chain. The validation chain is even one certificate shorter then the chain for the certificate I bought from Comodo. So in case anyone else is waiting for letsencrypt to start, you might want to check wosign until Mozilla et al are ready. From my point of view the only reason to pay one of the major CAs is for the service of running a reliable OCSP system. I also pointed that out here. It's more and more about the service you buy and no longer just money for a few ones and zeroes.

Debian now have over 22 000 source packages and 45 500 binary packages. To counter that, the FTP masters and I have created a dak tool to automatically remove packages from unstable! This is also much more efficient than only removing them from testing! :) The primary goal of the auto-decrufter is to remove a regular manual work flow from the FTP masters. Namely, the removal of the common cases of cruft, such as Not Built from Source (NBS) and Newer Version In Unstable (NVIU). With the auto-decrufter in place, such cruft will be automatically removed when there are no reverse dependencies left on any architecture and nothing Build-Depends on it any more. Despite the implication in the opening of this post, this will in fact not substantially reduce the numbers of packages in unstable. :) Nevertheless, it is still very useful for the FTP masters, the release team and packaging Debian contributors. The reason why the release team benefits greatly from this tool, is that almost every transition generates one piece of NBS -cruft. Said piece of cruft currently must be removed from unstable before the transition can progress into its final phase. Until recently that removal has been 100% manual and done by the FTP masters. The restrictions on auto-decrufter means that we will still need manual decrufts. Notably, the release team will often complete transitions even when some reverse dependencies remain on non-release architectures. Nevertheless, it is definitely an improvement. Omelettes and eggs: As an old saying goes You cannot make an omelette without breaking eggs . Less so when the only test suite is production. So here are some of the broken eggs caused by implementation of the auto-decrufter:

• About 30 minutes of dak rm (without no-action) would unconditionally crash.
• A broken dinstall when dak auto-decruft was run without dry-run for the first time.
• A boolean condition inversion causing removals to remove the override for partial removals (and retain it for full removals).
  • Side-effect, this broke Britney a couple of times because dak now produced some unexpected Packages files for unstable.
• Not to mention the single digit bug closure bug.

Of the 3, the boolean inversion was no doubt the worst. By the time we had it fixed, at least 50 (unique) binary packages had lost their override . Fortunately, it was possible to locate these issues using a database query and they have now been fixed. Before I write any more non-trivial patches for dak, I will probably invest some time setting up a basic test framework for dak first.
Filed under: Debian, Release-Team

What happened about the reproducible builds effort this week: Toolchain fixes

• Brendan O'Dea uploaded help2man/1.47.1 which adds support setting SOURCE_DATE_EPOCH for the date for the generated pages.
• Emmanuel Bourg uploaded maven-debian-helper/1.6.12 which sets the locale to en_US when generating the javadoc.
• Emmanuel Bourg uploaded javatools/0.51 which sets the locale to en_US when generating the javadoc.
• Joachim Breitner uploaded haskell-devscripts/0.9.10 which will always run sorts in LC_ALL=C.

Andreas Henriksson has improved Johannes Schauer initial patch for pbuilder adding support for build profiles. Packages fixed The following 12 packages became reproducible due to changes in their build dependencies: collabtive, eric, file-rc, form-history-control, freehep-chartableconverter-plugin , jenkins-winstone, junit, librelaxng-datatype-java, libwildmagic, lightbeam, puppet-lint, tabble. The following packages became reproducible after getting fixed:

• acorn/0.12.0-1 uploaded by Bas Couwenberg, original patch by Reiner Herrmann.
• avarice/2.13+svn347-3 by Tobias Frost.
• br.ispell/3.0~beta4-19 by Agustin Martin Domingo.
• cpp-netlib/0.11.1+dfsg1-3 by Ximin Luo.
• device3dfx/2013.08.08-3 by Guillem Jover.
• dnsmasq/2.73-1 uploaded by Simon Kelley, original patch by Chris Lamb.
• eigen3/3.2.5-4 by Anton Gladky.
• eo-spell/3.0~beta4-19 by Agustin Martin Domingo.
• espa-nol/3.0~beta4-19 by Agustin Martin Domingo.
• firejail/0.9.26-1 by Reiner Herrmann.
• gcc-mingw-w64/15.2 by Stephen Kitt.
• geoip-database/20150616-1 uploaded by Patrick Matth i, original patch by Reiner Herrmann.
• ikiwiki-hosting/0.20150614 by Simon McVittie.
• inkscape/0.91-5 by Mattia Rizzolo.
• jtreg/4.1-b12-1 by Emmanuel Bourg.
• libfile-scan-perl/1.43-3 uploaded by gregor herrmann, original patch by Niko Tyno.
• libgpiv/0.6.1-4.2 by akira.
• libnet-twitter-lite-perl/0.12006-4 by Niko Tyni.
• mingw-w64/4.0.2-5 by Stephen Kitt.
• oslo.messaging/1.8.3-3 uploaded by Thomas Goirand, original patch by Juan Picca.
• seabios/1.8.2-1 by Michael Tokarev. Suggested fix by Lunar based on recent upstream changes.
• thuban/1.2.2-7 uploaded by Bas Couwenberg, original patch by Reiner Herrmann.
• tiptop/2.2-3 by Tomasz Buchert.
• ucl/1.03+repack-3 by Robert Luberda.

Some uploads fixed some reproducibility issues but not all of them:

• amsynth/1.5.1-2 uploaded by Alessio Treglia, original patch by Dhole.
• brickos/0.9.0.dfsg-12 uploaded by Michael Tautschnig, original patch by akira.
• cucumber/2.0.0-1 by C dric Boutillier; currently FTBFS.
• netbeans/8.0.2+dfsg1-2 by Markus Koschany.
• pyopencl/2015.1-2 uploaded by Tomasz Rybak, original patch by Tomasz Rybak.

Patches submitted which have not made their way to the archive yet:

• #788747 on 0xffff by Dhole: allow embedded timestamp to be set externally and set it to the time of the debian/changelog.
• #788752 on analog by Dhole: allow embedded timestamp to be set externally and set it to the time of the debian/changelog.
• #788757 on jacktrip by akira: remove $datetime from the documentation footer.
• #788868 on apophenia by akira: remove $date from the documentation footer.
• #788920 on orthanc by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #788955 on rivet by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789040 on liblo by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789049 on mpqc by akira: remove $datetime from the documentation footer.
• #789071 on libxkbcommon by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789073 on libxr by akira: remove $datetime from the documentation footer.
• #789076 on lvtk by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789087 on lmdb by akira: pass HTML_TIMESTAMP=NO to Doxygen.
• #789184 on openigtlink by akira: remove $datetime from the documentation footer.
• #789264 on openscenegraph by akira: pass HTML_TIMESTAMP=NO to Doxygen.
• #789308 on trigger-rally-data by Mattia Rizzolo: call dh_fixperms even when overriding dh_fixperms.
• #789396 on libsidplayfp by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789399 on psocksxx by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789405 on qdjango by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789406 on qof by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
• #789428 on qsapecng by akira: pass HTML_TIMESTAMP=NO to Doxygen.

reproducible.debian.net Bugs with the ftbfs usertag are now visible on the bug graphs. This explain the recent spike. (h01ger) Andreas Beckmann suggested a way to test building packages using the funny paths that one can get when they contain the full Debian package version string. debbindiff development Lunar started an important refactoring introducing abstactions for containers and files in order to make file type identification more flexible, enabling fuzzy matching, and allowing parallel processing. Documentation update Ximin Luo detailed the proposal to standardize environment variables to pass a reference source date to tools that needs one (e.g. documentation generator). Package reviews 41 obsolete reviews have been removed, 168 added and 36 updated this week. Some more issues affecting packages failing to build from source have been identified. Meetings Minutes have been posted for Tuesday June 16th meeting. The next meeting is scheduled Tuesday June 23rd at 17:00 UTC. Presentations Lunar presented the project in French during Pas Sage en Seine in Paris. Video and slides are available.

Recently I've been experimenting with camlistore, which is yet another object storage system. Camlistore gains immediate points because it is written in Go, and is a project initiated by Brad Fitzpatrick, the creator of Perlbal, memcached, and Livejournal of course. Camlistore is designed exactly how I'd like to see an object storage-system - each server allows you to:

• Upload a chunk of data, getting an ID in return.
• Download a chunk of data, by ID.
• Iterate over all available IDs.

It should be noted more is possible, there's a pretty web UI for example, but I'm simplifying. Do your own homework :)

With those primitives you can allow a client-library to upload a file once, then in the background a bunch of dumb servers can decide amongst themselves "Hey I have data with ID:33333 - Do you?". If nobody else does they can upload a second copy. In short this kind of system allows the replication to be decoupled from the storage. The obvious risk is obvious though: if you upload a file the chunks might live on a host that dies 20 minutes later, just before the content was replicated. That risk is minimal, but valid. There is also the risk that sudden rashes of uploads leave the system consuming all the internal-bandwith constantly comparing chunk-IDs, trying to see if data is replaced that has been copied numerous times in the past, or trying to play "catch-up" if the new-content is larger than the replica-bandwidth. I guess it should possible to detect those conditions, but they're things to be concerned about. Anyway the biggest downside with camlistore is documentation about rebalancing, replication, or anything other than simple single-server setups. Some people have blogged about it, and I got it working between two nodes, but I didn't feel confident it was as robust as I wanted it to be. I have a strong belief that Camlistore will become a project of joy and wonder, but it isn't quite there yet. I certainly don't want to stop watching it :) On to the more personal .. I'm all about the object storage these days. Right now most of my objects are packed in a collection of boxes. On the 6th of next month a shipping container will come pick them up and take them to Finland. For pretty much 20 days in a row we've been taking things to the skip, or the local charity-shops. I expect that by the time we've relocated the amount of possesions we'll maintain will be at least a fifth of our current levels. We're working on the general rule of thumb: "If it is possible to replace an item we will not take it". That means chess-sets, mirrors, etc, will not be carried. DVDs, for example, have been slashed brutally such that we're only transferring 40 out of a starting collection of 500+. Only personal, one-off, unique, or "significant" items will be transported. This includes things like personal photographs, family items, and similar. Clothes? Well I need to take one jacket, but more can be bought. The only place I put my foot down was books. Yes I'm a kindle-user these days, but I spent many years tracking down some rare volumes, and though it would be possible to repeat that effort I just don't want to. I've also decided that I'm carrying my complete toolbox. Some of the tools I took with me when I left home at 18 have stayed with me for the past 20+ years. I don't need this specific crowbar, or axe, but I'm damned if I'm going to lose them now. So they stay. Object storage - some objects are more important than they should be!

debtags rewritten in python3 In my long quest towards closing #540218, I have uploaded a new libept to experimental. Then I tried to build debtags on a sid+experimental chroot and the result runs but has libc's free() print existential warnings about whatevers. At a quick glance, there are now things around like a new libapt, gcc 5 with ABI changes, and who knows what else. I figured how much time it'd take me to debug something like that, and I've used that time to rewrite debtags in python3. It took 8 hours, 5 of pleasant programming and the usual tax of another 3 of utter frustration packaging the results. I guess I gained over the risk of spending an unspecified amount of hours of just pure frustration. So from now on debtags is going to be a pure python3 package, with dependencies on only python3-apt and python3-debian. 700 lines of python instead of several C++ files built on 4 layers of libraries. Hopefully, this is the last of the big headaches I get from hacking on this package. Also, one less package using libept.

Quite a while ago, I wrote a small Haskell library called circle-packing to pack circles in a tight arrangement. Back then, I used the Haskell to JavaScript compiler fay to create a pretty online demo of that library, and shortly after, I create the identical demo using haste (another Haskell to JavaScript compiler). The main competitor of these two compilers, and the most promising one, is GHCJS. Back then, it was too annoying to install. But after two years, things have changed, and it only takes a few simple commands to get GHCJS running, so I finally created the circle packing demo in a GHCJS variant. Quick summary: Cabal integration is very good (like haste, but unline fay), interfacing JavaScript is nice and easy (like fay, but unlike haste), and a quick check seems to indicate that it is faster than either of these two. I should note that I did not update the other two demos, so they represent the state of fay and haste back then, respectively. With GHCJS now available at my fingertips, maybe I will produce some more Haskell to be run in your browser. For example, I could port FrakView, a GUI program to render, expore and explain iterated function systems, from GTK to HTML.

What happened about the reproducible builds effort for this week: Toolchain fixes

• Guillem Jover uploaded dpkg/1.18.0 which now uses an approximation to compute Installed-Size, making it indpendent from the underlying filesystem. It now always sort the Dpkg::Dist::Files files list on output to make the output stable with parallel builds.
• Lunar uploaded mozilla-devscripts/0.40 which told xpi-pack to skip saving extra zip attributes when making jar.
• Dominique Dumont uploaded libmodule-build-perl/0.421100-2 which makes the output deterministic. Original patch by Lunar.

Lunar rebased our custom dpkg on the new release, removing a now unneeded patch identified by Guillem Jover. An extra sort in the buildinfo generator prevented a stable order and was quickly fixed once identified. Mattia Rizzolo also rebased our custom debhelper on the latest release. Packages fixed The following 30 packages became reproducible due to changes in their build dependencies: animal-sniffer, asciidoctor, autodock-vina, camping, cookie-monster, downthemall, flashblock, gamera, httpcomponents-core, https-finder, icedove-l10n, istack-commons, jdeb, libmodule-build-perl, libur-perl, livehttpheaders, maven-dependency-plugin, maven-ejb-plugin, mozilla-noscript, nosquint, requestpolicy, ruby-benchmark-ips, ruby-benchmark-suite, ruby-expression-parser, ruby-github-markup, ruby-http-connection, ruby-settingslogic, ruby-uuidtools, webkit2gtk, wot. The following packages became reproducible after getting fixed:

• aisleriot/1:3.16.2-1 uploaded by Andreas Henriksson, original patch by Chris Lamb.
• aws-sdk-for-php/2.8.5-1 by David Pr vot.
• bamtools/2.3.0+dfsg-3 uploaded by Andreas Tille, fix by Michael R. Crusoe.
• base-files/9.2 by Santiago Vila, some patches by Lunar.
• debian-keyring/2015.05.17 by Daniel Kahn Gillmor.
• debram/2.0.0.2 by Thaddeus H. Black.
• dianara/1.3.0-2 by M nica Ram rez Arceda.
• gazebo/5.0.1+dfsg-1~exp1 by Jose Luis Rivero.
• glassfish/1:2.1.1-b31g+dfsg1-3 uploaded by Emmanuel Bourg, original patch by Daniel Kahn Gillmor.
• lcdproc/0.5.7-3 by Dominique Dumont.
• libalien-sdl-perl/1.446-2 by Dominique Dumont.
• libvirt-python/1.2.15-1 uploaded by Guido G nther, original patch by Chris Lamb.
• libxmpcore-java/5.1.2-3 by Emmanuel Bourg.
• pdb2pqr/2.0.0+dfsg-1 uploaded by Andreas Tille, original patch by Reiner Herrmann.
• puredata/0.46.6-2 by Paul Brossier.
• qt-gstreamer/1.2.0-2 by Diane Trout.
• socat/1.7.3.0-1 uploaded by Laszlo Boszormenyi, original patch by Lunar.
• swaks/20130209.0-5 by Andreas Metzler, some patches by Chris Lamb.
• tf/1:4.0s1-19 by Jan Niehusmann.
• unzip/6.0-17 uploaded by Santiago Vila, original patch by Lunar.
• yorick/2.2.04+dfsg1-2 by by Thibaut Paumard.
• zip/3.0-10 by Santiago Vila, some patches by Chris Lamb.

Some uploads fixed some reproducibility issues but not all of them:

• bible-kjv/4.27 uploaded by Matthew Vernon, original patch by Chris Lamb.
• calendar/2.04-1 by St phane Glondu.
• cupt/2.9.0 uploaded by Eugene V. Lyubimkin, original patch by Chris Lamb.
• dactyl/1.2~hg7166-1 uploaded by Michael Schutte, original patch by Chris Lamb.
• ghc/7.10.1-5 by Joachim Breitner.
• icedove/38.0~b5-1 by Carsten Schoenert.
• jd/1:2.8.9-150226-2 by Hideki Yamane.
• libparse-debianchangelog-perl/1.2.0-2 by intrigeri.
• winswitch/0.12.21+dfsg-1 by Dmitry Smirnov.

Patches submitted which did not make their way to the archive yet:

• #775531 on console-setup by Reiner Herrmann: update and split patch written in January.
• #785535 on maradns by Reiner Herrmann: use latest entry in debian/changelog as build date.
• #785549 on dist by Reiner Herrmann: set hostname and domainname to predefined value.
• #785583 on s5 by Juan Picca: set timezone to UTC when unzipping files.
• #785617 on python-carrot by Juan Picca: use latest entry in debian/changelog as documentation build date.
• #785774 on afterstep by Juan Picca: modify documentation generator to allow a build date to be set instead of the current time, then use latest entry in debian/changelog as reference.
• #786508 on ttyload by Juan Picca: remove timestamp from documentation.
• #786568 on linux-minidisc by Lunar: use latest entry in debian/changelog as build date.
• #786615 on kfreebsd-10 by Steven Chamberlain: make order of file in source tarballs stable.
• #786633 on webkit2pdf by Reiner Herrmann: use latest entry in debian/changelog as documentation build date.
• #786634 on libxray-scattering-perl by Reiner Herrmann: tell Storable::nstore to produce sorted output.
• #786637 on nvidia-settings by Lunar: define DATE, WHOAMI, andHOSTNAME_CMD to stable values.
• #786710 on armada-backlight by Reiner Herrmann: use latest entry in debian/changelog as documentation build date.
• #786711 on leafpad by Reiner Herrmann: use latest entry in debian/changelog as documentation build date.
• #786714 on equivs by Reiner Herrmann: use latest entry in debian/changelog as documentation build date.

Also, the following bugs have been reported:

• #785536 on maradns by Reiner Herrmann: unreproducible deadwood binary.
• #785624 on doxygen by Christoph Berg: timestamps in manpages generated makes builds non-reproducible.
• #785736 on git-annex by Daniel Kahn Gillmor: documentation should be made reproducible.
• #786593 on wordwarvi by Holger Levsen: please provide a --distrobuild build switch.
• #786601 on sbcl by Holger Levsen: FTBFS when locales-all is installed instead of locales.
• #786669 on ruby-celluloid by Holger Levsen: tests sometimes fail, causing ftbfs sometimes.
• #786743 on obnam by Holger Levsen: FTBFS.

reproducible.debian.net Holger Levsen made several small bug fixes and a few more visible changes:

• For packages in testing, comparisions will be done using the sid version of debbindiff.
• The scheduler will now schedule old packages from sid twice often as the ones in testing as we care more about the former at the moment.
• More statistics are now visible and the layout has been improved.
• Variations between the first and second build are now explained on the statistics page.

strip-nondeterminism Version 0.007-1 of strip-nondeterminism the tool to post-process various file formats to normalize them has been uploaded by Holger Levsen. Version 0.006-1 was already in the reproducible repository, the new version mainly improve the detection of Maven's pom.properties files. debbindiff development At the request of Emmanuel Bourg, Reiner Herrmann added a comparator for Java .class files. Documentation update Christoph Berg created a new page for the timestamps in manpages created by Doxygen. Package reviews 93 obsolete reviews have been removed, 76 added and 43 updated this week. New identified issues: timestamps in manpages generated by Doxygen, modification time differences in files extracted by unzip, tstamp task used in Ant build.xml, timestamps in documentation generated by ASDocGen. The description for build id related issues has been clarified. Meetings Holger Levsen announced a first meeting on Wednesday, June 3rd, 2015, 19:00 UTC. The agenda is amendable on the wiki. Misc. Lunar worked on a proof-of-concept script to import the build environment found in .buildinfo files to UDD. Lucas Nussbaum has positively reviewed the proposed schema. Holger Levsen cleaned up various experimental toolchain repositories, marking merged brances as such.

What happened about the reproducible builds effort for this week: Toolchain fixes Uploads that should help other packages:

• Stephen Kitt uploaded mingw-w64/4.0.2-2 which avoids inserting timestamps in PE binaries, and specify dlltool's temp prefix so it generates reproducible files.
• Stephen Kitt uploaded binutils-mingw-w64/6.1 which fixed dlltool to initialize its output's .idata$6 section, avoiding random data ending up there.

Patch submitted for toolchain issues:

• #787159 on openjdk-7 by Emmanuel Bourg: sort the annotations and enums in package-tree.html produced by javadoc.
• #787250 on python-qt4 by Reiner Herrmann: sort imported modules to get reproducible output.
• #787251 on pyqt5 by Reiner Herrmann: sort imported modules to get reproducible output.

Some discussions have been started in Debian and with upstream:

• ocaml using random intermediate filenames (Daniel Kahn Gillmor)
• Timestamps embedded in docbook called by pkg-kde-tools or kdoctools5 (Daniel Kahn Gillmor)
• Should --no-insert-timestamps be the default in binutils-mingw-w64? (Stephen Kitt)
• Make PDF ID field deterministic in PDFTeX (Nicolas Boulenguez)
• Add --deterministic option to Tar? (Lunar)
• Undeterministic output from GCC with -flto -ffat-lto-objects (Lunar)

Packages fixed The following 8 packages became reproducible due to changes in their build dependencies: access-modifier-checker, apache-log4j2, jenkins-xstream, libsdl-perl, maven-shared-incremental, ruby-pygments.rb, ruby-wikicloth, uimaj. The following packages became reproducible after getting fixed:

• debianutils/4.5.1 uploaded by Clint Adams, original patch by Lunar.
• exifprobe/2.0.1-5 by Joao Eriberto Mota Filho.
• gdb-mingw-w64/10 by Stephen Kitt.
• iceweasel/38.0.1-5 by Mike Hommey.
• liblingua-en-tagger-perl/0.25-1 uploaded by Axel Beckert, original patch by Reiner Herrmann.
• liboro-java/2.0.8a-10 by Emmanuel Bourg.
• mingw-w64/4.0.2-3 by Stephen Kitt.
• minidlna/1.1.4+dfsg-4 by Alexander GQ Gerasiov.
• miredo/1.2.6-3 by Tomasz Buchert.
• mpv/0.9.2-1 uploaded by Alessandro Ghedini, original patch by Lunar.
• nspr/2:4.10.8-2 by Mike Hommey.
• openstack-doc-tools/0.24-2 uploaded by Thomas Goirand, original patch by Reiner Herrmann.
• osgearth/2.5.0+dfsg-3 uploaded by Bas Couwenberg, original patch by Reiner Herrmann.
• pyexiv2/0.3.2-8 by Michal iha .
• python-netaddr/0.7.14-1 by Vincent Bernat.
• ruby-extlib/0.9.16-1 by C dric Boutillier.
• sane-backends/1.0.24-12 by J rg Frings-F rst.
• sed/4.2.2-6 uploaded by Clint Adams, original patch.
• sextractor/2.19.5+dfsg-3 by Ole Streicher.
• swarp/2.38.0+dfsg-2 by Ole Streicher.
• tiptop/2.2-3 by Tomasz Buchert.
• tomcat7/7.0.62-1 by Emmanuel Bourg.
• tomcat8/8.0.23-1 by Emmanuel Bourg.
• xapian-omega/1.2.21-1 by Olly Betts, also fixed upstream.
• y-u-no-validate/2013052401-3 by Jakub Wilk.

Some uploads fixed some reproducibility issues but not all of them:

• 389-admin/1.1.38-1 uploaded by Timo Aaltonen, original patch by Chris Lamb.
• alt-ergo/0.99.1+dfsg1-3 uploaded by Ralf Treinen, original patch by Juan Picca and Jakub Wilk.
• deets/0.2.1-2 uploaded by Clint Adams, original patch by Chris Lamb.
• fpc/2.6.4+dfsg-5 by Paul Gevers.
• inspircd/2.0.20-1 by Guillaume Delacour.
• libparse-debianchangelog-perl/1.2.0-3 by intrigeri.
• luakit/2012.09.13-r1-4 uploaded by Clint Adams, original patch by Chris Lamb.
• mod-authn-webid/0~20110301-3 uploaded by Clint Adams, original patch by Chris Lamb.
• powerline/2.1.4-1 uploaded by Jerome Charaoui, reported by akira, fixed upstream.
• svtplay-dl/0.10.2015.05.24-1 by Olof Johansson.

Patches submitted which did not make their way to the archive yet:

• #777308 on dhcp-helper by Dhole: fix mtimes of packaged files.
• #786927 on flowscan by Dhole: remove timestamps from gzip files and fix mtimes of packaged files.
• #786959 on python3.5 by Lunar: set build date of binary and documentation to the time of latest debian/changelog entry, prevent gzip from storing a timestamp.
• #786965 on python3.4 by Lunar: same as python3.5.
• #786978 on python2.7 by Lunar: same as python3.5.
• #787122 on xtrlock by Dhole: fix mtimes of packaged files.
• #787123 on rsync by Dhole: remove timestamps from gzip files and fix mtimes of packaged files.
• #787125 on pachi by Dhole: fix mtimes of packaged files.
• #787126 on nis by Dhole: remove timestamps from gzip files and fix mtimes of packaged files.
• #787206 on librpc-xml-perl by Reiner Herrmann: remove timestamps from generated code.
• #787265 on libwx-perl by Reiner Herrmann: produce sorted output.
• #787303 on dos2unix by Juan Picca: set manpage date to the time of latest entry in debian/changelog.
• #787327 on vim by Reiner Herrmann: remove usage of __DATE__ and __TIME__ macros.

Discussions that have been started:

• gst-plugins-base1.0 embeds current timestamp in .rodata of all objects (Daniel Kahn Gillmor)

reproducible.debian.net Holger Levsen added two new package sets: pkg-javascript-devel and pkg-php-pear. The list of packages with and without notes are now sorted by age of the latest build. Mattia Rizzolo added support for email notifications so that maintainers can be warned when a package becomes unreproducible. Please ask Mattia or Holger or in the #debian-reproducible IRC channel if you want to be notified for your packages! strip-nondeterminism development Andrew Ayer fixed the gzip handler so that it skip adding a predetermined timestamp when there was none. Documentation update Lunar added documentation about mtimes of file extracted using unzip being timezone dependent. He also wrote a short example on how to test reproducibility. Stephen Kitt updated the documentation about timestamps in PE binaries. Documentation and scripts to perform weekly reports were published by Lunar. Package reviews 50 obsolete reviews have been removed, 51 added and 29 updated this week. Thanks Chris West and Mathieu Bridon amongst others. New identified issues:

• timestamps in .dat files from the Allegro framework,
• random identifier when using gcc -flto -ffat-lto-objects,
• random import order in .ui generated by pyuic,
• random order in C files generated by ExtUtils::ParseXS,
• timestamps in files generated by eingenbase resource generator.

Misc. Lunar will be talking (in French) about reproducible builds at Pas Sage en Seine on June 19th, at 15:00 in Paris. Meeting will happen this Wednesday, 19:00 UTC.

The version of btrfs-tools in Debian/Jessie is incapable of creating a filesystem that can be mounted by the kernel in Debian/Wheezy. If you want to use a BTRFS filesystem on Jessie and Wheezy (which isn t uncommon with removable devices) the only options are to use the Wheezy version of mkfs.btrfs or to use a Jessie kernel on Wheezy. I recently got bitten by this issue when I created a BTRFS filesystem on a removable device with a lot of important data (which is why I wanted metadata duplication and checksums) and had to read it on a server running Wheezy. Fortunately KVM in Wheezy works really well so I created a virtual machine to read the disk. Setting up a new KVM isn t that difficult, but it s not something I want to do while a client is anxiously waiting for their data. BTRFS has been working well for me apart from the Jessie/Wheezy compatability issue (which was an annoyance but didn t stop me doing what I wanted). I haven t written a BTRFS status report for a while because everything has been OK and there has been nothing exciting to report. I regularly get errors from the cron jobs that run a balance supposedly running out of free space. I have the cron jobs due to past problems with BTRFS running out of metadata space. In spite of the jobs often failing the systems keep working so I m not too worried at the moment. I think this is a bug, but there are many more important bugs. Linux kernel version 3.19 was the first version to have working support for RAID-5 recovery. This means version 3.19 was the first version to have usable RAID-5 (I think there is no point even having RAID-5 without recovery). It wouldn t be prudent to trust your important data to a new feature in a filesystem. So at this stage if I needed a very large scratch space then BTRFS RAID-5 might be a viable option but for anything else I wouldn t use it. BTRFS still has had little performance optimisation, while this doesn t matter much for SSD and for single-disk filesystems for a RAID-5 of hard drives that would probably hurt a lot. Maybe BTRFS RAID-5 would be good for a scratch array of SSDs. The reports of problems with RAID-5 don t surprise me at all. I have a BTRFS RAID-1 filesystem on 2*4TB disks which is giving poor performance on metadata, simple operations like ls -l on a directory with ~200 subdirectories takes many seconds to run. I suspect that part of the problem is due to the filesystem being written by cron jobs with files accumulating over more than a year. The btrfs filesystem command (see btrfs-filesystem(8)) allows defragmenting files and directory trees, but unfortunately it doesn t support recursively defragmenting directories but not files. I really wish there was a way to get BTRFS to put all metadata on SSD and all data on hard drives. Sander suggested the following command to defragment directories on the BTRFS mailing list: find / -xdev -type d -execdir btrfs filesystem defrag -c + Below is the output of zfs list -t snapshot on a server I run, it s often handy to know how much space is used by snapshots, but unfortunately BTRFS has no support for this.

NAME	USED	AVAIL	REFER	MOUNTPOINT
hetz0/be0-mail@2015-03-10	2.88G		387G
hetz0/be0-mail@2015-03-11	1.12G		388G
hetz0/be0-mail@2015-03-12	1.11G		388G
hetz0/be0-mail@2015-03-13	1.19G		388G

Hugo pointed out on the BTRFS mailing list that the following command will give the amount of space used for snapshots. $SNAPSHOT is the name of a snapshot and $LASTGEN is the generation number of the previous snapshot you want to compare with. btrfs subvolume find-new $SNAPSHOT $LASTGEN awk ' total = total + $7 END print total ' One upside of the BTRFS implementation in this regard is that the above btrfs command without being piped through awk shows you the names of files that are being written and the amounts of data written to them. Through casually examining this output I discovered that the most written files in my home directory were under the .cache directory (which wasn t exactly a surprise). Now I am configuring workstations with a separate subvolume for ~/.cache for the main user. This means that ~/.cache changes don t get stored in the hourly snapshots and less disk space is used for snapshots. Conclusion My observation is that things are going quite well with BTRFS. It s more than 6 months since I had a noteworthy problem which is pretty good for a filesystem that s still under active development. But there are still many systems I run which could benefit from the data integrity features of ZFS and BTRFS that don t have the resources to run ZFS and need more reliability than I can expect from an unattended BTRFS system. At this time the only servers I run with BTRFS are located within a reasonable drive from my home (not the servers in Germany and the US) and are easily accessible (not the embedded systems). ZFS is working well for some of the servers in Germany. Eventually I ll probably run ZFS on all the hosted servers in Germany and the US, I expect that will happen before I m comfortable running BTRFS on such systems. For the embedded systems I will just take the risk of data loss/corruption for the next few years.