The Haskell minicourse at the University of Pennsylvania, also known as CIS-194, has always had a reach beyond the students of Penn. At least since Brent Yorgey gave the course in 2013, who wrote extensive lecture notes and eventually put the material on Github. This year, it is my turn to give the course. I could not resist making some changes, at least to the first few weeks: Instead of starting with a locally installed compiler, doing execises that revolve mostly around arithmetic and lists, I send my students to CodeWorld, which is a web programming environment created by Chris Smith¹. This greatly lowers the initial hurlde of having to set up the local toolchain, and is inclusive towards those who have had little expose to the command line before. Not that I do not expect my students to handle that, but it does not hurt to move that towards later in the course. But more importantly: CodeWorld comes with a nicely designed simple API to create vector graphics, to animate these graphics and even create interactive programs. This means that instead of having to come up with yet another set of exercieses revolving around lists and numbers, I can have the students create Haskell programs that are visual. I believe that this is more motivating and stimulating, and will nudge the students to spend more time programming and thus practicing. In fact, the goal is that in their third homework assignemnt, the students will implement a fully functional, interactive Sokoban game. And all that before talking about the built-in lists or tuples, just with higher order functions and custom datatypes. (Click on the picture above, which is part of the second weeks s homework. You can use the arrow keys to move the figure around and press the escape key to reset the game. Boxes cannot be moved yet -- that will be part of homework 3.) If this sounds interesting to you, and you always wanted to learn Haskell from scratch, feel free to tag along. The lecture notes should be elaborate enough to learn from that, and with the homework problems, you should be able to tell whether you have solved it yourself. Just do not publish your solutions before the due date. Let me know if you have any comments about the course so far. Eventually, I will move to local compilation, use of the interpreter and text-based IO and then start using more of the material of previous iterations of the course, which were held by Richard Eisenberg in 2014 and by Noam Zilberstein in 2015.

Lintian's humble description of "Debian package checker" belies its importance within the Debian GNU/Linux project. An extensive static analysis tool, it's not only used by the vast majority of developers, falling foul of some of its checks even cause uploads to be automatically rejected by the archive maintenance software. As you may have read in my recent monthly report, I've recently been hacking on Lintian itself. In particular:

• #798983: Check for libjs-* binary package name outside of the web section
• #814326: Warn if filenames contain wildcard characters
• #829744: Add new-package-should-not-package-python2-module tag
• #831864: Warn about Python packages that ship Coverage.py information
• #832096 Check for common typos in debian/rules target names
• #832099: Check for unnecessary SOURCE_DATE_EPOCH assignments
• #832771: Warn about systemd .service files with a missing Install key

However, this rest of this post will go through the steps needed to start contributing yourself. To demonstrate this I will be walking through submitting a patch for bug #831864 which warns about Python packages that ship .coverage files generated by Coverage.py.

---

---

---

---

DebConf17 will take place in Montreal, Canada from August 6 to August 12, 2017. It will be preceded by DebCamp, July 31 to August 4, and Debian Day, August 5. We invite everyone to join us in organizing DebConf17. There are different areas where your help could be very valuable, and we are always looking forward to your ideas. The DebConf content team is open to suggestions for invited speakers. If you'd like to propose somebody who is not a regular DebConf attendee follow the details in the call for speaker proposals blog post. We are also beginning to contact potential sponsors from all around the globe. If you know any organization that could be interested, please consider handing them the sponsorship brochure or contact the fundraising team with any leads. The DebConf team is holding IRC meetings every two weeks. Have a look at the DebConf17 website and wiki page, and engage in the IRC channels and the mailing list. Let s work together, as every year, on making the best DebConf ever!

Wow, a mouthful of a title, for a much simpler post. Up until earlier this year, I had only one sport GPS device, and that was a watch. Made sense, as my "main" sport was running (not that I did it very consistently). Upgraded over a few generations of Garmins, I mainly used it to record my track and statistics (pace, heart rate, etc.) The newest generation of watch and heart rate monitor even give more statistics (left-right leg balance, ground contact time, so on). Most of this data can be looked at while running, but only as an exception; after all, it's very hard to run with one hand up in front of your face. The other useful features guided workouts and alerts during normal runs I've used, but not consistently. So when I started biking a bit more seriously, I wondered whether it would make sense to buy a bike computer. The feature intersection between watch and bike GPS is quite large, so clearly this is a "want" not a "need" by far. Reading forum posts showed the same question repeated multiple times What convinced me to nevertheless buy such a bike GPS were the mapping/routing features. A bike GPS with good routing features, or even just maps on which tracks can be overlayed, can certainly ease the discovery of new routes. After a few months of use, my most useful feature is one that I didn't expect. While the mapping is useful and I do use it, the one that actually is significantly better than my watch is the large display with data fields that I can trivially check almost continuously on road biking, and during technically easy climbing sections for mountain biking. My setup looks like this: It's a 9-field setup; the Edge 1000 can go to 10, but I like "headline" field. The watch can only go to four, and is basically not usable during rides, unless one would use a quick-release kit for mounting on the handle bar. This setup allowed me to learn much better my physical capabilities, why I sometimes run out of energy, and how the terrain affects me. Random things that I learned:

• Gradient: on a road bike, +2% grade is just fun, -2% grade is excellent; on a mountain bike, -2% is somewhat felt but not so much. Going above 6-8% on a mountain bike is tiring, and above 15% means I can bike but I will dip too much into my reserves. Not sure yet what the numbers are on a road bike
• Cadence: on flatter routes, my most efficient cadence is 102-108 RPM; between 98-102 I feel I need to extra push, and below 98 I know (now) my muscles will get tired too early; on significant ascents, I don't have enough gearing to sustain such an RPM, and that tires me faster. On medium distance flat rides (~70Km), I usually do ~100 averaged over the whole distance.
• Heart rate: below ~140 is recovery, ~140-150 is sustained effort, ~150-160 is short-duration pushes, and anything above ~160 will eat through my anaerobic budget, which means I'd better stop soon or my performance for the rest of the ride will suffer; this, surprisingly, matches quite well with my latest run lactate threshold (as computed by my watch), which was 161bpm.
• Condition: when cruising without pedalling or when stopping, I can ballpark my current condition very easily by seeing how fast my heart rate goes down.
• Total ascent: useful for two things: to make me proud how much I've already climbed, and if I know the total ascent for the route either make me despair how much I have left, or make me happy that the climbs are done

Seeing all this data only after the ride is less useful since you don't remember exactly how you felt during specific points in the ride. But as real-time data, one can easily see/feel the correlation between how the body feels and what data says. One thing that would be also useful would be real-time power data (3 sec average, not instantaneous) to correlate even better with the body state. I now use heart rate and cadence as a proxy for that, but being able to see actual power numbers would increase the usefulness of the data even more. Unfortunately, none of these makes the climbs easier. But at least it allows me to understand why one climb feels hard and another easy (relatively speaking). I wonder if, and how this could be applied to running; maybe with smart glasses? Conclusion: yes, I do recommend a bike computer with a large display (to be able to see many fields at once). Just in case one has disposable income at hand and doesn't know which hobby to spend it on

Even though the Chess Olympiad takes some attention right now, development on Nageru (my live video mixer) has continued steadily throughout since the 1.3.0 release. I wanted to take a little time to talk about the upcoming 1.4.0 release, and why things are as they are; writing down things often make them a bit clearer. Every major release of Nageru has had a specific primary focus: 1.0.0 was about just getting everything to work, 1.1.0 was about NVIDIA support for more oomph, 1.2.0 was about stabilization and polish (and added support for Blackmagic's PCI cards as a nice little bonus), and 1.3.0 was about x264 integration. For 1.4.0, I wanted to work on multitrack audio and mixing. Audio has always been a clear focus for Nageru, and for good reason; video is 90% about audio, and it's sorely neglected in most amateur productions (not to mention that processing tools are nearly non-existant in most free or cheap software video mixers). Right from the get-go, it's had a chain with proper leveling, compressors and most importantly visual monitoring, so that you know when things are not as they should be. However, it was also written with an assumption that there would be a single audio input source (one of the cameras), and that's going to change. Single-input is less of a showstopper than one'd think at first; you can work around it by buying a mixer, plug everything into that and then feed that signal into the computer. However, there are a few downsides: If you want camera audio, you'll need to pull more cable from each camera (or have SDI de-embedders). Your mixer is likely to require an extra power supply, and that means yet more cable (any decent USB video card can power over USB, so why shouldn't your audio?). You'll need to buy and transport yet another device. And so on. (If you already have a PA mixer, of course you can use it, but just reusing the PA mix as a stream mix rarely gives the best results, and mixing on an aux bus gives very little flexibility.) So for 1.4.0, I wanted something to get essentially the processing equivalent of a mid-range mixer. But even though my education is in DSP, my experience with mixers is rather limited, so I did the only reasonable thing and went over to a friend who's also an (award-winning) audio engineer. (It turns out that everything on a mixer is the way it is for a pretty good reason, tuned through 50+ years of collective audio experience. If you just try to make up something on your own without understanding what's going on, you have a 0.001% chance of stumbling upon some genius new way of doing things by accident, and a significantly larger chance than that of messing things up.) After some back and forth, we figured out a reasonable set of basic tools that would be useful in the right hands, and not too confusing for a beginner. So let's have a look at the new controls you get: There's one set of these controls for each bus. (This is the expanded view; there's also a compact view that has only the meters and the fader, which is what you'll typically want to use during the run itself the expanded view is for setup and tweaking.) A bus in Nageru is a pair of channels (left/right), sourced from a video capture or ALSA card. The channel mapping is flexible; my USB sound card has 18 channels, for instance, and you can use that to make several buses. Each bus has a name (here I named it very creatively Main , but in a real setting you might want something like Blue microphone or Speaker PC ), which is just for convenience; it doesn't mean much. The most important parts of the mix are given the most screen estate, so even though the way through the signal chain is left-to-right top-to-bottom, I'll go over it in the opposite direction. By far the most important part is the audio level, so the fader naturally is very prominent. (Note that the scale is nonlinear; you want more resolution in the most important area.) Changing a fader with the mouse or keyboard is possible, and probably most people will be doing that, but Nageru will also support USB faders. These usually speak MIDI, for historical reasons, and there are some UI challenges when they're all so different, but you can get really small ones if you want to get that tactile feel without blowing up your budget or getting a bigger backpack. Then there's the meter to the left of that. Nageru already has R128 level meters in the mastering section (not shown here, but generally unchanged from 1.3.0), and those are kept as-is, but for each bus, you don't want to know loudness; you want to know recording levels, so you want a peak meter, not a loudness meter. In particular, you don't want the bus to send clipped data to the master (which would happen if you set it too high); Nageru can handle this situation pretty well (unlike most digital mixers, it mixes in full 32-bit floating-point so there's no internal clipping, and there's a limiter on the master by default), but it's still not a good place to be in, so you can see that being marked in red in this example. The meter doubles as an input peak check during setup; if you turn off all the effects and set the fader to neutral, you can see if the input hits peak or not, and then adjust it down. (Also, you can see here that I only have audio in the left channel; I'd better check my connections, or perhaps just use mono, by setting the right channel on the bus mapping to the same input as the left one.) The compressor (now moved from the mastering section to each bus) should be well-known for those using 1.3.0, but in this view, it also has a reduction meter, so that you can see whether it kicks in or not. Most casual users would want to just leave the gain staging and compressor settings alone, but a skilled audio engineer will know how to adjust these to each speaker's antics (some speak at a pretty even volume and thus can get a bit of headroom, while some are much more variable and need tighter settings). Finally (or, well, first), there's the EQ section. The lo-cut is again well-known from 1.3.0 (the cutoff frequency is the same across all buses), but there's now also a simple three-band EQ per bus. Simply ask the speaker to talk normally for a bit, and tweak the controls until it sounds good. People have different voices and different ways of holding the microphone, and if you have a reasonable ear, you can use the EQ to your advantage to make them sound a little more even on the stream. Either that, or just put it in neutral, and the entire EQ code will be bypassed. The code is making pretty good progress; all the DSP stuff is done (save for some optimizations I want to do in zita-resampler, now that the discussion flow is started again), and in theory, one could use it already as-is. However, there's a fair amount of gnarly support code that still needs to be written: In particular, I need to do some refactoring to support ALSA hotplug (you don't want your entire stream to go down just because an USB soundcard dropped out for a split second), and similarly some serialization of saving/loading bus mappings. It's not exactly rocket science, but all the code still needs to be written, and there are a number of corner cases to think of. If you want to peek, the code is in the multichannel_audio branch, but beware; I rebase/squash it pretty frequently, so if you pull from it, expect frequent git breakage.

Earlier today I was handling a reproducibility bug and decided I had to try a reproducibility test by myself. I tried reprotest, but I was being hit by a disorderfs issue and I was not sure whether the problem was with reprotest or not (at this point I cannot reproduce that anymore). So I decided to hack a simple script to that, and it works. I even included it in devscripts after writing a manpage. Of course reprotest is more complete, extensible, and supports arbitrary virtualization backends for doing the more dangerous/destructive variations (such as changing the hostname and other things that require root) but for quick tests debrepro does the job. Usage examples:

$ debrepro                                 # builds current directory
$ debrepro /path/to/sourcepackage          # builds package there
$ gbp-buildpackage --git-builder=debrepro  # can be used with vcs wrappers as well

debrepro will do two builds with a few variations between them, including $USER, $PATH, timezone, locale, umask, current time, and will even build under disorderfs if available. Build path variation is also performed because by definition the builds are done in different directories. If diffoscope is installed, it will be used for deep comparison of non-matching binaries. If you are interested and don t want to build devscripts from source or wait for the next release, you can just grab the script, save it as debrepro somewhere on your $PATH and make it executable.

It was a real surprise when I saw activity on #791965, which is my ITP bug to package openzwave. As Ralph wrote, the legal status of the Z-Wave standard has been changed. According to a press release of Sigma Designs, the Z-Wave standard is now put into the public domain. As even the specification of the Z-Wave S2 security application framework is available now, the openzwave community is finally able to create a really compatible application which might also pass the Z-Wave certification. Thus there is new hope that there will be an openzwave package in Debian.

FTP assistant This month I marked 257 packages for accept and rejected only 26. Seems to be that I mostly choosed the high quality packages this month. I also sent 12 emails to maintainers asking questions. Debian LTS This was my twenty-sixth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 14.75h. Again, most of the time I choosed packages, where at the end the vulnerable code of the corresponding CVE was not present in the Wheezy version. So I could mark several CVEs for lshell and wget as not-affected, without doing an upload. Unfortunately I had to give up working on chicken. My scheme abilities appear to be rather rusty. Further, I uploaded a test version for php5 that takes care of 17 CVEs and as requested by the LTS users, two additional bugs. After all tests are passed, I will do a real upload with DLA. This month I also had another term of frontdesk work. Other stuff For the Alljoyn framework I fixed a compile issue with gcc 6 and could close RC-bugs
#831127, #831091, and #831198.
My patch was also accepted by upstream. Unfortunately a bug in gtest resulted in #833636. As gcc 6 is the default compiler now in testing, I could also close RC bug #831106.

Nationalpark Bike Marathon 2016 report: gravity rules! "Marathon" is a bit of a misnomer, since I did the short route, not the medium nor the long one. But hey, it's the official name! Setup This race has four possible lengths: "Valader" which is full round-trip Scuol-Scuol (137km/4'026m), "Jauer" (Fuldera-Scuol, 103km/2'934m), "Livignasco" (Livigno-Scuol, 66km/1'871m) and "Put r" (S-chanf to Scuol, 47Km/1'051m). After many debates, I settled on the Livignasco route, as that was what I was reasonably confident to be able to do. The only problem was that this route (and all the longer ones, of course) were going over Pass Chaschauna, which is a quite hard climb. My bike shop person, who did the full length a number of times, just said when hearing the route I was planning: "Chaschauna is a bitch " Bad language, proper characterisation. The route choice also impacts the logistics: except for the full route, the question is where to sleep the night before? One can sleep either at the start place, or sleep in Scuol itself and take the official shuttle to the start place, but this means waking up much earlier. I decided to sleep in Livigno, as with a 7:45 start this would allow me to sleep until ~6:30, have a quick breakfast and be in time for the race. Problem #1 So on Friday (26th) I drove to Scuol, picked up my start number, and then drove over to Livigno and checked in to the hotel. Speaking of picking up the start number, I saw this at the start area: An STI needed as pacer? How fast are these people, I wondered? Day of the race My plan to have a relaxing sleep until relatively late failed. I went to sleep with difficulties: late dinner due to my late arrival in Livigno, and then I was stressed enough about not missing the race that at 05:05 I already woke up, and was checking the clock every five minutes. Finally at 05:30 I gave up and got out of bed, with only about 6 hours of sleep. Problem #2. The other problem was that I na vely thought the hotel will have breakfast from 07:00 or even 07:30. Hah, as if this was Switzerland. The hotel had breakfast from 08:00, no reason to hurry, right? So I had a poor man's breakfast in the room, some energy bars and an ice tea. Problem #3. At least I had time to grab this picture from the hotel room: Not bad. On to the race! But before that, remember to fill my hydration pack with 3L of water, and put it in my not-so-light backpack. Of course I had tools and some spare parts with me, what if something happens? Problem #4, over-planning. The race The setup at the start was pretty easy. Wait near the start until 07:45, then go. For me, not too in front, of course: Pass Chaschauna About 3 kilometres of easy/flat road, and then it starts. Pass Chaschauna, killing your legs softly But at least the pictures were nice! The climb is difficult. It was around 22% for most of the time, and very few people were able to bike up. Not only the absolute meters were the problem, but also the fact that the ground was quite loose, and pushing the bike, or rather climbing up while pushing the bike, was difficult to do in bike shoes with cleats. Proper hiking shoes would have been much more adequate. Maybe this is why despite the effort, some people biked as much as possible? I was careful not to over-exert myself and walked almost all the way up. Hence the pictures But at last, reached the top of the pass, and was very happy to be done with it. The GPX file from my Garmin says it took only 4 minutes (1.2km) after the race start to actually start climbing (it was not obvious we were climbing so early), and then 1h:20m/8Km to get from 1'817m altitude to the highest point at 2'658m. 8Km in 1h:20m, faster than walking pace but quite slower than running pace (on flat terrain, of course). At the top of the pass I was quite happy: And then the descent started: All the way leading to the race I kept thinking only about climbing Chaschauna, and not about going down on it. It was quite an experience, which I won't forget soon. The trail was already well travelled, which means that besides the loose ground in form of large clumps of dried earth, there was a layer of fine, somewhat moist earth, about 3-5 centimetres deep, which made the downhill "interesting". Coupled with what was a much steeper trail than going up I saw most of the time ~33% it's no surprise that many, or even most, of the people were walking their bikes down. The GPX track says that the trail goes 319m down in 1Km (so a 31.9% over one entire kilometre), then another 130m over the next 1.4Km (9.28%), and then (not sure if this is the pass per se or after) another longer segment of ~260m over 3Km (8.56%). The first steep segment needs lots of skill and concentration, the others are normal gradients. But at least things go fast. What took uphill took ~40 minutes for the last 450m up, downhill it took only 15 minutes! Falling I was able, thanks to recent training, to bike down the trail, or at least I thought so. So on the steep segment I was struggling to bike down and keep control of my bike, putting one leg down on some sections but in general being "on the bike", being hit by the dirt I was throwing in the air despite the mud-guards, being careful but at the same time enjoying this difficult section. That is, right up to the point where I got, I think, too comfortable. After the second-to-last curve, the rear wheel slides, I lose balance and fall in the (thankfully soft) dirt a bit more forcefully than I should have, the bike also sliding along in the dirt. No damage to myself, just some trivial scratches; the bike seems a bit shaken (fork was turned 180 , full of dirt, etc.). I get again on the bike and I continue down; however, something seems off: the rear brake level doesn't have a return anymore, I have to push it to go back. Still usable, but strange. So I continue the other two segment of this descent, and over these ~4-5Km I feel the brake issue going worse and worse: it's actually hard to push the brake lever back, and even pressing it doesn't seem to have much effect. And then the realisation dawns on my: I have hydraulic brakes, and the worsening thing leads me to think my brake hose has been punctured (by what? I fell in loose dirt) and I'm losing the brake fluid with every press. Now, on flat road, biking with only the front brake is tricky but doable. On an MTB course, having only one brake is not a smart thing . Not easy, not smart, and definitely dangerous. It also felt very strange to not have both brakes, and then I realised how much I work with the brakes in unison. I knew that there are repair posts in the race, but I didn't know if they can fix such things (and doing a brake fluid refill how complicated is it, how much time does it take?). Anyway, I resolved to try and continue if the terrain allows until I reach the first repair post, and stop if it gets too steep. It would be pretty sad to have to abort the race after only 10Km, right? The terrain did cooperate: up and down, so I was able to bleed speed easily, and even though through forest, the visibility was good enough that I could plan ahead. It was strangely lonely at this time: I was going slowly and thus not reaching from behind anyone, but also not being reached by riders from behind. I did enjoy it a bit, the quietness of the forest, just interrupted by the squeaking of my front brake And then I finally see the sign for "Repair: 1Km". Yaaay! maybe I don't have to quit the race. And then, about 100m further on, another sign: "Danger! Steep descent". Uh-oh tantalisingly close Let's go slow. And yes, here I was overtaken by 3-4 other cyclists. This was the single moment in the race where I did have a moment of real fear: at one point, I was going down with the remaining brake as pressed as I could without compromising stability, and I was still gaining speed. Not smart, and as I was debating how to fall to stop my descent, the terrain started to get less steep, and I finally reached the repair station, at kilometre 21. Difficult repairs There was another person being serviced for a flat rear tire, so I wait in line and discuss with him my problem. Upon hearing I only had the front brake, he said: "Wow, how did you manage to descend this last segment?" "Well, slowly ". My turn came up, I explain my problem and my theory, and the repair guy gives me one look, the bike one look, but doesn't look at the brake hose. Instead, he looks once at the brake lever, sees that another cable guide on my handle bar was bent during the fall and actually was mechanically interfering with the brake lever, unbends that thing, and says "here you go". I was stunned. 15 seconds of looking at the problem without stress, and the problem was solved. I could have done this myself if I took the time to look and think, and not hurry. And I could have enjoyed all the downhill on the pass, with two functioning brakes. Some lessons have to be learned the hard way, and this was one. I got off easy though, as I didn't have a real accident while biking with only one brake. And they say you get wiser as you get older Anyway, brake repaired, I stopped at the Sanit r tent to have my scratches disinfected, and onwards. The following few kilometres until S-chanf I enjoyed very much. S-chanf And then, 2 hours 10 minutes into the race, I exit the forest trail, reach S-chanf, and start climbing and descending and climbing and descending. In the sun, which now (after 10 AM) is not that light anymore. The initial climb at S-chanf, while not much, took away my remaining short-term reserves. The route is 50m ascent, then down and then another 50m ascent, and after a bit of flat another 75m ascent I was getting tired, despite eating energy bars and drinking water (and juice at stops). This goes one for an entire hour, during which I cover about 23Km, so not a bad speed everything considered. But getting more and more tired. There were also some very nice segments, along the river, in the shade of the forest. Some fast downhill, some fast flat. I got overtaken by quite a few real racers, that were going so fast I couldn't keep up with them as in I didn't dare ride that fast even when I had the energy. Clearly they were from the longer segments, and boy were they fast! A number of couples (or pairs?) one woman one man, a number of bigger groups, all going as if they had E-bikes or super-powers, not just biking along tired like me Last climb And then, the last big climb: half an hour of going about 300m up (absolute values, so cumulative number would probably be higher), in the sun, over about 6Km, so only 5% average gradient. But I was tired and even resorted to walking on some portions At least I could take some pictures again: Or have pictures taken: Nice views: At one point, in a bit of a shade, I stop to catch my breath. Somebody else stopped as well and, seeing as I was checking my GPS, asks:

• "How much do we have left, about 10K?"
• I say yes (or was it 15K? the numbers don't match well)
• "And how about altitude?"
• "About 350m left "
• "Sigh "

I got overtaken by a lot of cyclists here, although many of them were from S-chanf route (based on the number colour). Sad The thing is, besides the climbing and the sun, this and the previous part was a very nice route: through small towns, high above the valley, beautiful landscapes, etc. Just enjoying it was hard, since I wasn't a) trained well enough and b) prepared well enough. Last segment, last downhill In any case, after that hard half hour, it's flat (or rather said, average gradient zero): Good road, good views, but I'm on the last virtual bar of my battery, so the remaining 15Km I do in 40 minutes, also "eating" the last ascent meters still about 300 left, I think, but up/down/up/down, so you could reuse kinetic energy to gain potential one. The only question I had now was, the route was at this moment about 400m higher (in absolute altitude) than the finish point, so when do we lose that altitude? The answer was a very very nice answer: in the last 5Km, for an average gradient of 8%, just perfect. I was tired however, so I couldn't really enjoy or go fast here, but it was a good feeling. Finish! And then, entering Scuol, not really believing I'm near the finish, that I will be able to actually stop and relax. My finish picture doesn't do justice to how happy I was at that moment: However, after passing the finish line, it was a bit weird. I just stop now? What next? So weird to just walk, not push pedals, and not go up. Or down. The funny thing here was that I thought I didn't use much of my water (hydration pack) during the race. But soon after finishing, I drank the equivalent of about two glasses (let's say 0.5L) and it was over. So I did manage to drink 2.5L during the race, plus the sport drinks at the stops. Time for stats: my Garmin says 67.07Km, 4h:29m, average speed 15.0kph (hah), 1'615m elevation gain. Official numbers say 66Km, 1'871m elevation, and thus 14.22kph. Ranking-wise:

• 147/192 in the overall men Livignasco route
• 44/60 in my age category

Sigh, could have done better. I was aiming for somewhere around last 33%, not last 25% First classed was ~66% faster than me Going to the train station was hard as in difficult to move my legs. Took one nice picture of Scuol though, so was able to think a bit: And then the long way back to Livigno. Took me 4 h to bike to Scuol, and about 3 hours to go back via public transport. At least I ate a sandwich in the meantime. Then reached Livigno, which was also looking nice: Then got on my bike, which at this point was magically light and easy to ride (it seems I was at least partially recovered), rode to the hotel, packed my stuff, and drove home. On the highway I had my first real meal for that day, but I was quite tired so didn't feel like the achievement it should have been Lessons The biggest takeaway from this race was the huge learning experience I got. The race was itself awesome, but the learning even more so. First, for this particular race, it's much better to sleep in Scuol and (if not doing the whole route) take the shuttle to the actual start. Yes, one has to wake up earlier, but you have assured breakfast via the organisers (++++!), and after the race you don't have to get even more tired by taking the train and the bus and Second, I need better logistics: I don't need to carry that heavy load (spare tube, pump, tools, jacket in case it's cold) when the race is well organised. The water in the hydration pack was useful, but carrying it up Chaschauna was a pain. Not sure what I'll do next; it's possible to survive only on water/drinks from the food posts (basically 2.5-2.8L is equivalent to only 4 750ml bottles, and there were I think about 6 stops?), but the hydration pack is so easy to use Keeping one's head cool and not stress about things also would help: had I looked at my brake lever in peace, I could have solved it myself, and not ride ~10Km with only one brake. Also, losing some weight (as in losing fat, not having a lighter bike) would definitely help. I could stand to lose 5Kg easily, probably 10Kg, and that would bring me more in line with other people in the race (at least looking at the pictures). And hey, my Watts/Kg would improve magically! And last but most important, I need better training. Training on how to use my energy, training longer rides so that my body gets used to it and doesn't bonk after 3 hours, training training training. Which I like, but the problem is time to do it Also, in case anybody wonders: yes, I do recommend this race! It was definitely fun, and the route itself is very nice, nicer than the pictures here show. Closing words: gravity rules. Up or down, it's hard and punishing. Still: Ride on!

Did that ever happen to you: You run apt update, it fetches a Release file, then starts fetching DEP-11 metadata, then any pdiff index stuff, and then applies them; all after another? Or this: You don t see any update progress until very near the end? Worry no more: I tweaked things a bit in 1.3~rc4 (git commit). Prior to 1.3~rc4, acquiring the files for an update worked like this: We create some object for the Release file, once a release file is done we queue any next object (DEP-11 icons, .diff/Index files, etc). There is no prioritizing, so usually we fetch the 5MB+ DEP-11 icons and components files first, and only then start working on other indices which might use Pdiff. In 1.3~rc4 I changed the queues to be priority queues: Release files and .diff/Index files have the highest priority (once we have them all, we know how much to fetch). The second level of priority goes to the .pdiff files which are later on passed to the rred process to patch an existing Packages, Sources, or Contents file. The third priority level is taken by all other index targets. Actually, I implemented the priority queues back in Jun. There was just one tiny problem: Pipelining. We might be inserting elements into our fetching queues in order of priority, but with pipelining enabled, stuff of lower priority might already have their HTTP request sent before we even get to queue the higher priority stuff. Today I had an epiphany: We fill the pipeline up to a number of items (the depth, currently 10). So, let s just fill the pipeline with items that have the same (or higher) priority than the maximum priority of the already-queued ones; and pretend it is full when we only have lower priority items. And that works fine: First the Release and .diff/Index stuff is fetched, which means we can start showing accurate progress info from there one. Next, the pdiff files are fetched, meaning that we can apply them in parallel to any targets downloading later in parallel (think DEP-11 icon tarballs). This has a great effect on performance: For the 01 Sep 2016 03:35:23 UTC -> 02 Sep 2016 09:25:37 update of Debian unstable and testing with Contents and appstream for amd64 and i386, update time reduced from 37 seconds to 24-28 seconds. In other news I recently cleaned up the apt packaging which renamed /usr/share/bug/apt/script to /usr/share/bug/apt. That broke on overlayfs, because dpkg could not rename the old apt directory to a backup name during unpack (only directories purely on the upper layer can be renamed). I reverted that now, so all future updates should be fine. David re-added the Breaks against apt-utils I recently removed by accident during the cleanup, so no more errors about overriding dump solvers. He also added support for fingerprints in gpgv s GOODSIG output, which apparently might come at some point. I Also fixed a few CMake issues, fixed the test suite for gpgv 2.1.15, allow building with a system-wide gtest library (we really ought to add back a pre-built one in Debian), and modified debian/rules to pass -O to make. I wish debhelper would do the latter automatically (there s a bug for that). Finally, we fixed some uninitialized variables in the base256 code, out-of-bound reads in the Sources file parser, off-by-one errors in the tagfile comment stripping code[1], and some memcpy() with length 0. Most of these will be cherry-picked into the 1.2 (xenial) and 1.0.9.8 (jessie) branches (releases 1.2.15 and 1.0.9.8.4). If you forked off your version of apt at another point, you might want to do the same. [1] those were actually causing the failures and segfaults in the unit tests on hurd-i386 buildds. I always thought it was a hurd-specific issue PS. Building for Fedora on OBS has a weird socket fd #3 that does not get closed during the test suite despite us setting CLOEXEC on it. Join us in #debian-apt on oftc if you have ideas.
Filed under: Debian, Ubuntu

Me and Ana travelled to Cambridge last weekend for the Debian UK BBQ. We travelled by train and it was a rather scenic journey. In the past, on long journeys, I ve used APRS-IS to beacon my location and plot my route but I have recently obtained the GPS module for my Yaesu VX-8DE and I thought I d give some real RF APRS a go this time. While the APRS IGate coverage in the UK is a little disappointing, as is evidenced by the map, a few cool things did happen. I recieved a simplex APRS message from a radio amateur 2M0RRT with the text test test IO86ML (but unfortunately didn t notice until we d long passed by, sorry for not replying!) and quite a few of my packets, sent from a 5 watt handheld in Cambridge, were heard by the station M0BPQ-1 in North London (digipeated by MB7UM).

My APRS Position Reports for the Debian UK BBQ 2016

Some of you will know that since my trip to the IETF in Berlin, I ve been without a working laptop (it got a bit smashed up on the plane). This also caused me to miss the reminder to renew the expiry on my old GPG key, which I have now retired. My new GPG key is not yet in the Debian keyring but can be found in the Tor Project keyring already. A request has already been filed to replace the key in the Debian keyring, and thanks to attendees at the BBQ, I have some nice shiny signatures on my new key. (I ll get to returning those signatures as soon as I can.) We ve been making a lot of progress with Debian Live and the BBQ saw live- tasks being uploaded to the archive. This source package builds a number of metapackages, each of which configures a system to be used an a live CD for a different desktop environment. I would like for as much of the configuration that can be done within the image as possible to be done within the image, as this will help with reproducibility. A new upload for live-wrapper should be coming next week, and this version will allow these live-task-* packages to be used to build images for testing. I hope to have weekly builds for the live images running very soon. As I ve been without a computer for a while, I m just getting back into things now, so expect that I ll be slow to respond to communication for a while but I ll also be making commits and uploads and trying to clear this backlog as quickly as I can (including my Tor Project backlog).

Some time ago, I built a static program that I wanted to run on an Android tablet. What was my surprise when I saw a message saying "FATAL: kernel too old". After some investigation, it turns out that GNU libc may assume some Linux features are present during build time. This means that given a minimum Linux version, that built libc might only work on that version or newer. Since 2014, GNU libc itself requires 2.6.32 as the minimum. Previously, it was 2.6.16, changed in 2012. Debian, as of 2015, builds it with a required minimum Linux version of 3.2. To give an idea about the history of these kernel releases, we have:

• December 2009, Linus releases Linux 2.6.32.
• November 2010, Red Hat releases RHEL 6.0, Linux 2.6.32.
• February 2011, Debian 6.0 Squeeze, Linux 2.6.32.
• January 2012, Linus releases Linux 3.2.
• April 2014, GNU Libc requires Linux 2.6.32.
• December 2015, Debian GNU Libc uploaded to unstable requires Linux 3.2.

So, at least for GNU libc upstream, it would appear that not many devices would stop being supported, though the situation would not be as good for binary versions of Debian. However, I have a small list of devices that might show otherwise.

• Nokia N900 shipped with Linux 2.6.28.
• Android emulator, a platform called Goldfish, uses Linux 2.6.29.
• The Wii port most widely tested is based on Linux 2.6.32.

Many Android devices have been shipped with Linux 3.4, but I encountered at least one using Linux 3.0. So, even though many new devices ship with newer versions of Linux, it's still possible to find some new and older devices using versions older than 3.2, and even versions older than 2.6.32 may be found. Another interesting note is that, without a few patches, it's not possible to build Linux 2.6.32 with GCC 5 and newer. For that and many other reasons, it's important that we update. For bug fixes, and so we can make progress and use better software are some of the other reasons. So, it's imperative that we have devices support upstream. Otherwise, the task of doing updates with forward porting becomes daunting. And that's the current state for many devices, which is why I have been trying to use new software with older versions of Linux. But as time passes, I realize how hard a task this is, as most new software these days, even a building block like GNU libc, requires ever new versions of Linux. For now, most gadgets I have support Linux 3.4 or newer. But not long ahead, that support will be dropped as well. And that means there will be no more updates for those devices. It's a consequence of both targeting time-to-market and programmed obsolescence as business practices. Upstream support is no priority, and maintenance is only that required until the next model is available on the stores. This is one more reason why we need to have more operating systems available for those devices. Systems that are designed to last more than a couple of years. As I said, upstream support is imperative, but as a step forward, I still believe we can provide the GNU experience to lots of devices out there.

Here is my monthly update covering what I have been doing in the free software world (previously):

• Worked on nsntrace, a userspace tool to perform network traces on processes using kernel namespaces:
  • Overhauled error handling to ensure the return code of the wrapped process is returned to the surrounding environment. (#10).
  • Permit the -u argument to also accept uids as well as usernames. (#16).
  • Always kill the (hard-looping) udp_send utility, even on test failures. (#13).
  • Updated configure.ac to look for iptables in /sbin & /usr/sbin (#11) and to raise an error if pcap.h is missing (#15).
  • Drop bashisms in #!/bin/sh script (#14) and ignore the generated manpage in the Git repository (#12).
• Independently discovered an regression in the Django web development framework where field__isnull=False filters were not working with some foreign keys, resulting in extending the testsuite and release documentation. (#7104).
• Proposed a change to django-enumfield (a custom field for type-safe constants) to ensure passing a string type to Enum.get returned None on error to match the documentation. (#36).
• Fixed an issue in the Mopidy music player's podcast extension where the testsuite was failing tests in extreme timezones. (#40).
• Proposed changes to make various upstream's reproducible:
  • botan, a crypto/TLS library for C++11. (#587).
  • cookiecutter, a project template generator, removing nondeterministic keyword arguments from appearing in the documentation. (#800).
  • pyicu, a Python wraper for the IBM Unicode library. (#27).
• Integrated a number of issues raised by @piotr1212 to python-fadvise, my Python interface to posix_fadvise(2), where the API was not being applied to open file descriptors (#1) and moving the .so to a module directory (#2).
• Various improvements to try.diffoscope.org, a hosted version of the diffoscope in-depth and content-aware diff utility, including introducing an HTTP API (#21), updating the SSL certificate and correcting a logic issue where errors in diffoscope itself were not being detected correctly (b0ff49). Continued thanks to Bytemark for sponsoring the hardware.
• Fixed a bug in django-slack, my library to easily post messages to the Slack group-messaging utility, correcting an EncodeError exception under Python 3 (#53) and updated the minimum required version of Django to 1.7 (#54).
• Various updates to tickle-me-email, my Getting Things Done-inspired email toolbox, to also match / in IMAP's LIST separators (#6) and to encode the folder list as UTF-7 (#7). Thanks to @resiak.
• Clarified the documentation for travis.debian.net my hosted script to easily test and build Debian packages on the Travis CI continuous integration platform regarding how to integrate with Github (#20).

---

Chris Done s automatic Haskell formatter hindent is released in a new version, and getting quite a bit of deserved attention. He is polling the Haskell programmers on whether two or four spaces are the right indentation. But that is just cosmetics I am in principle very much in favor of automatic formatting, and I hope that a tool like hindent will eventually be better at formatting code than a human. But it currently is not there yet. Code is literature meant to be read, and good code goes at length to be easily readable, and formatting can carry semantic information. The Haskell syntax was (at least I get that impression) designed to allow the authors to write nicely looking, easy to understand code. One important tool here is vertical alignment of corresponding concepts on different lines. Compare

maze :: Integer -> Integer -> Integer
maze x y
  abs x > 4    abs y > 4  = 0
  abs x == 4   abs y == 4 = 1
  x ==  2    && y <= 0     = 1
  x ==  3    && y <= 0     = 3
  x >= -2    && y == 0     = 4
  otherwise                = 2

with

maze :: Integer -> Integer -> Integer
maze x y
  abs x > 4   abs y > 4 = 0
  abs x == 4   abs y == 4 = 1
  x == 2 && y <= 0 = 1
  x == 3 && y <= 0 = 3
  x >= -2 && y == 0 = 4
  otherwise = 2

The former is a quick to grasp specification, the latter (the output of hindent at the moment) is a desert of numbers and operators. I see two ways forward:

• Tools like hindent get improved to the point that they are able to detect such patterns, and indent it properly (which would be great, but very tricky, and probably never complete) or
• We give the user a way to indicate intentional alignment in a non-obtrusive way that gets detected and preserved by the tool.

What could such ways be?

• For guards, it could simply detect that within one function definitions, there are multiple on the same column, and keep them aligned.
• More general, one could take the approach lhs2Tex (which, IMHO, with careful input, a proportional font and with the great polytable LaTeX backend, produces the most pleasing code listings) takes. There, two spaces or more indicate an alignment point, and if two such alignment points are in the same column, their alignment is preserved even if there are lines in between! With the latter approach, the code up there would be written

  maze :: Integer -> Integer -> Integer
  maze x y
    abs x > 4      abs y > 4   = 0
    abs x == 4     abs y == 4  = 1
    x ==  2     &&  y <= 0      = 1
    x ==  3     &&  y <= 0      = 3
    x >= -2     &&  y == 0      = 4
    otherwise                   = 2

  And now the intended alignment is explicit.

(This post is cross-posted on reddit.) Update (2016-09-05) Shortly after this post, the Haskell formatter brittany gets released, which supports vertial alignment. Yay!

Four months after 1.0 release, motranslator 1.1 is out. If you happen to use it for untrusted data, this might be as well called security release, though this is still not good idea until we remove usage of eval() used to evaluate plural formula. Full list of changes:

• Improved handling of corrupted mo files
• Minor performance improvements
• Stricter validation of plural expression

The motranslator is a translation library used in current phpMyAdmin master (upcoming 4.7.0) with focus on speed and memory usage. It uses Gettext MO files to load the translations. It also comes with testsuite (100% coverage) and basic documentation. Recommended way to install it is using composer from Packagist repository: The Debian package will be available probably at point phpMyAdmin 4.7.0 will be out, but if you see need to have it earlier, just let me know.

Filed under: Debian English phpMyAdmin 0 comments

There is an ongoing campaign for motion comic. It will be done entirely with FLOSS tools (Blender, Krita, GNU/Linux) and besides that, it really looks great (and no, it is not only for the kids!). Please support this effort if you can because it also shows the power of Free software tools. All will be released Creative Commons Atribution-ShareAlike license together with all sources.

It has been a while that I have been contacted by a recruiter, and the last few ones were fairly decent conversations, where they made an effort to research me first, and even if they did not get everything right, they still listened, and we had a productive talk. But four days ago, I had another recruiter reach out to me, from a company I know oh so well: one I ranted about before: Google. Apparently, their recruiters still do carpet-bombing style outreach. My first thought was "what took them so long?" - it has been five years since my last contact with a Google recruiter. I almost started missing them. Almost. To think that Google is now powerful enough to read my mind, is scary. Yet, I believe, this is not the case; rather, it's just another embarrassing mistake.To make my case, let me quote the full e-mail I was sent, with the name of the sender redacted, and my comments - which I'm also sending to the recruiter:

Hi Gergely,

Hi!

How are you? Hope you're well.

Thank you, I'm fine, just back from vacation, and I was thrilled to read your e-mail. Although, I did find it surprising too, and considering past events, I thought it to be spam first.

My name is XXX and I am recruiting for the Google Engineering team. I have just found your details on Github...

I'm happy that you found me through GitHub, but I'm curious why you mailed me at my debian.org address then? That address is not on my GitHub profile, and even though I have some code signed with that address, that's not what I use normally. My GitHub profile also links to a page I created especially for recruiters, which I don't think you have read - but more on that below.

...and your experience with software development combined with your open source contributions is particularly relevant to Google

Which part of my recent contributions are relevant to Google? For the past few months, the vast majority (over 90%) of my open source work has been on keyboard firmware. If Google is developing a keyboard, then this may be relevant, otherwise, I find it doubtful.Some of my past OSS contributions may be more relevant, but that's in the past, and it would take some digging to see those from GitHub. And if you did that kind of digging, you would have found the page on my site for recruiters, and would not have e-mailed me at my Debian address, either.

We are always interested in talking to top engineers with your mix of skills and I was wondering if you are at all open to exploring roles with Google in EMEA.

To make things short, my stance is the same as it was five years ago, when I wrote - and I quote - "So, here's a public request: do not email me ever again about job opportunities within Google. I do not wish to work for Google. Not now, not tomorrow, not ever."This still stands. Please do not ever e-mail me about job opportunities within Google. I do not wish to work for Google, not now, not tomorrow, not five years from now, not ever. This will not change. Many things may change, this however, will not.But even if I ignore this, let me ask you mix of skills are you exactly interested in? Keyboard firmware hacking, mixed with Emacs Lisp, some Clojure, and hacking on Hy (which, if you have explored my GitHub profile, will surely know) from time to time? Or is it my Debian Developer hat that got your interest? If so, why didn't you say so? Not that it would change anything, but I'm curious.Is it my participation in 24pullrequests? Or my participation in GSoC in years past?Nevertheless, my list of conditions I mentioned above, apply. Is Google able to fulfill all my requirements, and the preferences too? Last time I heard, working for Google required one to relocate, which I clearly said on that page, I'm not willing to do.

The teams I recruit for are responsible for Google's planet-scale systems. Just to give you more of an idea, some of the projects we work on that might be of interest to you would include:

• MapReduce
• App Engine
• Mesa
• Maglev

And how would these be interesting to me, considering my recent OSS work? (Hint: none of them interest me, not a bit. Ok perhaps MapReduce, a little.)

I think you could be a great fit here, where you can develop a great career and at the same time you will be part of the most mission critical team, designing and developing systems which run Google Search, Gmail, YouTube, Google+ and many more as you can imagine.

My career is already developing fine, thank you, I do not need Google to do that. I am already working on things I consider important and interesting, if I wanted to change, I would. But I certainly would not consider a company which I had asked numerous times NOT to contact me about opportunities. If you can't respect this wish, and forget about this in mere five years, why would I trust you to keep any other promises you make now?

Thank you so much for your time and I look forward to hearing from you.

I wish you had spent as much time researching me - or even half that - as I have spent replying to you. I suppose this is not the reply you were expecting, or looking for, but this is the only one I'll ever give to anyone from Google.And here, at the end, if you read this far, I'm asking you, and everyone at Google to never contact me about job opportunities. I will not work for Google. Not now, not tomorrow, not five years from now, not ever. Please do not e-mail me again, and do not reply to this e-mail either. I'm not interested in neither apologies, nor promises that you won't contact me - just simply don't do it.Thank you.

I was recently asked to get data from a computer that controlled security cameras after a crime had been committed. Due to the potential issues I refused to collect the computer and insisted on performing the work at the office of the company in question. Hard drives are vulnerable to damage from vibration and there is always a risk involved in moving hard drives or systems containing them. A hard drive with evidence of a crime provides additional potential complications. So I wanted to stay within view of the man who commissioned the work just so there could be no misunderstanding. The system had a single IDE disk. The fact that it had an IDE disk is an indication of the age of the system. One of the benefits of SATA over IDE is that swapping disks is much easier, SATA is designed for hot-swap and even systems that don t support hot-swap will have less risk of mechanical damage when changing disks if SATA is used instead of IDE. For an appliance type system where a disk might be expected to be changed by someone who s not a sysadmin SATA provides more benefits over IDE than for some other use cases. I connected the IDE disk to a USB-IDE device so I could read it from my laptop. But the disk just made repeated buzzing sounds while failing to spin up. This is an indication that the drive was probably experiencing stiction which is where the heads stick to the platters and the drive motor isn t strong enough to pull them off. In some cases hitting a drive will get it working again, but I m certainly not going to hit a drive that might be subject to legal action! I recommended referring the drive to a data recovery company. The probability of getting useful data from the disk in question seems very low. It could be that the drive had stiction for months or years. If the drive is recovered it might turn out to have data from years ago and not the recent data that is desired. It is possible that the drive only got stiction after being turned off, but I ll probably never know. Doing it Properly Ever since RAID was introduced there was never an excuse for having a single disk on it s own with important data. Linux Software RAID didn t support online rebuild when 10G was a large disk. But since the late 90 s it has worked well and there s no reason not to use it. The probability of a single IDE disk surviving long enough on it s own to capture useful security data is not particularly good. Even with 2 disks in a RAID-1 configuration there is a chance of data loss. Many years ago I ran a server at my parents house with 2 disks in a RAID-1 and both disks had errors on one hot summer. I wrote a program that s like ddrescue but which would read from the second disk if the first gave a read error and ended up not losing any important data AFAIK. BTRFS has some potential benefits for recovering from such situations but I don t recommend deploying BTRFS in embedded systems any time soon. Monitoring is a requirement for reliable operation. For desktop systems you can get by without specific monitoring, but that is because you are effectively relying on the user monitoring it themself. Since I started using mon (which is very easy to setup) I ve had it notify me of some problems with my laptop that I wouldn t have otherwise noticed. I think that ideally for desktop systems you should have monitoring of disk space, temperature, and certain critical daemons that need to be running but which the user wouldn t immediately notice if they crashed (such as cron and syslogd). There are some companies that provide 3G SIMs for embedded/IoT applications with rates that are significantly cheaper than any of the usual phone/tablet plans if you use small amounts of data or SMS. For a reliable CCTV system the best thing to do would be to have a monitoring contract and have the monitoring system trigger an event if there s a problem with the hard drive etc and also if the system fails to send a I m OK message for a certain period of time. I don t know if people are selling CCTV systems without monitoring to compete on price or if companies are cancelling monitoring contracts to save money. But whichever is happening it s significantly reducing the value derived from monitoring.

What happened in the Reproducible Builds effort between Sunday August 21 and Saturday August 27 2016: GSoC and Outreachy updates

• Satyam Zode's final report for GSOC 2016

Packages reviewed and fixed, and bugs filed

• #834976 filed against auto-apt-proxy by Chris Lamb.
• #835637 filed against myghty by Chris Lamb.
• #835061 filed against varnish by Chris Lamb.
• #835633 filed against pleiades by Chris Lamb.
• #835625 filed against nikwi by Chris Lamb.
• #835263 filed against binutils-m68hc1x by Chris Lamb.
• #835448 filed against eekboek by Chris Lamb.
• #835143 filed against ttf-tiresias by Chris Lamb.
• #835637 filed against myghty by Chris Lamb.
• #835495 filed against broccoli by Chris Lamb.
• #835129 filed against dateutils by Chris Lamb.
• #835051 filed against sheepdog by Chris Lamb.
• #835145 filed against udpcast by Chris Lamb.
• #834983 filed against eyed3 by Chris Lamb.
• #835617 filed against congress by Chris Lamb.
• #835376 filed against lilyterm by Chris Lamb.
• #835130 filed against ircd-ircu by Chris Lamb.
• #835262 filed against radare2 by Chris Lamb.
• #835193 filed against phpdox by Chris Lamb.
• #835265 filed against argyll by Chris Lamb.
• #835259 filed against quvi by Chris Lamb.
• #835371 filed against dispcalgui by Chris Lamb.
• #835147 filed against javatools by Chris Lamb.
• #834988 filed against twitter-bootstrap3 by Chris Lamb.
• #835463 filed against fdroidserver by Chris Lamb.
• #835646 filed against dh-lua by Chris Lamb.
• #835447 filed against libmodule-build-withxspp-perl by Chris Lamb.
• #835418 filed against libfm by Chris Lamb.
• #835143 filed against ttf-tiresias by Chris Lamb.
• #834993 filed against oss4 by Reiner Herrmann.

Reviews of unreproducible packages 10 package reviews have been added and 6 have been updated this week, adding to our knowledge about identified issues. A large number of issue types have been updated:

• Add captures_build_path issue and some packages affected by it
• Add golang_compiler_captures_build_path_in_binary and move obfs4proxy to it
• Move 7 golang packages from captures_build_path to golang_compiler_captures_build_path_in_binary
• Add new:
  • random_order_in_lua_version_substvar
  • href_links_mangled_by_node_marked
  • perl_extutils_xspp_captures_build_path
  • timestamp_added_by_java_util_properties
• Fixes for issues created:
  • perl_extutils_xspp_captures_build_path
  • random_order_in_lua_version_substvar
  • random_order_in_javahelper_substvars
• Patch for zope_random_field_order_in_dzproduct uploaded.
• Rename:
  • use_epydoc randomness_in_documentation_generated_by_epydoc so it at least matches the others
  • random_order_in_javahelper_depends issue to random_order_in_javahelper_substvars
• Fix the link for the golang issue, previous link is for random_build_path_by_golang_compiler a different issue
• Add a tip regarding how to call ./configure for rust
• Add offending source line for gcc-defaults.

Weekly QA work 29 FTBFS bugs have been reported by:

• Chris Lamb (27)
• Daniel Stender (1)
• Santiago Vila (1)

diffoscope development

• Chris Lamb:
  • Add tests for skip_unless_tool_exists helper.
  • comparators/elf.py: Specify string format arguments as logging function parameters, not using interpolation.
  • presenters/html:
    • Use html.escape over xml.sax.saxutils.escape.
    • Don't use unsafe cgi.escape method as its quote kwarg -- which enables escaping of quotation marks -- is False by default.
  • Tidy imports in Debian comparator and tests.
  • Skip Haskell tests if GHC version does not match. (Closes: #835055)
  • Use pytest.xfail over assert False.
  • Use the debian_fallback.X as the fallback for debian.DotBuildinfoFile (not debian.X).
  • Rename skip_unless_tool_exists -> skip_unless_tools_exist and fix logic.
  • Avoid ugly DRY violations in diffoscope.comparators.__init__ by dynamically importing classes via a single list.
• Satyam Zode:
  • Improve diffoscope behaviour for .changes
• Mattia Rizzolo:
  • Be even more verbose about failing tests
  • d/control: alternate build-dependency on default-jdk-headless default-jdk, to ease backporting to older debian releases
  • add default-jdk to the alternate packages for javap for Debian; default-jdk-headless is not available in older Debian releases
  • in the tests only, normalize xxd's output so that we can compare jessie's xxd with stretch's
  • tests:
    • skip test_squashfs.py.test_superblock if squashfs is too old
    • rewrite tool_older_than() into skip_unless_tool_is_older_than()
    • factor out a tools_missing() function
• J r my Bobbio:
  • Properly skip test requiring python-debian when unavailable
• Ximin Luo:
  • Add a --no-max flag to disable all limits and have max_report_size also honour 0 to mean "no limit"
  • Actually only scan whole file when filename ends in ".rom"
  • Show the timestamp when logging so I know which steps take longer

Holger also created another test job for diffoscope on jenkins.debian.net, so that now also all commits to branches other than master are being tested. strip-nondeterminism development strip-nondeterminism 0.023-1 was uploaded by Chris Lamb:

 * Support Android .apk files with the JAR normalizer.
 * handlers/png.pm: Drop unused Archive::Zip import
 * Remove hyphen from non-determinism and non-deterministic.
 * javaproperties.pm: Match more styles of .properties and loosen filename matching.
 * Improve tests:
   - Make fixture runner generic to all normalizer types.
   - Replace (single) pearregistry test with a fixture.
   - Set a canonical time for fixture tests.
   - Add gzip testcase fixture.
   - Replace t/javadoc.t with fixture
   - Replace t/ar.t with a fixture.
   - t/javaproperties: move pom.properties and version.properties tests to fixtures
   - t/fixtures.t: move to using subtests
   - t/fixtures.t: Explicitly test that we can find a normalizer
   - t/fixtures.t: Don't run normalizer if we didn't find one.

strip-nondeterminism 0.023-2 uploaded by Mattia Rizzolo to allow stderr in autopkgtest. disorderfs development

• Chris Lamb:
  • Add --sort-dirents=yes no option for forcing deterministic.
  • Testsuite:
    • Add tests for sorting and reversing directory entries.
    • shuffle: Test that --shuffle-dirents works as directed.
    • common: Factor out utility to get variations from mount target.
    • common: Factor out "Fail" utility.

tests.reproducible-builds.org Debian:

• Since we introduced build path variations for unstable and experimental last week, our IRC channel has been flooded with notifcations about packages becoming unreproducible - and you might have noticed some of your packages having become unreproducible recently too. To make our IRC more bearable again, notifications for status changes on i386 and armhf have been disabled, so that now we only get notifications for status changes in unstable. (h01ger)
• Link to jenkins documentation in every page (h01ger)
• The "pre build" check, whether a node is up, now also detects if a node has a read-only filesystem, which sometimes happens on some broken armhf nodes. (h01ger)
• To further improve monitoring of those armhf nodes Work to make them send mails (through an ISP which is blocking outgoing mails) has been started and should be finished next week. (h01ger)
• As one of the armhf nodes (opi2a) is acting strange, a workaround has been added to make it's deployment work despite that. (h01ger)
• Collapse whitespace to avoid ugly "trailing underlines" in hyperlinks for diffoscope results and pkg sets (Chris Lamb)
• Give details HTML elements "cursor: pointer" CSS property to highlight they are clickable. (Chris Lamb)
• The db connection timeout has been raised to a minute when using SQLAlchemy too. (h01ger).

Somewhat related to reproducible builds there has been a first Debian jenkins team maintainance meeting on the #debian-qa IRC channel, to discuss current issues with the setup and to start the work of migrating jenkins.debian.net to jenkins.debian.org. The next meeting will take place on September 28th 2016 at 19 UTC. Misc. This week's edition was written by Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

There has been some discussion [0] around my decision to drop sysvinit support in the conntrackd package in Debian [1] (version 1:1.4.4-2).

The rationale I used for such a movement was sent to the debian-devel [2] mailing list, and here it is:

• the default init system in debian is systemd
• I don't have any sysvinit system to keep sysvinit files under any kind of maintenance
• sysvinit conntrackd script is really poor, to reliably use conntrackd as a system daemon you should use systemd
• conntrackd & systemd are very good integrated (using libsystemd)
• systemd is starting to drop support for some sysvinit mechanisms [3]
• it's time to let sysvinit die

Lots of people have talked to me with both support and disagreement with the change.

Before reading the rest of this blogpost, please note that I'm not interested in the 'systemd vs sysvinit' war, and starting now I will focus mainly on the subject of building a firewall cluster with netfilter technology and the reasons why I think sysvinit here is irrelevant.

I started working with firewall clusters by the time of Debian Squeeze. By then, I used only sysvinit, because it was the Debian default init system and because I did not dive so much in the internals of the firewall cluster itself.

By then, the conntrackd debian package included support for sysvinit by means of two files (the two files that I dropped in 1:1.4.4-2) :

• /etc/init.d/conntrackd
• /etc/default/conntrackd

Using both files, you could start/stop conntrackd, and nothing more (for example, a proper status check was not implemented).

The conntrackd daemon is used in HA firewall clusters to replicate connection states between nodes of the cluster, so flow states are known in all nodes and they can properly perform stateful firewalling.

When you build HA clusters, there are 2 basic states of the cluster you may check and adjust: failover and failback.

With conntrackd, the failover situation is straight forward: the other node has the state information already from the dying node.
The failback situation is different: depending on the configuration of both the cluster and conntrackd, the new node may ask for a complete synchronisation to the other node when it become alive again (at boot time, for example) . This is the case if you are building a multi-master firewall cluster (i.e: both nodes are seeing traffic and filtering at the same time).

Asking for a complete synchronisation is done by means of a new instance of conntrackd, which communicates using a UNIX socket to the main conntrackd daemon (the one which is actually communicating with the other node). A failback boot procedure may look like this:

1. system boots
2. firewall ruleset loading
3. networking up
4. conntrackd up (main sync daemon: using 'conntrackd -d'; UNIX socket is opened)
5. request sync with other node (using conntrackd -n' which uses the UNIX socket opened in step 4)

In both sysvinit and systemd cases, the step 5 may be implemented using another dummy service which performs the required operations and that depends on the main service representing the step 4.

The point here is that steps 4 and 5 suffer a very bad race condition: the conntrackd daemon may open the UNIX socket (step 4) *after* the we try to use it (step 5).

As you could probably imagine, this is the worst possible scenario: as one of the nodes of the cluster is missing important flow state information the stateful firewall will drop packets and all the network monitoring alarms will start to ring... Ironically, just after a failback operation, when all of our cluster is supposed to be up an running.

To avoid the race conditions, some typical hacks could be implemented:

• a wait loop for the socket
• manual delay (sleep time) between step 4 and step 5
• some kind of poll mechanism
• conntrackd somehow notifies of UNIX socket being opened

After a bit of research, I found that the last point could be implemented very easily using libsystemd, so the daemon inform of its internal state to systemd [4].

The solution is elegant, simple and offers more direct benefits:

• systemd watchdog support
• no guess games with the PID of the daemon
• shutdown detection by systemd (so if the daemon was killed with 'conntrackd -k', systemd detects it)
• we get the service 'status' check (missing in sysvinit)

[0] https://lists.debian.org/debian-devel/2016/08/msg00448.html
[1] https://tracker.debian.org/pkg/conntrack-tools
[2] https://lists.debian.org/debian-devel/2016/08/msg00456.html
[3] https://sources.debian.net/src/systemd/231-4/debian/systemd.NEWS/
[4] http://git.netfilter.org/conntrack-tools/tree/src/systemd.c