Traffic jams suck; but they do give you a chance to think and blog. Truckers hams are telling me that four trucks are totalled and need to be removed by the THW. As the trucker next to me just broke out his gas grill and several people are joining him in a spontaneous BBQ, I don't expect to be moving any time soon. Kicking things along This is coming along nicely RAID 4 Unless you are forced to support legacy systems or playing RAIDemon and gotta try them all, there is no valid use case for RAID 4. So doing so is a patently bad idea. RAID 5 is always the better option as you are not relying on a single disk which contains the checksums, but use striping to distribute them. RAID 4 has a chance of 1/n that a failed disk will force all remaining disks to read all respective sectors and then the controller/CPU to XOR everything. There is a reason why RAID 3 was replaced with RAID 4; just as there's a reason why RAID 4 was replaced by RAID 5. Don't use RAID 4. Seriously. Tom Tom vs OSM In case you have been following the news in the GPS circles, you will have noticed that TomTom pretty gave up on innovating and resigned themselves to dying a slow death. Well that and to start a nice amount of FUD. While they have been rebuked pretty thoroughly already, I wanted to add my own POV. There is a kernel of truth in what TomTom said inasmuch they state that pedestrian paths are better than car paths. Of course, while they got the relative quality right, they are wrong about the absolute quality of the data. They claim pedestrian is OK and motorized is bad. In my experience, maps for motorized traffic are good while pedestrian tracks are mind-boggingly great. These days, most people (and probably all who read this) will be at least somewhat familiar with the quality of OSM in the random cities they visit over the year. The common verdict is usually "very nice". But that's not where OSM shines. It's in the lesser-known places where you don't expect too many tech-minded people to come along where it starts to really impress me. While I found a few stubs on a dead-end street in Akureyri which were not mapped, every foot trail I ever walked along in Iceland has been mapped. ski-doo tracks on Svalbard? Check. Backyard in Novosibirsk? Check, as well. And we are talking "packed earth track in between garages opening up into a 'playground' where a man and a pregnant women are passing a bottle back and forth while smoking and watching over their two kids who play next to a middle-aged man who is lying on the ground wasted, using a rock as a pillow all the while an extremely blingy Mercedes with two muscle-packed thugs is cruising by, all in the time it took us to walk through". Mapped. Horse tracks and hiking trails in Mongolia? Yep. Hutongs in Beijing including alleys so narrow you have to walk single file? Of course. Richard Fairhurst is right that the plural of anecdote is not data, but OSM is good. TomTom is afraid for a reason. As an aside, I find the picture used in TomTom's newsletter extremely funny; I am sure they didn't intend this, but hey. Windows 100-300 MiB for any random driver installation package is par for the course. Be it GFX, audio, or network card. If you plug in a different USB mouse, Windows 7 will take several seconds while "setting up" a standard USB HID device. And hitting ctrl-alt-del in the login screen or while logged in does not give you a quick way to shut down or reboot the machine. As I am breaking out the Windows installation every month to install updates and every six months to actually use this does not affect me a lot. Still, it's amazing that so many people are putting up with this. UEFI It's nice to see that Matthew Garrett is still going strong. I am not sure if this is the best way, but it seems to be a workable solution while in between a rock and a hard place. Not sure about the requirements of GPLv3 with regards to the keys Fedora will use to sign GRUB 2 with. From what I remember without rereading the license, as long as Fedora is not selling a system on which you can not disable secure boot they should be in the clear. IANAL, IIRC, etc. Wrapping up And I have not moved a single meter, other than making way to a police motorbike, in over an hour. Pity I don't have anything to BBQ with me; smells delicious over there.

Edit 16:17EDT 31/5/12: Clarification of who gets the $99
Edit 02:10EDT 01/6/12: Clarification that it's a one-off payment

(Brief disclaimer - while I work for Red Hat, I'm only going to be talking about Fedora here. Anything written below represents only my opinions and my work on Fedora, not Red Hat's opinions or future plans)

Fedora 17 was released this week. It's both useful and free, and serves as a welcome addition to any family gathering. Do give it a go. But it's also noteworthy for another reason - it's the last Fedora release in the pre-UEFI secure boot era. Fedora 18 will be released at around the same time as Windows 8, and as previously discussed all Windows 8 hardware will be shipping with secure boot enabled by default. While Microsoft have modified their original position and all x86 Windows machines will be required to have a firmware option to disable this or to permit users to enrol their own keys, it's not really an option to force all our users to play with hard to find firmware settings before they can run Fedora. We've been working on a plan for dealing with this. It's not ideal, but of all the approaches we've examined we feel that this one offers the best balance between letting users install Fedora while still permitting user freedom.

Getting the machine booted
Most hardware you'll be able to buy towards the end of the year will be Windows 8 certified. That means that it'll be carrying a set of secure boot keys, and if it comes with Windows 8 pre-installed then secure boot will be enabled by default. This set of keys isn't absolutely fixed and will probably vary between manufacturers, but anything with a Windows logo will carry the Microsoft key[1].

We explored the possibility of producing a Fedora key and encouraging hardware vendors to incorporate it, but turned it down for a couple of reasons. First, while we had a surprisingly positive response from the vendors, there was no realistic chance that we could get all of them to carry it. That would mean going back to the bad old days of scouring compatibility lists before buying hardware, and that's fundamentally user-hostile. Secondly, it would put Fedora in a privileged position. As one of the larger distributions, we have more opportunity to talk to hardware manufacturers than most distributions do. Systems with a Fedora key would boot Fedora fine, but would they boot Mandriva? Arch? Mint? Mepis? Adopting a distribution-specific key and encouraging hardware companies to adopt it would have been hostile to other distributions. We want to compete on merit, not because we have better links to OEMs.

An alternative was producing some sort of overall Linux key. It turns out that this is also difficult, since it would mean finding an entity who was willing to take responsibility for managing signing or key distribution. That means having the ability to keep the root key absolutely secure and perform adequate validation of people asking for signing. That's expensive. Like millions of dollars expensive. It would also take a lot of time to set up, and that's not really time we had. And, finally, nobody was jumping at the opportunity to volunteer. So no generic Linux key.

The last option wasn't hugely attractive, but is probably the least worst. Microsoft will be offering signing services through their sysdev portal. It's not entirely free (there's a one-off $99 fee to gain access edit: The $99 goes to Verisign, not Microsoft - further edit: once paid you can sign as many binaries as you want), but it's cheaper than any realistic alternative would have been. It ensures compatibility with as wide a range of hardware as possible and it avoids Fedora having any special privileges over other Linux distributions. If there are better options then we haven't found them. So, in all probability, this is the approach we'll take. Our first stage bootloader will be signed with a Microsoft key.

Bootloaders
We've decided to take a multi-layer approach to our signing for a fairly simple reason. Signing through the Microsoft signing service is a manual process, and that's a pain. We don't want to have bootloader updates delayed because someone needs to find a copy of Internet Explorer and a smartcard and build packages by hand. Instead we're writing a very simple bootloader[2]. This will do nothing other than load a real bootloader (grub 2), validate that it's signed with a Fedora signing key and then execute it. Using the Fedora signing key there means that we can build grub updates in our existing build infrastructure and sign them ourselves. The first stage bootloader should change very rarely, and we don't envisage updating it more than once per release cycle. It shouldn't be much of a burden on release management.

What about grub? We've already switched Fedora 18 over to using grub 2 by default on EFI systems, but it still needs some work before it's ready for secure boot. The first thing is that we'll be disabling the module loading. Right now you can load arbitrary code into grub 2 at runtime, and that defeats the point of secure boot. So that'll be disabled. Next we'll be adding support for verifying that the kernel it's about to boot is signed with a trusted key. And finally we'll be sanitising the kernel command line to avoid certain bits of functionality that would permit an attacker to cause even a signed kernel to launch arbitrary code. These restrictions will all vanish if secure boot is disabled.

Kernel
Secure boot is built on the idea that all code that can touch the hardware directly is trusted, and any untrusted code must go through the trusted code. This can be circumvented if users can execute arbitrary code in the kernel. So, we'll be moving to requiring signed kernel modules and locking down certain aspects of kernel functionality. The most obvious example is that it won't be possible to access PCI regions directly from userspace, which means all graphics cards will need kernel drivers. Userspace modesetting will be a thing of the past. Again, disabling secure boot will disable these restrictions.

Signed modules are obviously troubling from a user perspective. We'll be signing all the drivers that we ship, but what about out of tree drivers? We don't have a good answer for that yet. As before, we don't want any kind of solution that works for us but doesn't work for other distributions. Fedora-only or Ubuntu-only drivers are the last thing anyone wants, and this really needs to be handled in a cross-distribution way.

Wait signed what
Secure boot is designed to protect against malware code running before the operating system. This isn't a hypothetical threat. Pre-boot malware exists in the wild, and some of it is nastier than you expect. So obviously bootloaders need to be signed, since otherwise you'd just replace the signed bootloader with an unsigned one that installed malware and booted your OS.

But what about the kernel? The kernel is just code. If I take a signed Linux bootloader and then use it to boot something that looks like an unsigned Linux kernel, I've instead potentially just booted a piece of malware. And if that malware can attack Windows then the signed Linux bootloader is no longer just a signed Linux bootloader, it's a signed Windows malware launcher and that's the kind of thing that results in that bootloader being added to the list of blacklisted binaries and suddenly your signed Linux bootloader isn't even a signed Linux bootloader. So kernels need to be signed.

And modules? Again, modules are just code. It's a little trickier, but if your signed kernel loads an unsigned module then that unsigned module can set up a fake UEFI environment and chain into a compromised OS bootloader. Now the attacker just has to include a signed kernel and a minimal initramfs that loads their malware module. It'd slow down boot by a couple of seconds, but other than that it'd be undetectable. X? If you can access the registers on a GPU then you can get the GPU to DMA over the kernel and execute arbitrary code. Trickier again, but still achievable - and if you've locked down every other avenue of attack, it's even attractive.

If we produce signed code that can be used to attack other operating systems then those other operating systems are justified in blacklisting us. That doesn't seem like a good outcome.

Customisation
A lot of our users want to build their own kernels. Some even want to build their own distributions. Signing our bootloader and kernel is an impediment to that. We'll be providing all the tools we use for signing our binaries, but for obvious reasons we can't hand out our keys. There's three approaches here. The first is for a user to generate their own key and enrol it in their system firmware. We'll trust anything that's signed with a key that's present in the firmware. The second is to rebuild the shim loader with their own key installed and then pay $99 and sign that with Microsoft. That means that they'll be able to give copies to anyone else and let them install it without any fiddling. The third is to just disable secure boot entirely, at which point the machine should return to granting the same set of freedoms as it currently does.

But I don't trust Microsoft
A system in custom mode should allow you to delete all existing keys and replace them with your own. After that it's just a matter of re-signing the Fedora bootloader (like I said, we'll be providing tools and documentation for that) and you'll have a computer that will boot Fedora but which will refuse to boot any Microsoft code. It may be a little more awkward for desktops because you may have to handle the Microsoft-signed UEFI drivers on your graphics and network cards, but this is also solvable. I'm looking at ways to implement a tool to allow you to automatically whitelist the installed drivers. Barring firmware backdoors, it's possible to configure secure boot such that your computer will only run software you trust. Freedom means being allowed to run the software you want to run, but it also means being able to choose the software you don't want to run.

You've sold out
We've been working on this for months. This isn't an attractive solution, but it is a workable one. We came to the conclusion that every other approach was unworkable. The cause of free software isn't furthered by making it difficult or impossible for unskilled users to run Linux, and while this approach does have its downsides it does also avoid us ending up where we were in the 90s. Users will retain the freedom to run modified software and we wouldn't have accepted any solution that made that impossible.

But is this a compromise? Of course. There's already inequalities between Fedora and users - trademarks prevent the distribution of the Fedora artwork with modified distributions, and much of the Fedora infrastructure is licensed such that some people have more power than others. This adds to that inequality. It's not the ideal outcome for anyone, and I'm genuinely sorry that we weren't able to come up with a solution that was better. This isn't as bad as I feared it would be, but nor is it as good as I hoped it would be.

What about ARM
Microsoft's certification requirements for ARM machines forbid vendors from offering the ability to disable secure boot or enrol user keys. While we could support secure boot in the same way as we plan to on x86, it would prevent users from running modified software unless they paid money for a signing key. We don't find that acceptable and so have no plans to support it.

Thankfully this shouldn't be anywhere near as much of a problem as it would be in the x86 world. Microsoft have far less influence over the ARM market, and the only machines affected by this will be the ones explicitly designed to support Windows. If you want to run Linux on ARM then there'll be no shortage of hardware available to you.

Is this all set in stone?
No. We've spent some time thinking about all of this and are happy that we can implement it in the Fedora 18 timescale, but there's always the possibility that we've missed something or that a new idea will come up. If we can increase user freedom without making awful compromises somewhere else then we'll do it.

[1] In fact, chances are that everything will carry the Microsoft key. Secure boot requires that UEFI drivers also be signed. The signing format only permits a single signature per binary. For compatibility, approximately all add-on hardware shipped will be signed with Microsoft's key, and that means that all system vendors have to recognise Microsoft's key in order to permit that hardware to run on their systems.
[2] Current source is here. It relies on a port of the UEFI crypto library and OpenSSL that I have building with some handholding, and which I'll upload as soon as possible.

comments

I recently got a question by email about the lecture I presented at FOSDEM, and by that mail I discovered that the video of the lecture was uploaded to YouTube (FOSDEM 2012 Debian packaging for beginners). While I thank the person who did the upload, he/she did not keep the right license for the video which is CC-BY-SA 3.0 (as noted at FOSDEM archive). You can also grab the webm file directly. Well, the next task will be to listen to myself in order to improve for FOSDEM 2013 See you there.
Filed under: Debian GNU/Linux, FOSDEM

After a relatively long period of flux, I've managed to settle into a routine where I've got all the time I can dedicate to open source stuff bunched together: every Tuesday night. I've started calling it "Debian Day", although Debian won't get all that time exclusively. One evening a week doesn't sound like much, but that's unfortunately a realistic amount given my other commitments. Still, having a firm idea of how much resource I have makes planning what I can and can't do much easier. It also means I need to be quite selective about what I spend my time on. This also impacts the type of interactions I can use. I can respond to email, bugs, etc., but I cannot effectively interact with folks on IRC, for example: the period between windows is too large for it to work effectively. Therefore, I can't work reliably with any team or area of Debian that relies on IRC to get things done. Tonight, then. I have recently been ripping some CDs, a task for which I use grip, despite it long since having been dropped by Debian. It still works very well, whereas Sound Juicer, which still pops up whenever I insert an audio CD, does not. I've been interested in helping out with Debian GNOME packaging for a while, and it seems the issue has been fixed upstream, but after asking around in #debian-gnome, the root cause is that the newer musicbrainz API packages have not been uploaded to Debian. (the existing package uses the old API). Two separate people appear to have been working on a new Music Brainz package, but neither filed an ITP bug. I've spent the largest chunk of my Debian time just trying to figure out what has been going on, sadly, but at least there's the likelyhood this will be fixed for wheezy. There's also the tail end of a musicbrainz-2.1 transition finishing up. Another thing I considered this evening was the future of debgtd in Debian stable. On reflection, and after sitting on the fence for several months, I've decided I don't have the time to fix it up enough to have it ready for wheezy, so with regret I've filed a removal bug. With any luck, at some point in the wheezy cycle I'll manage to put more time towards this, because I think it addresses a genuine need and could be useful to people. Finally, whenever I sit down and look at Debian stuff to finish up, I have a strong temptation to fix some issues belonging to the games team. In particular, Gustavo Panizzo has put a lot of work into vavoom packages, but sadly his requests for sponsorship are falling on deaf ears. There's also Johey Shmit's work on packaging zdoom which needs some attention to finish removing some embedded libraries. I have to keep reminding myself I left the team for a reason, to focus on something else, and finding that something else should be the priority. So all in all not the most uplifting evening: I'm starting to think I simply couldn't work effectively with the Debian GNOME team without being able to dedicate more time to use IRC effectively; I spent more time that could be necessary figuring out a bunch of other folks have potentially wasted each other's time by working on packages and not filing ITPs; and I've removed one of my own packages from the distribution altogether. It's not all doom and gloom, though: last week I packaged Simon Howard's lhasa, a library and tool for decompressing LHA files, as used by various old DOS games. (See? I just can't escape that stuff ) I also managed to 'resolve' two bup bugs, one RC. I also finished transitioning from my old PGP key to my new one.

Hi! I have been selected to work on multiarch cross-toolchains as part of the Google summer of Code. Although coding officially started last week, I have been busy with exams and other uni stuff at the time. Anyway, let's talk about the project itself, that is, multiarch cross-toolchains. What is multiarch? In order to understand what the project is about, one need to know what multiarch is. I'll quickly describe it, and I invite anyone interested to read the spec (really, it's interesting, and not that long). Basically, multiarch lets you install binary packages for multiple architectures on one system, and let package managers resolve dependencies as one would expect. This works by identifying four sets of packages:

• Packages that don't need to be aware of multi-arch. Such packages can only be installed for one architecture at a time, and can only be used to resolve dependencies of packages having the same architecture. Those packages are Multi-Arch: none or multiarch-unaware packages.
• Packages that can be installed for multiple architectures at a time, and can only be used to resolve dependencies of packages having the same architecture. Those packages are Multi-Arch: same packages. They typically are libs.
• Packages that can only be installed for one architecture at a time but can satisfy dependencies of any package regardless of its architecture. Those packages are Multi-Arch: foreign packages. They typically are tools working with architecture-independent interfaces. (for instance, gpg, zenity or tar).
• Packages that can only be installed for one architecture at a time but can satisfy any dependency annotated with :any , regardless of the architecture. Those packages are Multi-Arch: allowed packages. They are packages providing both arch-dependent and arch-independent interfaces. They are typically things like python, which can be used to interpret arch-indep files, but providing arch-dependent interfaces for C extensions.

Anyway, multiarch is an amazing thing that may be of use in various cases, such as running i386-only software on an amd64 system, running armel software on an armhf system, running i386 software on an armhf system using qemu-user-static and binfmt, cross-grading (switching, for instance, from i386 to amd64 on a running system), and, in our case, easing cross-compilation. State of cross-toolchains in Debian There aren't really cross-toolchains available in Debian (although some can be built from source packages), but they are available in emdebian's repositories. However, those packages predate multiarch, and don't make use of it. Instead, they depend on special packages converted using dpkg-cross. Such packages are merely arch: all versions of foreign packages, with files moved to arch-qualified paths to prevent conflicts with normal packages. Such packages thus have to be generated from real packages, either manually or using some sort of automation. In addition, this additional step brings several issues, like keeping in sync converted packages with real packages from the archive. With multiarch, such packages are rendered useless, and may in fact bring additional problems, since converted and multiarch packages may share the same files, and thus may not be co-installable. Switching from dpkg-cross to cross-arch deps Cross-toolchains packages currently depend on a few packages converted using dpkg-cross (ex: libc6-$ARCH-cross and libc6-dev-$ARCH-cross for some packages). They need to depend on the real package with the right arch (ex: libc6:$ARCH, libc6-dev:$ARCH). The obvious way to do so is by annotating the dependency with a specific arch qualifier (ex: Depends: libc6:armhf ). This is, however, not covered by the multiarch spec. Note that some lib*-$ARCH-cross packages will remain (for instance, libgcc1-armhf-cross). Indeed, those packages are different from the packages mentioned above in that they are of the architecture of the build system and actually part of the toolchain itself.
The above isn't actually true. Indeed, those packages are runtime library cross-built during gcc's build. Likewise, cross-toolchains may build-depend on foreign packages (ex: libc6-dev). Thus, specific arch qualifiers must be handled in the Build-Depends source package field too. Changing dpkg and apt to support that should be fairly straightforward. In fact, I already have a few patches, but I need to refresh them, maybe clean them up a little, and push them a little harder to get them accepted!

In 2003, a German teacher showed up on the Debian Edu and Skolelinux mailing list with interesting problems and reports proving he setting up Linux for a (for us at the time) lot of pupils. His name was Ralf Gesellensetter, and he has been an important tester and contributor since then, helping to make sure the Debian Edu Squeeze release became as good as it is.. Who are you, and how do you spend your days? I am a teacher from Germany, and my subjects are Geography, Mathematics, and Computer Science ("Informatik"). During the past 12 years (since 2000), I have been working for a comprehensive (and soon, also inclusive) school leading to all kind of general levels, such as O- or A-level ("Abitur"). For quite as long, I've been taking care of our computer network. Now, in my early 40s, I enjoy the privilege of spending a lot of my spare time together with my wife, our son (3 years) and our daughter (4 months). How did you get in contact with the Skolelinux/Debian Edu project? We had tried different Linux based school servers, when members of my local Linux User Group (LUG OWL) detected Skolelinux. I remember very well, being part of a party celebrating the Linux New Media Award ("Best Newcomer Distribution", also nominated: Ubuntu) that was given to Skolelinux at Linux World Exposition in Frankfurt, 2005 (IIRC). Few months later, I had the chance to join a developer meeting in Ulsrud (Oslo) and to hand out the award to Knut Yrvin and others. For more than 7 years, Skolelinux is part of our schools infrastructure, namely our main server (tjener), one LTSP (today without thin clients), and approximately 50 work stations. Most of these have the option to boot a locally installed Skolelinux image. As a consequence, I joined quite a few events dealing with free software or Linux, and met many Debian (Edu) developers. All of them seemed quite nice and competent to me, one more reason to stick to Skolelinux. What do you see as the advantages of Skolelinux/Debian Edu? Debian driven, you are given all the advantages of a community project including well maintained updates. Once, you are familiar with the network layout, you can easily roll out an entire educational computer infrastructure, from just one installation media. As only free software (FOSS) is used, that supports even elderly hardware, up-sizing your IT equipment is only limited by space (i.e. available labs). Especially if you run a LTSP thin client server, your administration costs tend towards zero. What do you see as the disadvantages of Skolelinux/Debian Edu? While Debian's stability has loads of advantages for servers, this might be different in some cases for clients: Schools with unlimited budget might buy new hardware with components that are not yet supported by Debian stable, or wish to use more recent versions of office packages or desktop environments. These schools have the option to run Debian testing or other distributions - if they have the capacity to do so. Another issue is that Debian release cycles include a wide range of changes; therefor a high percentage of human power seems to be absorbed by just keeping the features of Skolelinux within the new setting of the version to come. During this process, the cogs of Debian Edu are getting more and more professional, i.e. harder to understand for novices. Which free software do you use daily? LibreOffice, Wikipedia, Openstreetmap, Iceweasel (Mozilla Firefox), KMail, Gimp, Inkscape - and of course the Linux Kernel (not only on PC, Laptop, Mobile, but also our SAT receiver) Which strategy do you believe is the right one to use to get schools to use free software?

1. Support computer science as regular subject in schools to make people really "own" their hardware, to make them understand the difference between proprietary software products, and free software developing.
2. Make budget baskets corresponding: In Germany's public schools there are more or less fixed budgets for IT equipment (including licenses), so schools won't benefit from any savings here. This privilege is left to private schools which have consequently a large share among German Skolelinux schools.
3. Get free software in the seminars where would-be teachers are trained. In many cases, teachers' software customs are respected by decision makers rather than the expertise of any IT experts.
4. Don't limit ourself to free software run natively. Everybody uses free software or free licenses (for instance Wikipedia), and this general concept should get expanded to free educational content to be shared world wide (school books e.g.).
5. Make clear where ever you can that the market share of free (libre) office suites is much above 20 p.c. today, and that you pupils don't need to know the "ribbon menu" in order to get employed.
6. Talk about the difference between freeware and free software.
7. Spread free software, or even collections of portable free apps for USB pen drives. Endorse students to get a legal copy of Libreoffice rather than accepting them to use illegal serials. And keep sending documents in ODF formats.

I've received two separate emails now about a spammy looking tweet from Sarah's Twitter account, so I thought I'd attempt to head off any more. Yes, it appears her account got hacked (weak password), but surprisingly the password didn't get changed. Sarah has changed her password to a stronger one and deleted the offending tweet. All is right with the world.

I just come across a blog post from Glyn Moody reporting the claimed cost from Microsoft on requiring ODF to be used by the UK government. I just sent him an email to let him know that his assumption are most likely wrong. Sharing it here in case some of my blog readers have seem the same numbers float around in the UK.

Hi. I just noted your http://blogs.computerworlduk.com/open-enterprise/2012/04/does-microsoft-office-lock-in-cost-the-uk-government-500-million/index.htm comment:

"They're all in Danish, not unreasonably, but even with the help of Google Translate I can't find any figures about the savings of "moving to a flexible two standard" as claimed by the Microsoft email. But I assume it is backed up somewhere, so let's take it, and the 500 million figure for the UK, on trust."

I can tell you that the Danish reports are inflated. I believe it is the same reports that were used in the Norwegian debate around 2007, and Gisle Hannemyr (a well known IT commentator in Norway) had a look at the content. In short, the reason it is claimed that using ODF will be so costly, is based on the assumption that this mean every existing document need to be converted from one of the MS Office formats to ODF, transferred to the receiver, and converted back from ODF to one of the MS Office formats, and that the conversion will cost 10 minutes of work time for both the sender and the receiver. In reality the sender would have a tool capable of saving to ODF, and the receiver would have a tool capable of reading it, and the time spent would at most be a few seconds for saving and loading, not 20 minutes of wasted effort. Microsoft claimed all these costs were saved by allowing people to transfer the original files from MS Office instead of spending 10 minutes converting to ODF. :) See http://hannemyr.com/no/ms12_vl02.php and http://hannemyr.com/no/ms12.php for background information. Norwegian only, sorry. :)

Email spam is a plague, but sometimes there is at least on category of spammers it is easy to have fun with: naive manual spammers. People that try to do more or less legal business, using Internet in an illegitimate way, for instance (this is a real example, including the spammer's address: spamming bots, please use it ;-) !):

From: Emma Perroni <emmaperroni@gmail.com>
Subject: Hello
Hello,
Do you have a Facebook page? If you do, I can offer you to add fans by
hunders/thouthands in a single day using an original (and confidential)
technique. We can also add subscribers to Twitter, Youtube or Google+1.
Prices: 10  for 100 fans, -20% for more than 1000 fans ordered.
Regards

Reporting Well, in that case it may or may not be a scam, but even if it is not a criminal activity such as selling fake drugs, it is not a very honest business. Anyway, that lady is using her Google account to send her crap, so neutralizing her is quite easy:

1. checking that it really comes from Google Mail, using DKIM or SPF;
2. writing to Google, reporting that spam (and, in that specific case, the abuse they are offering to commit on Google +1);
3. optional, do not do that with scammers: replying to the spammer explaining that this is spamming, that you reported it and that she can expect to loose her Google account and everything on it.

That should work with any good email provider, that is, any provider that listens to complaints at <abuse@> and takes action on them. For what I have seen, Google and Yahoo! are, but fossils such as traditional national telephonists are not. If the message headers allow you to identify it, you can also report the abuse to the spammer's network operator if it is a good one (this is rarely the case). Consequences With a bit of luck, that can seriously damage the business of the spammer, and if he was really careless (e.g. storing important documents on the same Google account with no local backup), cause serious trouble in his personal stuff. If you replied to the spammer, you can get fun answers such as (these are taken from real examples I got):

• Ha ha ha, why not calling the FBI? That's it, laugh while you can, do you think your provider does not care? Too bad, he does, so you may change of mood when you see the effects loosing one's online account.
• We are doing legal business, you will stop your harrassment or we will file a complain. Yes, please do so if you dare, but may I remember you that you are the one at fault here? (in fact, such a reaction is a very good sign, since it shows that the spammer is actually afraid, probably for his business)
• Why did you do that? I promise, I shall not do it again, please Too late, lamer, you should have thought of that before sending your crap

Conclusion This is too complicated to be done for each spam of course, but it is worth doing for spam messages that are not catch by one's antispam. And the rules for email marketing are well known: only write to people that explicitly allowed you to do so, full stop, so there is no reason to show any mercy for a spammer.

jffnms version 0.9.3 has been released today. This is a vast improvement over the 0.9.x releases and anyone using that train is strongly recommended to upgrade.So what changed? What didn t change! A nice summary would be fixing a lot of things that were broken or needed some tweaking. A really, really big thanks to Marek for all the testing and bug reports and also patient just run this and tell me what it says tests he did too. If something wasn t right before and works now, it is quite likely it is working because Marek told me how it broke. A brief overview of what has changed:

• TFTP transfers work again
• A lot of the wierd polling effects due to caching fixed
• Lots of the selects in sub-tables now work
• The PHP string-to-float brokeness in SLAs worked-around
• Even more SNMP library cruft removed or escaped
• HostMIB apps match properly
• Interface autodiscovery delete and update fields back working

You can download the file off sourceforge at https://sourceforge.net/projects/jffnms/files/JFFNMS%20Releases/

Related articles

• PHP floats and locales (enc.com.au)

The best thing about rockets is when they go up successfully, and don't come back down. My git-annex Kickstarter reached escape velocity 24 hours after launch, and is now past 200% funded. This is great news, because it gives me more time to spend hacking on git-annex, and will let me add more features and polish it better. I had set the goal at a minimal amount because I'd have hated not to get funded at all if this hadn't taken off. But the current funding is much more comfortable, and the further up it goes, the more scope the project can have. At the point shown above, the project had started to be highlighted as popular, based largely on kind and generous readers of this blog and git-annex users who chipped in. For a while, I personally knew around 1/3rd of contributors. That was enough to get it noticed by Kickstarter staff, which in turn has led to more growth. I love the juxtaposition of geeky tech project with other stuff here. I have 17 days remaining until the Kickstarter is finished. I really had worried it might take that long to get funded. As it is, I can't wait to see where the rocket's trajectory takes it in that time!

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:

In Total:	1534
Affecting Wheezy:	1095
Wheezy only:	238
Remaining to be fixed in Wheezy:	857

Of these 857 bugs, the following tags are set:

Pending in Wheezy:	82
Patched in Wheezy:	175
Duplicates in Wheezy:	62
Can be fixed in a security Update:	27
Contrib or non-free in Wheezy:	11
Claimed in Wheezy:	2
Delayed in Wheezy:	18
Otherwise fixed in Wheezy:	86

Ignoring all the above (multiple tags possible) 500 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 975 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

I was worried about having to learn hundreds of configuration options to properly set up mailman, postfix and lighttpd on Debian Squeeze. Turned out, that except for lighttpd it all works out of the box.

apt-get install postfix

When asked by debconf, I specified lists.mister-muffin.de as the fully qualified domain name.

apt-get install mailman
newlist mailman

The newlist command reminds me that I have to add its output to /etc/aliases. After doing so, I have to run:

newaliases

From now on, I can add any mailinglist by running newlist, editing /etc/aliases and running newaliases. Mailinglists can also be added through the mailman webinterface but one still has to put the according entries into /etc/aliases. Following is a working lighttpd configuration that works out of the box with the default settings of mailman on Debian squeeze. This was the only part that caused me some headaches.

server.modules += ("mod_alias", "mod_cgi", "mod_accesslog")
$HTTP["host"] == "lists.mister-muffin.de"   accesslog.filename =
    accesslog.filename = "/var/log/lighttpd/lists-access-log"
    alias.url += (
        "/cgi-bin/mailman/private/" => "/var/lib/mailman/archives/private/",
        "/cgi-bin/mailman/public/" => "/var/lib/mailman/archives/public/",
        "/pipermail/" => "/var/lib/mailman/archives/public/",
        "/cgi-bin/mailman/"=> "/var/lib/mailman/cgi-bin/",
        "/images/mailman/" => "/usr/share/images/mailman/",
    )
    cgi.assign = (
        "/admin" => "",
        "/admindb" => "",
        "/confirm" => "",
        "/create" => "",
        "/edithtml" => "",
        "/listinfo" => "",
        "/options" => "",
        "/private" => "",
        "/rmlist" => "",
        "/roster" => "",
        "/subscribe" => "")
 
server.document-root        = "/var/www"
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/var/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
index-file.names            = ( "index.html" )
server.dir-listing          = "disable"
include_shell "/usr/share/lighttpd/create-mime.assign.pl"

As a bonus, I wanted to import my existing email exchange with my GSoC mentors into the mailinglist. First I was planning on manually sending the email messages to the list, but a much easier option is to just import them in mbox format. To extract all email messages, I first wrote the following python snippet: It iterates through messages in my mbox and maildir mailboxes, filters them for emails by wookey or pietro, strips away some messages I found to not be relevant and then saves the filtered result into the mbox mailbox ~/out. It is important to specify factory=None for the Maildir parser, because it otherwise defaults to rfc822.Message instead of MaildirMessage. Also do not forget to call box.close(). I initially forgot to do so and ended up with missing messages in ~/out. I then copy the archive in its place:

scp out lists.mister-muffin.de:/var/lib/mailman/archives/private/debian-bootstrap.mbox/debian-bootstrap.mbox

Another thing that initially caused me trouble, was that the mbox didnt have the correct permissions due to the scp. Fixing them:

chown -R list:www-data /var/lib/mailman/archives/private/
chmod 664 /var/lib/mailman/archives/private/debian-bootstrap.mbox/debian-bootstrap.mbox

And update the mailman archive like this:

sudo -u list /usr/lib/mailman/bin/arch debian-bootstrap /var/lib/mailman/archives/private/debian-bootstrap.mbox/debian-bootstrap.mbox

Initially I was running the above command as root which screws up permissions as well.

Fedora 17 will be shipping next week. It's got a bunch of new features, none of which I contributed to in the slightest. What I did work on was improving our support for installation on x86 Apple hardware. There's still a few shortcomings in this so it's not an announced or supported feature, but it's sufficient progress that it's worth writing about.

There's a few ways that Apple platforms differs from normal PC hardware. The first is that Apples are very much intended to be EFI first and BIOS second, while that's still not really true for commodity PC hardware. Apple launched Boot Camp shortly after they started shipping x86 Macs (and shortly after I'd got Ubuntu running on one for the first time[1]) but the focus of Boot Camp has always been to be good enough to boot Windows and not much else[2]. The other is the integration with the boot environment. You can boot Linux easily enough by setting the standard EFI boot variables, but if you ever boot back into OS X it's easy to trigger deletion of those. There's then no straightforward way of resetting them, and you're left having to recover your installation.

The other difficult bit has been actually starting the installer at all. If you're happy to use BIOS emulation than this can be done without too much misery, but you're then left with dealing with Apple's custom synchronised GPT/MBR. More modern Macs will boot via the standard EFI mechanisms, but there's a range of problems that can be caused that way. Fedora 17 is the first Linux distribution to ship install media that will EFI boot a Mac when either written to optical media or a USB stick[3]. Put the install media in your system and then either select it in the OS X Startup Disk preferences menu and reboot, or reboot and hold down the alt key. In the latter case there'll be a nice Fedora logo. Click on it and it'll boot.

The biggest difference between Apple hardware and anyone else is that we'd normally put the bootloader in a FAT partition that's shared with any other installed operating systems. That does actually work on Apples, but then you run into the earlier point I mentioned. The Apple startup picker that you get by holding down the alt key doesn't pay any attention to the standard EFI boot variables, so it won't show a FAT partition merely because it's got an EFI bootloader on it. The same is true of the OS X Startup Disk preferences. The best way to get a drive to appear in the menu is to make it look like an OS X drive.

This was problematic in a few ways, due to the requirement for an OS X drive to be HFS+. The first was that we didn't have any support for that in our installer, so work had to be done there. The second was that the state of HFS+ was woefully poor in Linux[4]. Linux will refuse to write to an HFS+ filesystem if it hasn't been cleanly unmounted or fscked, and since we can't rely on everyone always shutting their machine down cleanly that means a working fsck.hfsplus. In theory we were shipping one of those. In practice, it reliably segfaulted on 64-bit systems[5]. So I had to package the latest version of Apple's code, and doing that involved dealing with the fact that Apple now use blocks almost as much as Grub 2 uses nested functions, and that meant using Clang and that meant dealing with a bug where Clang segfaulted if it had been built with gcc 4.7[6], but finally once all of those yaks had been shaven we could trust our filesystem.

Of course, more problems existed. You can't just drop a bootloader onto an HFS+ partition and expect it to work. You need to write its inode value into the superblock. Trivial, except that if you do this from userspace then the kernel kindly overwrites your update when you unmount the disk and it updates the superblock. That required a mechanism for updating that data via the kernel, which meant adding an ioctl. This would have been fine, except the OS X Startup Disk preference looks for a bootloader in a location other than where Fedora installs it. Changing our bootloader install path wasn't a great option, so the easiest thing to do was just to link them together. Except HFS+ doesn't really support hardlinks. When you hardlink something the original file gets moved into a magic directory in the filesystem root and the links are special files that refer to it. And it turns out that it's not actually the inode that the firmware wants when finding the bootloader, it's the catalogue ID, and these aren't the same when hardlinks are involved. So, another kernel patch.

All that was left for the boot partition then was adding an icon and a drive label. We already had code for generating the icon, and the drive label wasn't that difficult. Hurrah!

Ok, so we can construct a filesystem that (a) appears in the bootpicker, and (b) appears in the OS X Startup Disk preferences. We can even put a bootloader on it. This led to the next problem. The bootloader started without any trouble. But the bootloader couldn't read any files off the boot partition, including its configuration. This should have been obvious with hindsight - the problem was simply that grub legacy[7] doesn't support HFS+. Writing an HFS+ driver seemed like an excessive amount of work, especially since the firmware already supported reading HFS+, so instead I wrote a simple grub filesystem driver that uses the UEFI interfaces to read files.

A working bootloader! One more problem there. grub looks in its startup directory to find its configuration file. We have two "copies" of grub hardlinked to each other, each looking in a different directory for configuration. Hardlinking the config files together worked fine, except that most tools that update config files have this irritating habit of writing to a temporary file and moving that over the original, thus breaking the hardlink. Easily solved by using a symlink instead, except that the UEFI filesystem semantics don't really support symlinks because UEFI only really envisaged using FAT. If you use Apple's UEFI HFS+ driver to open a symlink, you get a short file containing the target path of the symlink. Rather less than ideal, and a rather gross hack to cope with it.

All set now, other than a bug in some older Mac firmware where read would return BUFFER_TOO_SMALL without returning the real buffersize and the odd way that Apple treat CDs. Oh, and a bug in Apple's Broadcom wifi driver that could overwrite the kernel, and case-sensitivity, something that obviously isn't something you often hit on UEFI when all you usually see is FAT.

Kernel-side, we had a couple of things. Obviously there the bugs I'd mentioned in the past, but those had already been dealt with. New issues included the fact that we were frequently picking the wrong framebuffer address, especially on machines with multiple GPUs. Fixed now, along with another issue where we'd sometimes get confused by models with different GPU configurations. And at that point, most of the implementation work was done.

Of course, there was also the work of integrating this into the tools that we use to generate our install media, and I'd like to thank Will Woods, Brian Lane and Dennis Gilmore for the work they put into that, along with anyone I've forgotten. Tom Calloway handled getting the device label graphics generated right before deadline. A cast of thousands helped with testing.

So why did I mention shortcomings? First, this only works on machines with 64-bit firmware. Early 64-bit Apples still had 32-bit firmware. In theory we can support that case - in practice it's a moderate amount of writing, including some mildly awkward kernel code. But anything later than mid-2007 should have 64-bit firmware, so it's not a massive concern. Second, we seem to have fairly significant trouble with Radeon hardware on a lot of Macs. We fixed one bug there, but something's still going wrong in setup and you'll frequently end up with a black screen. And thirdly, I'm sure there's some other machines that still don't work for (as yet) undetermined reasons.

With luck we'll get these things dealt with by Fedora 18. But even with these flaws, Fedora 17 will work fine on a lot of Apple hardware, and testing it is as simple as downloading and booting a live image. Bugs go in the usual place.

[1] Note my charming belief that the Ubuntu installer would fully support this hardware via EFI in the near future. 6 years ago. It still doesn't.
[2] This can be fairly easily seen from little things like the name "Windows" being hardcoded into every UI that sees a Boot Camp drive.
[3] I cover this in more depth here
[4] To be fair, it mostly still is - there's plenty of work to be done there.
[5] Approximately nobody had noticed this, which is all kinds of unreassuring.
[6] Thanks to Kalev Lember for dealing with that
[7] Still the EFI bootloader in Fedora 17. We've already swapped Fedora 18 over to grub 2.

comments

Something a little low level for this post. I have been asked recently how to "test" for the maximum duration of an Interrupt Service Routine (ISR) in Linux

To do this I probably ought to explain what the heck an ISR is!

A CPU executes one instruction after another and runs your programs. However early in the history of the electronic computer it soon became apparent that sometimes there were events happening, generally caused by a hardware peripheral, that required some other code to be executed without having to wait for the running program to check for the event.

This could have been solved by having a second processor to look after those exceptional events but that would have been expensive, difficult to synchronize and the designers took the view that there was a perfectly good processor already sat there just running some users program. This Interruption in the code flow became known as, well, an Interrupt (and the other approach as polling).

The hardware for supporting interrupts started out very simply, the processor would complete execution of the current instruction and when the Program Counter (PC) was about to be incremented if an Interrupt ReQest (IRQ) was pending the PC would be stored somewhere (often a "special" IRQ stack or register) and then execution started at some fixed address.

The interrupting event would be dealt with by some code and execution returned to the original program without it ever knowing the CPU just wandered off to do something else. The code that deals with the interrupt is known as the Interrupt Service Routine (ISR).

Now I have glossed over a lot of issues here (sufficient to say there are a huge number of details in which to hide the devil) but the model is good enough for my purpose. A modern CPU has a extraordinarily complex system of IRQ controllers to deal with numerous peripherals requesting the CPU stop what its doing and look after something else.

This system of controllers will ultimately cause program execution to be delivered to an ISR for that device. If we were living in the old single thread of execution world we could measure how long execution remains within an ISR, perhaps by using a physical I/O line as a semaphore and an external oscilloscope to monitor the line.

You may well ask "Why measure this?" well historically while the ISR was running nothing else could interrupt it executing which meant even if there was an event that was more important it would not get the CPU until the first ISR was complete. This was known as IRQ latency which was undesirable if you were doing something that required an IRQ to be serviced in a timely manner (like playing audio)

This is no longer how things are done while the top half runs with IRQ disabled many are threaded interrupt handlers and are preemptable (I.e. can be interrupted themselves) which leads to the first issue with measuring ISR time in that the ISR may be executed in multiple chunks if something more important interrupts. Indeed it may appear an ISR has taken many times longer one time than another because the CPU has been off servicing multiple other IRQ.

Then we have the issue that Linux kernel drivers often do as little as possible within their ISR, often only as much as is required to clear the physical interrupt line. Processing is then continued in a "bottom half" handler this leads to ISR which take practically no time to execute but processing is still being caused elsewhere in the system.

The next issue is the world is not uniprocessor any more, how many processors does a machine have these days? even a small ARM SoC can often have two or even four cores. This makes our timing harder because it is now possible to be servicing multiple interrupts from a single peripheral on separate cores at the same time!

In summary measuring ISR execution time is not terribly enlightening and almost certainly not what you are interested in. The actual question is much more likely that you really want to be examining something that the ISR time was an historical proxy for like IRQ latency or system overheads in locking.

BROWN Anne After a long illness bravely fought, Anne Margaret (n e Wilson), wife of Fred and mother to Mark, enthusiastic Scottish country dancer. Service to be held at Borders Crematorium, on Friday, May 25, 2012, at 11 am. Donations, if desired, to Marie Curie Cancer Support. Family flowers only.

This weekend procps-ng version 3.3.3 was tagged and released for distribution. There have been many patches and fixes involved in this release as we move from an unchanging static sort of code into something that is easier to maintain and build on various architectures. The good thing is I m down to 1 or 2 patches in the debian archive which is a big change from the 30 or 40-odd I used to carry. For the sole metric of getting that number down, the project fork has been a success. There were some post-release bugs I found and these were more to do with the various options turned on or off rather than what you d see if you did the basic ./configure && make. One of them was how the version numbers are defined in git, but would only appear if certain files were older than others (such as aclocal.m4 versus config.h.in) Others were when certain features were turned on. The make check doesn t see all of this because it uses the default configure flags. One annoying thing of the autotools is conditionally installing man pages. This is where you don t or cannot compile a binary so you don t install the corresponding man page. The automake documentation is of course obscure about this but I cannot see a way of distributing only a man page, so we have this fiddle where a file goes into distmanMANS or EXTRA_DIST depending. Interestingly, there has been some bike-shedding around Fedora-land (see the link below) regarding the name procps-ng versus procps. Debian is lucky that we do have different upstream and package names (though not ideal) so apt-get install procps still gives you procps. There also has been discussion about merging procps-ng into util-linux whichin the short-term won t be happening.

Related articles

• Fedora 18 Approves Controversial Feature (phoronix.com)
• Change top Command Default Delay (technologiquepark.wordpress.com)

Third post about ganeti. Ganet-debootstrap-instance contains a nice set of scripts to create a debian (or derivatives) image using debootstrap. Images can be configured and customized by writing simple hooks script to modify various aspects of the default installation. However writing these script is not really fun and pushing it too far can lead to long messy scripts, loosing the overall benefit of automatic configuration. Puppet is my configuration management tool of choice, but installing puppet on a new machine requires few magic incantations that the user should perform manually, or in a semi automatic mode (autosign=true) to make it work. My goal is to install puppet automatically on the newly created instance so it will run and configure the new instance at the first boot. From that moment on I'll forget about ganeti and configure all remaining services of my new VM using puppet. In order to do so, we need to install puppet (and apt-get update/upgrade...), create the ssl certificates for the client and enabling the puppet daemon on the client. We add another hook in /etc/ganeti/instance-debootstrap/hooks/ : This script uses puppetca to create on the puppet (and ganeti) server the client key, sign it, and then copy it to the target machine. Notice that we create the certificate for a fqnd name $INSTANCE_NAME.$DOMAIN or otherwise puppet will complain loudly. This is not strictly needed, but if you want to do otherwise, you'll need to fiddle with the puppet configuration a bit more. The procedure to create a puppet certificate server-side is well documented on the puppet website, so if you are curious about the details duck-duck-it .

John Scalzi recently wrote a piece on straight white male privilege. If you haven't read it already, go and do so. No rush. I'll wait.

So. Some facts:

• Women are underrepresented in free software development
  
• Those women who are involved in free software development are overwhelmingly more likely to have been subject to sexual harassment, belittling commentary or just plain ignored because of their gender
  
• When asked, women tend to believe that these two facts are fairly strongly related

(If you disagree with any of these then that's absolutely your right. You're wrong, but that's ok. But please do me a favour and stop reading here. Otherwise you'll just get angry and then you'll write something ill-tempered and still wrong in the comments and then I'll have to delete it and why not just save everybody the time and effort and go and eat ice cream or something instead)

I know I've said this before, but inappropriate and marginalising behaviour is rife in our community, and at all levels of our community. There's the time an open source evangelist just flat out told a woman that her experiences didn't match his so she must be an outlier. There's the time a leading kernel developer said that most rape statistics were basically made up. There's the time that I said the most useful thing Debian could do with its money would be to buy prostitutes for its developers, simultaneously sexualising the discussion, implying that Debian developers were all straight men and casting sex workers as property. These aren't the exceptions. It's endemic. Almost all of us have been part of the problem, and in doing so we've contributed to an environment that has at best driven away capable contributors. You probably don't want to know what it's done at worst.

But what people have done in the past isn't important. What's important is how we behave in the future. If you're not angry about social injustice like this then you're doing it wrong. If you're reading this then there's a pretty high probability that you're a white male. So, it's great that you're angry. You should be! As a straight white male born into a fairly well-off family, a native English speaker in an English speaking country, I have plenty of time to be angry before going back to my nice apartment and living my almost entirely discrimination-free life. So if it makes me angry, I have absolutely no way of comprehending how angry it must make the people who actually have to live with this shit on a daily basis.


The point isn't to be smugly self aware of our own shortcomings and the shortcomings of others. The point is to actually do something about it. If you're not already devoting some amount of your resources to improving fairness in the world, then why not? It doesn't have to be about women in technology - if you're already donating to charity or helping out at schools or engaging in local politics or any of the countless other ways an individual can help make the world a better place, large or small, then keep on doing that. But do consider that many of us have done things in the past that contributed to the alienation of an astounding number of potential community members, and if you can then please do do something to make up for it. It might be donating to groups like The Ada Initiative. It might be mentoring students for projects like the GNOME Outreach Program for Women, or working to create similar programs. Even just making our communities less toxic by pointing out unacceptable behaviour when you see it makes a huge difference.

But most importantly, be aware that it was people like me who were responsible for this problem in the first place and people like me who need to take responsibility for solving it. We can't demand the victims do that for us.

comments

PHPUnit, according to its PTS, has been in Debian since 2009. But it was orphaned, and nobody took care of it for a while. That is until few weeks ago, Luis Uribe started to work on it. Since he isn t a DD, and that I take care of, I believe, about half of the all the PHP PEAR packages in Debian, I started working with him on re-doing the work of packaging PHPUnit for Debian. Previously, the old version was quite wrong, with missing dependencies, and not really working, what a shame PHPUnit 3.6 has been in Debian SID for 3 days now. And it s working well. I m now adding runs of unit testings from upstream packages at build time (of course, only if DEB_BUILD_OPTIONS doesn t contains nocheck , as per the policy). All together, it s been quite some fun to hack this, and I m quite happy of the results, even though there s still a lot of work remaining. PHP itself is running unit tests at build time. And not just a few: more than 11000 of them. The only small issue, is that they are totally outdated. Over the time, the vardump() function has evolved, and doesn t print things the same way. One may say that it prints things in a nicer way, but as a result, many of the tests that were suppose to work, actually fails because of these differences in vardump() over time. So I started working on fixing some of the tests. It s most of the time very easy to fix, but it takes a long time to fix all these small unit tests. So far, I ve been able to fix most of what s in the tests/, Zend/tests (more than 161 tests fixed), and the beginning of ext/*/tests: for the moment: bcmath, bz2, calendar, curl, date, dba, dom, ereg, and a part of exif, which for the moment represent 214 fixed tests, and I ll try to do more fixes when I have time. I hope to send the patches upstream soon. The final goal is of course to have the build of PHP to fail if unit tests are failing as well. For the moment, unit tests do run at build time, but the build don t care of the results, which I think is stupid (it s wasting CPU cycles for no reasons, IMO). If you maintain some PEAR packages, I would welcome you to first join the PKG-PEAR team on Alioth, and team maintain your packages, switch over to pkg-php-tools and dh 8 auto-sequencer, and to follow the PKG-PHP-PEAR packaging guide lines so that we have consistency in the archive. And of course, run unit testings, by doing a build-depends on phpunit (>= 3.6). Note that unit testings in PEAR packages are tracked on the Debian wiki. This post is also a call for help, because I feel quite alone doing this work of packaging PEAR packages, which many PHP applications depend on (think about roundcube, horde, extplorer, and many more). Other teams are doing very well, like the Perl team, and there s no reason why Debian wouldn t maintain PHP libraries as well as the ones for Perl.