Recently two companies have suffed problems due to compromised AWS credentials:

• Code Spaces
  • The company has effectively folded. Thier AWS account was compromised, and all their data and backups were deleted.
• Bonsai
  • Within two minutes all their instances were terminated.
  • This is still live - watch updates of the recovery process.

I'm just about to commit to using Amazon for hosting DNS for paying customers, so this is the kind of thing that makes me paranoid. I'll be storing DNS-data in Git, and if the zones were nuked on the Amazon-side I could re-upload them, but users would be dead regardless - because they'd need to update the nameservers in whois before the re-uploaded data would be useful. I suspect I need to upload to two DNS providers, to get more redundency. Currently I have a working system which allows me to push DNS records to a Git repository, and that seamlessly triggers a DNS update (i.e. A webhook trigged by github/bitbucket/whatever). Before I publish anything I need to write more code, more documentation, and agree on pricing details. Then I'll setup a landing-page at http://dns-api.com/. I've been challenged to find paying customers before launching, and thus far have two, which is positive. The DHCP.io site has now been freed. I'm no longer going to try to commercialize it, instead I will only offer the Git-based product as a commercial service. On that basis I upped the service so users could manage up to five names per account, more if you mail me privately and beg ;) (ObRandom: Google does hosted DNS with an API. They're expensive. I'm surprised I'd not heard of them doing this.)

When I gave my Haskell Bytes talk on the runtime representation of Haskell values the first time, I wrote here It is in German, so [..] if you want me to translate it, then (convince your professor or employer to) invite me to hold the talk again . This has just happened: I got to hold the talk as a Tech Talk at Galois in Portland, so now you can fetch the text also in English. Thanks to Jason for inviting me!
This was on my way to the Oregon Summer School on Programming Languages in Eugene, where I m right now enjoying the shade of a tree next to the campus. We ve got a relatively packed program with lectures on dependent types, categorical logic and other stuff, and more student talks in the evening (which unfortunately always collide with the open board game evenings at the local board game store). So at least we started to have a round of diplomacy, where I am about to be crushed from four sides at once. (And no, I don t think that this has triggered the illegal download warning that the University of Oregon received about our internet use and threatens our internet connectivity.)

When you type docker.io run -it debian sh, it goes off and gets "debian" and runs it. But what is in this "debian" image? How was it built? The docker hub does not really say. All it tells us is this is a "(Semi) Official Debian base image" and that its sources.list uses http.debian.net for geolocation. There's a link to https://github.com/dotcloud/stackbrew/blob/master/library/debian which in turn uses a very strange git repository, owned by Debian maintainer Tianon Gravi, that contains compressed tarballs of Debian: http://github.com/tianon/docker-brew-debian "Git is not a fan of what we're doing here." The "source", such as it is, that is used to build this image consists of:

FROM scratch
ADD rootfs.tar.xz /
CMD ["/bin/bash"]

and

mkimage.sh -t tianon/debian:wheezy -d . debootstrap --variant=minbase --components=main --include=inetutils-ping,iproute wheezy http://http.debian.net/debian

I don't know where mkimage.sh is. [Update: Probably /usr/share/docker.io/contrib/mkimage-debootstrap.sh or a modified version] And anyway, I have no reason to trust that this image is built the way it claims to be built. So, the question remains: What is in this image? To find out, I did a debootstap --variant=minbase stable and diffed the entire docker debian image against it. The diff was 6738 lines, from which I found the following interesting differences. added packages The image has iputils-ping and netbase and iproute added. These are not in a minbase debootstrap, but are in a regular debootstrap. It's rather weird that the docker image is based on a minbase debootstrap, since this means they have to add back important stuff like this on an ad-hoc basis.

If the expectation is that an experienced Unix person who found it missing would say "What on earth is going on, where is 'foo'?", it must be an 'important' package. -- Debian Policy

apt hooks

DPkg::Post-Invoke   "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin   true";  ;
APT::Update::Post-Invoke   "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin   true";  ;
Dir::Cache::pkgcache "";
Dir::Cache::srcpkgcache "";
Acquire::Languages "none";

These are some strange modifications to apt's config. The intent is clearly to avoid wasting disk space, even at the expense of making apt slower (by disabling caches) and losing translations. I am curious if apt might ever invoke the DPkg::Post-Invoke twice in an upgrade in which it runs dpkg twice. I'm also curious whether deleting /var/cache/apt/archives/lock could cause a problem. unsafe-io dpkg is configured to use unsafe-io. motd

Linux viper 3.12.20-gentoo #1 SMP Sun May 18 12:36:24 MDT 2014 x86_64

Yes, that's "gentoo". Presumably this tells us something about the build host. policy-rc.d /usr/sbin/policy-rc.d contains "exit 101", which prevents daemons from being automatically started after they are installed. This may or may not be desirable, depending on what you're doing with docker. It notably also prevents restarting running daemons in this container if they're upgraded for eg, a security fix. It would almost certianly be better if this script allowed restarting running daemons. diversions /sbin/initctl is diverted and replaced with /bin/true. This is a workaround for a bug in sysvinit; when upgraded inside a docker container it hangs while trying to run initctl. missing devices Some versions of the debian image are missing things in /dev. See this bug. (I had listed some device files that I thought were missing, but I was wrong.) some gpg thing is different

Binary files pure-debootstrap/etc/apt/trustdb.gpg and from-docker/etc/apt/trustdb.gpg differ

Oh well, that can't be important.. Or can it? I did not check. conclusions I would hardly consider this to be an "(Semi) Official Debian image". Some of the changes are quite dubious. The build environment is not Debian. There is no guarantee you'll get the same image I examined. Diffing thousands of lines of filesystem changes is not particularly fun or reliable way to spot accidental or malicious changes. I'd recommend only trusting docker images you build yourself. I have some docker images published somewhere that are built with 100% straight debootstrap with no modifications (and even an armel image that can be used on an x86 system thanks to qemu). But I'm not going to link to them, because again, you should only trust docker images you built yourself. To help increase your mistrust of me, I present this IRC snippet:

<joeyh> I'll bet I could publish an image that just did a killall5 as root on startup and get plenty of people to nuke their container hosts

Here are some ideas for things Debian could do to improve this:

• Make a package that can build docker images of Debian, in a fully reproducible fashion. Ie, same versions of debs in, same byte stream out.
• If it makes sense for the docker image to not contain all the packages in a standard debootstrap (maybe leaving out init systems), or to contain other packages, write down the rationalle for this, and make a --variant=docker.
• Make a package that provides appropriate tweaks for Debian in a container. This might include a policy-rc.d that allows restarting daemons on upgrade if they're already running in the container, and otherwise prevents running daemons.
• Make a low-disk-space package that eg, prevents apt from caching debs.
• Provide some way to verify, through gpg signatures, that docker has pulled an actual trusted image and not some https-MITMed thing. (See also #746394

PS, if this wasn't enough fun, just consider the tweaks made to the "Debian" images on all the VPS hosts out there.

The press picked up the recent press release about Debian LTS but mainly to mention the fact that it s up and running. The call for help is almost never mentioned. It s a pity because while it s up, it s not really running satisfactorily yet. As of today (2014-06-19), 36 packages in squeeze need a security update, yet squeeze-lts has only seen 7 updates. As usual what we lack is contributors doing the required work, but in this specific case, there s a simple solution: pay people to do the required work. This extended support is mainly for the benefit of corporate users and if they see value in Debian LTS, it should not be too difficult to convince companies to support the project. With some other Debian developers, we have gone out of our way to make it super easy for companies to support the Debian LTS project. We have created a service offer for Debian-using companies. Freexian (my company) collects money from all contributing companies (by way of invoices) and then uses the money collected to pay Debian contributors who will prepare security updates. On top of this we added some concrete benefits for contributing companies such as the possibility to indicate which packages should have priority, or even the possibility to provide functional tests to ensure that a security update doesn t introduce a regression in their production setup. To make a good job of maintaining Debian Squeeze, our goal is to fund the equivalent of a full-time position. We re currently far from there with only 13 hours per month funded by 4 companies. That makes a current average of 3.25 hours/month funded by each contributing company, for a price of 276 EUR/month or 3315 EUR/year. This is not much if you compare it with the price those companies would have to pay to upgrade all their Debian 6 machines now instead of keeping them for two supplementary years. Assuming the average contribution level will stay the same, we only need the support of 50 other companies in the world. That s really not much compared to the thousands of companies using Debian. Can you convince your own company? Grab the subscription form and have a chat with your company management. Help us reach that goal, share this article and the link to Freexian s Debian LTS offer. Long Term Support is important if we want Debian to be a good choice for servers and big deployments. We need to make Squeeze LTS a success! Thank you!

5 comments Liked this article? Click here. My blog is Flattr-enabled.

Tree Frog Dashboard Mat I ve been given some more MobileZap products to review. The first is the Tree Frog anti-slip dashboard mat [1]. This is designed to allow a phone to stick to an angled surface of the inside of the car without slipping. The pictures on the web site show a phone stuck to a curved angular surface of a car dash. To test this I stuck my phone to my fridge, the flat metal surface of the fridge and the almost flat plastic surface of the gel case of my phone are pretty much ideal surfaces for the sticky mat to adhere to. But even so keeping my phone stuck to a vertical surface is impressive. This mat is cheap enough that it could be used for many other tasks than securing a phone in a car, for example it could be used to stick a phone to the wall of your office. It s apparently washable and can be expected to keep gripping things if washed regularly. One thing that could be improved in the mat would be to have some holes in it. When I detached my phone from the fridge some large air bubbles developed between the mat and the fridge, this would be a problem if I wanted to regularly stick my phone to a flat surface such as an office wall. When in my car it didn t work nearly as well. The pattern on the surface of the car dash which is designed to avoid glare when driving is difficult to stick to, it s a bit like trying to attach sticky-tape to unpainted wood for similar reasons. A phone will still remain in place on a 30 degree angle from the horizontal (which is significant) but the mat grips much more tightly to the phone than the dash, which is a little inconvenient as the mat stays attached to the phone when you remove it from the dash. Sorry for the boring picture, it s black and flat so there s not much scope for making it interesting. 3.1A USB Car Charger Above is a picture of a 3.1A USB car charger and a AA battery charger. The car charger isn t particularly exciting, but providing 3.1A through two USB sockets is noteworthy it can charge one device at 2.1A (the maximum any device draws) or two devices for a total of up to 3.1A [2] (with protection against over-current). Previously in my car I used the 2.1A charger that shipped with my Galaxy Note 2 and an inverter to provide 240VAC as the devices I had to charge a phone from a car socket didn t provide enough current. This new device uses less space and also allows charging two phones at once. One problem with this device was finding USB cables that could handle the current, for benchmarking I used a phone that was running Ingress and acting as a Wifi access point. I discovered that the cables that Samsung ships with their recent devices (Galaxy Note 2 and Galaxy Note 10.1 tablet) seem to work best and support charging my Galaxy Note 2 or my wife s Nexus 5 while playing Ingress and running as a Wifi AP. Most of the other cables that I have in my collection won t even allow a phone to remain at the same charge level while playing Ingress. AA Emergency Phone Charger The AA phone charger is best suited for emergency use only, it takes one AA battery and has limited current capacity [3]. A NiMh rechargeable battery has a nominal Voltage of 1.2V, to provide 5V to charge a phone the Voltage has to be boosted by a factor of 4.16 which would reduce the current by a factor of 4.16 if the charger was 100% efficient. As 100% efficiency is impossible the current would be reduced even more. It seems unlikely that a AA battery would be able to sustain a current of 2A so the supply from that charger would be less than 500mA IE less than the least capable PC USB port or mains powered phone charger. A test with a Nexus 5 showed the phone charge level decreasing while being very lightly used (mostly just checking whether the charge level was increasing) when a NiMH battery was connected. A test with an alkaline battery and the same Nexus 5 showed the phone charge level increasing when lightly used but it probably wouldn t do so while playing Ingress. It seems that a single NiMH AA battery can only be used for charging a phone in a real emergency situation (IE an act of god type disaster not a need to level up in Ingress). If mains power was out then you could charge a phone while it s turned off (AFAIK all Android phones support this) and then use it once it s charged. This charger supports a wide range of phones (including LG phones from 5+ years ago and Nokia from ~10 years ago) so it could be good for charging an old phone in an emergency. Older phones need less power to charge and generally last longer between charges. With an alkaline battery the charger works a lot better, but it s still for emergency use as I generally only use rechargeable batteries. This is a device I might put in the bottom of my case when travelling, not a device that I ll use regularly. I don t think that this is a deficiency in the product, it s just a limit of what can be done with the power requirements of modern phones and the capabilities of AA batteries. If you want a battery to use while playing Ingress you definitely need something a lot larger. This charger is good for emergencies but not suitable for my main use (charging phones while playing Ingress) so I ll give it to my parents, they do a lot of hiking and camping and my father often goes fishing. Taking a few AA batteries to charge a phone would be much more convenient for such use than taking one of the larger batteries I use for Ingress. Also as the charger is relatively cheap there s less potential financial loss if you drop it in sea-water. 20800mAh Battery Above is a picture of a 20800mAh battery pack that MobileZap sent me [4]. I haven t yet discovered what it s capacity is, during the recent Interitus anomaly it kept my Galaxy Note 2 and my wife s Nexus 5 adequately charged for 5 hours of intensive Ingress playing (we alternated between phones). Without an external battery it would be unusual for either of those phones to last for 2 hours. The battery has two ports labeled 1A and 2.1A which implies that it can supply a total of 3.1A. I am a little dubious of that implication. After the Interitus anomaly my wife and I had dinner with some friends and I let a friend charge her Samsung Galaxy S5 on the battery at the same time as my Note 2 (which claimed to be 2% charged). When I connected my friend s phone my phone instantly shut itself down due to lack of charge, so I presume that the supply on the 2.1A port is reduced when a demanding device is connected to the 1A port. The next day my wife and I went to the city again to play Ingress and attend a protest against some of the awful things that the Abbott government is doing. As an experiment I didn t charge this battery to see how it would go for two days of use charging two phones. Again there was no problem and it still claimed to be half charged when we got home. So far the only problem I ve found with this battery is that it never reports being fully charged, not even if I leave it on charge for days. That doesn t seem to be a great problem to me, if I disconnect it before it s fully charged then that won t hurt the battery life (unlike NiCd batteries that have to be fully charged every time) and even if it s not fully charged it will still charge phones for a long time. The battery is about 19cm long. I have jeans with unusually large pockets to fit large phones [5] so I can fit this battery in my pocket while a USB cable is connected to charge my phone. I also have a new winter jacket from Scottwear which has pockets that can fit the battery (and lots of other things). Anyone who doesn t have such clothing should plan to use a backpack, handbag, or some other bag this type of battery won t fit in the pockets in most clothing. MobileZap also has some other interesting Nexus 5 accessories [6].

• [1] http://www.mobilezap.com.au/38308-tree-frog-anti-slip-dashboard-pad.htm
• [2] http://www.mobilezap.com.au/39366-avantree-high-power-31a-dual-usb-universal-car-charger.htm
• [3] http://www.mobilezap.com.au/13786-chargetube-emergency-phone-charger.htm
• [4] http://www.mobilezap.com.au/42214-kit-high-power-20800-mah-dual-usb-portable-charger.htm
• [5] http://etbe.coker.com.au/2012/11/17/geeky-jeans/
• [6] http://www.mobilezap.com.au/34597/google/nexus-5.htm

Five years ago Debian and most derivatives switched from the standard GNU C Library (GLIBC) to the Embedded GLIBC (EGLIBC). Debian is now about to take the reverse way switching back to GLIBC, as EGLIBC is now a dead project, the last release being the 2.19 one. At the time of writing the glibc package has been uploaded to experimental and sits in the NEW queue. EGLIBC is dead for a good reason: the GLIBC development has changed a lot in the recent years, due to two major events: Ulrich Drepper leaving Red Hat and the GLIBC development, and the GLIBC steering committe self-dissolving. This has resulted in a much more friendly development based on team work with good cooperation. The development is now based on peer review, which results in less buggy code (humans do make mistakes). It has also resulted in things that were clearly impossible before, like using the same repository for all architectures, and even getting rid of the ports/ directory. Before we used to have two sets of architectures, the main ones in the glibc repository with architectures like x86, SuperH or SPARC, and the secondary ones in the glibc-ports repository with architectures like ARM or MIPS. As you can see the separation was quite arbitrary, and often leaded to missing changes on the secondary architectures. We also got real stable branches, with regular fixes. The most important EGLIBC features have been merged to GLIBC, including for example the use of non-bash but POSIX shell, or the renaming of reserved keywords. The notable exception is the support for configurable components, which we originally planned to use for Debian-Installer, by building a smaller flavor using -Os and without NIS and RPC support. At the end we never worked on that, and it seems that the hardware targeted by Debian has grown faster than the GLIBC size, so that is not really a big loss. At the end, we ended up with only 5 missing patches from the EGLIBC tree:

• Installation of *_pic.a files for use of mklibs in debian-installer
• Dynamic reload of /etc/resolv.conf
• Installation of headers during bootstrapping
• Workarounds for PowerPC 8xx CPUs
• Access to FPSCR on SH4

The package names are unchanged (except the source package and the binary package containing the sources) so the transition is fully transparent for the users. I would like to thank all the CodeSourcery employees who worked on EGLIBC, with a special thank to Joseph Myers who spent countless hours to merge the most important EGLIBC changes back to GLIBC, and sent regular emails about the merge status. I would also like to thanks all the people on the GLIBC side that made the change to happen, and all persons participating in the GLIBC development.

The Debian Edu / Skolelinux project provide an instruction manual for teachers, system administrators and other users that contain useful tips for setting up and maintaining a Debian Edu installation. This text is about how the text processing of this manual is handled in the project. One goal of the project is to provide information in the native language of its users, and for this we need to handle translations. But we also want to make sure each language contain the same information, so for this we need a good way to keep the translations in sync. And we want it to be easy for our users to improve the documentation, avoiding the need to learn special formats or tools to contribute, and the obvious way to do this is to make it possible to edit the documentation using a web browser. We also want it to be easy for translators to keep the translation up to date, and give them help in figuring out what need to be translated. Here is the list of tools and the process we have found trying to reach all these goals. We maintain the authoritative source of our manual in the Debian wiki, as several wiki pages written in English. It consist of one front page with references to the different chapters, several pages for each chapter, and finally one "collection page" gluing all the chapters together into one large web page (aka the AllInOne page). The AllInOne page is the one used for further processing and translations. Thanks to the fact that the MoinMoin installation on wiki.debian.org support exporting pages in the Docbook format, we can fetch the list of pages to export using the raw version of the AllInOne page, loop over each of them to generate a Docbook XML version of the manual. This process also download images and transform image references to use the locally downloaded images. The generated Docbook XML files are slightly broken, so some post-processing is done using the documentation/scripts/get_manual program, and the result is a nice Docbook XML file (debian-edu-wheezy-manual.xml) and a handfull of images. The XML file can now be used to generate PDF, HTML and epub versions of the English manual. This is the basic step of our process, making PDF (using dblatex), HTML (using xsltproc) and epub (using dbtoepub) version from Docbook XML, and the resulting files are placed in the debian-edu-doc-en binary package. But English documentation is not enough for us. We want translated documentation too, and we want to make it easy for translators to track the English original. For this we use the poxml package, which allow us to transform the English Docbook XML file into a translation file (a .pot file), usable with the normal gettext based translation tools used by those translating free software. The pot file is used to create and maintain translation files (several .po files), which the translations update with the native language translations of all titles, paragraphs and blocks of text in the original. The next step is combining the original English Docbook XML and the translation file (say debian-edu-wheezy-manual.nb.po), to create a translated Docbook XML file (in this case debian-edu-wheezy-manual.nb.xml). This translated (or partly translated, if the translation is not complete) Docbook XML file can then be used like the original to create a PDF, HTML and epub version of the documentation. The translators use different tools to edit the .po files. We recommend using lokalize, while some use emacs and vi, others can use web based editors like Poodle or Transifex. All we care about is where the .po file end up, in our git repository. Updated translations can either be committed directly to git, or submitted as bug reports against the debian-edu-doc package. One challenge is images, which both might need to be translated (if they show translated user applications), and are needed in different formats when creating PDF and HTML versions (epub is a HTML version in this regard). For this we transform the original PNG images to the needed density and format during build, and have a way to provide translated images by storing translated versions in images/$LANGUAGECODE/. I am a bit unsure about the details here. The package maintainers know more. If you wonder what the result look like, we provide the content of the documentation packages on the web. See for example the Italian PDF version or the German HTML version. We do not yet build the epub version by default, but perhaps it will be done in the future. To learn more, check out the debian-edu-doc package, the manual on the wiki and the translation instructions in the manual.

Maybe someone else is using iDataPlex HW, so this might come in helpful... TL, DR: Dear Hardware Vendors! Please don't force people to use specific Linux distributions when a simple unzip command would do the same. At work we run (amongst others) an HPC system consisting of 120 IBM System X iDataPlex dx360 M4 Nodes. Recently, after some MCE logs caused by DIMM errors, the IBM guy refused to send me replacement DIMMs if if didn't update IMM firware and tortue test the machine some days more. Well, ok... IBM uses the so called UpdateXpress tool to update IMM and UEFI firmware. The tool is available for rhel4 to rhel7 (both 32 and 64 bit), SLES10 & SLES11 and Microsoft Windows. I tried all of the available UpdateXpress System Pack Installers, with basically the same result everytime:

WARNING! This package doesn't appear to match your system.
         The following information was determined for your system:
           distribution = Unknown
           release = -1
           processor architecture = Unknown

I tried to set up a fake RHEL environment (customized /usr/local/uname, custom /et/RHel-release, etc...) but after some more tries, the installer (yes, it's still the installer I am trying to run) fails with some strange SEGFAULTS. After some closer inspection, (well, the RHEL4 uxspi binary is about 109MB big!) i tried to unzip it, and voila:

~/firmware$ unzip ibm_utl_uxspi_9.60_rhel4_32-64.bin
Archive:  ibm_utl_uxspi_9.60_rhel4_32-64.bin
   creating: rhel6_64/image/
  inflating: rhel6_64/image/libpegasus.nsp.so  
  inflating: rhel6_64/image/libUxliteImmUsbLan.so  
  inflating: rhel6_64/image/eccConnect.properties  
  inflating: rhel6_64/image/libpegslp_client.so  
   creating: rhel6_64/image/esxi_assoc_templates/
  inflating: rhel6_64/image/esxi_assoc_templates/elxFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/elxCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/ibmc.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/FPGA-S.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/degraded.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/brocadeFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/qlgcFC.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/broadcom.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/LSI.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/brocadeCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/uefi.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/diags.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/qlgcCNA.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/VMwareESXi.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/ibmc2.uxt  
  inflating: rhel6_64/image/esxi_assoc_templates/FPGA.uxt  
  inflating: rhel6_64/image/libipmi_client.so  
  inflating: rhel6_64/image/UXLite_UPDATEID.dat  
  inflating: rhel6_64/image/libacpi.so  
  inflating: rhel6_64/image/libibmsp6_openipmi.so  
  ...

I managed to find a iflash64 binary, and after so trial and error, came up with the following workflow:

• Unzip the .uxz firmware file (you get this from the IBM Download Central):

  unzip ibm_fw_imm2_1aoo56d-3.73_anyos_noarch.uxz
  Archive:  ibm_fw_imm2_1aoo56d-3.73_anyos_noarch.uxz
  replace imm2_1aoo56d-3.73.upd? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
   extracting: imm2_1aoo56d-3.73.upd
    inflating: immalert.mib
    inflating: immRegistries.zip
    inflating: imm.mib
    inflating: OpenSourceNotices.html
    inflating: payload.xml
  

• Run the iflash tool:

./iflash64 --host your_host_imm_IP --user your_username --password your_password --package imm2_1aoo56d-3.73.upd --reboot
IBM Command Line IMM Flash Update Utility v2.02.11
Licensed Materials - Property of IBM
(C) Copyright IBM Corp. 2009 - 2014  All Rights Reserved.
Connected to IMM at IP address x.x.x.x.
Update package firmware type: IMM2
Update package build level:   xxxxxx
Target's current build level: xxxxxx
Would you like to continue with the update? y/n: y
Node 0: The IMM is preparing to receive the update.
Node 0: Transferring image: 98%
Node 0: Transfer complete.
Node 0: Validating image.
Node 0: Updating firmware:  0%
Node 0: Updating firmware:  100%
Node 0: Update complete.
Performing activation of the firmware:
Connected to IMM at IP address 10.8.2.4 successfully.
..........
Waiting up to 360 seconds for the activation to complete:
..
..

And so on... Conclusio: Dear Hardware Vendors! Please don't force people to use specific Linux distributions when a simple unzip command would do the same.

A new package pkgKitten is now on CRAN in its infant version 0.1.0. It does only one thing: create R packages that do not suck. Or at least not as much as those created by the (R default) function package.skeleton() which creates files which the very standard, and very recommended, R CMD check then complains about. That is silly. Because each time R CMD check complains, something bad happens to a kitten somewhere. So pkgKitten offers a wrapper function kitten() which creates neat little packages which do not upset R CMD check. A complete little demo is on the pkgKitten webpage which also has pointers to the pkgKitten GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

RcppRedis, another new package of mine, appeared on CRAN a few weeks ago just in time for our annual R/Finance conference. And as of today, we also have Windows binaries thanks to generous help from John Buonagurio who help building the required 32- and 64-bit libraries for Windows, and Uwe Ligges who now installed them for the CRAN and win-builder machiness. Binaries for release 0.1.1 should be available within a day. To describe the package, I also just set up a page about RcppRedis so rather than repeating everything here I invite you to follow the link. The ever-so-brief NEWS entry for the two first uploads follows.

Changes in version 0.1.1 (2014-06-09)

• Now with Windows support thanks to the installation of builds of the hiredis library (created by John Buonagurio) at CRAN / win-builder (thanks to Uwe Ligges)
• Added support for new command zcount

Changes in version 0.1.0 (2014-05-10)

• Initial CRAN upload

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

A new minor release 0.11.2 of Rcpp is now on the CRAN network for GNU R, and binaries for Debian have also been uploaded. The release smoothes a few edges on both the Rcpp side itself, as well as on the interaction between Rcpp and R which since release 3.1.0 offers a few new features related to C++11. Kevin added a couple of neat extensions related to vectors, a new ListOf templated list class, as well as a new option to warn on implicit casts. We decided not to make this option the default as it may be too common in some packages. JJ took care of a few buglets related to the wonderful Rcpp Attributes. See the NEWS file section below for details, or the ChangeLog file in the package and on the Rcpp Changelog page. Note that the diffstat reported by CRANberries is very large as Kevin also committed a whitespace cleanup which touched almost all files. As before, we tested this release by building against all CRAN packages which depend upon Rcpp. In fact we did three such runs leading up to the release. Only one package was blacklisted (as I currently don't have CUDA set-up), two had what may be internal errors or tests which were too restrictive, sixteen suffered from missing packages or RGL devices --- but the remaining 202 packages all built and tested cleanly. Detailed results of those tests (and the scripts for it) are in the rcpp-logs repo GitHub. There are a number of other fixes, upgrades and other extensions detailed in NEWS file extract below, in the ChangeLog file in the package and on the Rcpp Changelog page.

Changes in Rcpp version 0.11.2 (2014-06-06

• Changes in Rcpp API:
  • Implicit conversions, e.g. between NumericVector and IntegerVector, will now give warnings if you use #define RCPP_WARN_ON_COERCE before including the Rcpp headers.
  • Templated List containers, ListOf<T>, have been introduced. When subsetting such containers, the return is assumed to be of type T, allowing code such as ListOf<NumericVector> x; NumericVector y = x[0] + x[1] + x[2].
  • In a number of instances, returned results are protected and/or cast more carefully.
• Changes in Rcpp Attributes
  • Trailing line comments are now stripped by the attributes parser. This allows the parser to handle C++ source files containing comments inline with function arguments.
  • The USE_CXX1X environment variable is now defined by the cpp11 plugin when R >= 3.1. Two additional plugins have been added for use with C++0x (eg when using g++ 4.6.* as on Windows) as well as C++1y for compilers beginning to support the next revision of the standard; additional fallback is provided for Windows.
  • compileAttributes() now also considers Imports: which may suppress a warning when running Rcpp.package.skeleton().

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

The Debian project is excited to announce that we are now accepting presentations, discussion sessions and tutorials for our DebConf14 conference which will take place in Portland State University, Oregon, USA from 23 to 31 August. Submitting an event To submit an event, first register as an attendee for DebConf14 in the conference management system. If you have any doubts or have problems with the registration process please check the Registration FAQ. After registering, go to the event submission page, or click on the Create an event option from the management system. Describe your submission in the web form. The most common event types are Lecture or Open Discussion (BoF). Please include a short title (to make it easy to produce a compact schedule) and an engaging description of the event. Tracks We will organize some talks into thematic tracks. If you have a proposal for a DebConf track, such as "Debian ARM", "Debian Infrastructure", or "Community Outreach" please contact talks@debconf.org. If you would like to be a track coordinator, please volunteer on the given mail address. Format of the events A regular session will be 45 minutes long, including time for questions. There will be a 15 minute breaks between events. Submissions are not limited to traditional talks: you could propose a performance, art installation, debate, or anything else. If you have any specific requirements for your event, please send an email to talks@debconf.org with the details of your requirements and be sure to mention your event title in the subject. Deadline While we ask speakers to submit their events before the deadline of 7 July 2014, 23:59:59 UTC, late submissions will continue to be accepted for scheduling until the end of DebConf. All attendees will have an opportunity to schedule ad-hoc events during DebConf itself if we have space for them. Very promising late submissions may be considered for inclusion in the main conference. Note that ad-hoc events have a much lower chance of video archiving, and streaming, so if you want these services it's better to get your submissions in early. DebConf official events will be broadcast live on the Internet when possible, and videos of the talks will be published on the web along with the presentation slides and papers. For private communication regarding your talk, or for more general ideas, or questions about the event and talks, please mail us We hope to you see you and share some good times with you this year in Portland during DebConf14!

I m writing this on the train back from the ZuriHac Haskell Hackathon in Z rich, generously sponsored by Better and Google. My goal for this event was to attract new people to work on GHC, the Haskell compiler, so I announced a GHC bugsquashing project . I collected a few seemingly simple ticket that have a good effort/reward ratio for beginners and encouraged those who showed up to pick one to work on. Roughly six people started, and four actually worked on GHC on all three days. The biggest hurdle for them was to get GHC built for the first time, especially those using a Mac or Windows. They also had to learn to avoid recompilation of the whole compiler, which takes an annoying amount of time (~30 minutes for most people). But once such hurdles weren taken all of them managed to find their way around the source code to the place they need to touch and were able to produce a patch, some of which are already merged into GHC master. When I wasn t giving tips and hints I was working on various small tickets myself, but nothing of great impact. I very much hope that this event will pay off and one or two of the newcomers end up being regular contributors to GHC.

We took breaks from our respective projects to listen to interesting talks by Edward Kmett and Simon Marlow, and on Saturday evening we all went to the shores of the Zurisee and had a nice Barbecue. It was a good opportunity to get into contact with more of the attendees (the hacking itself was separated in multiple smaller office rooms) and I was happy to hear about people having read my recent Call Arity paper, and even found it valuable.

Thanks to the organizers and sponsors for this nice opportunity!

According to the Free Software Foundation, the Debian distribution is not a Free System Distribution. While I, as a Debian Developer, disagree with that point of view, it should be obvious that the opinion of an organization that engages in politics (or, at the very least, in political lobbying) and of which I am not a member should not be my concern. However, it has long irked me the wrong way that these claims exist. A few weeks ago, I found the System distribution commitments page on Wikipedia. This does change the situation somewhat; I cannot say that a page explaining an opinion of the FSF is wrong; but if that opinion is copied nearly verbatim to a website which claims neutrality as one of its "Core content policies", then the situation is significantly different. When I originally found that page, it was immediately obvious where the content came from, but there was no reference listed, so I added a "citation needed" tag, the first time I ever did so on Wikipedia. Now, a few weeks later, a reference has been added, and the "citation needed" tag removed. However, as expected, the only reference which has been added is the highly opinionated document on the FSF's website, which is hardly neutral. I've added another tag (this time a "which" tag, asking for more clarification) and added a Talk page listing my concerns with the article. But I can't really explain in Wikipedia why I think the whole premise is wrong; that's what a blog is for. So, here goes. The FSF lists two reasons why they do not endorse Debian:

1. The non-free and contrib repositories
2. An unsubstantiated claim that "the installer in some cases recommends [...] non-free firmware files for the peripherals on the machine."

Let's have a look at them, each in turn.

Non-free and contrib It is true that Debian provides non-free software in separate repositories, and that software in these repositories does not comply with either the Debian Free Software Guidelines (DFSG) or the FSF's Free Software Definition. However, the non-free and contrib repositories are fairly small:

wouter@carillon:/var/lib/apt/lists$ awk -F':' 'BEGIN total=0 /^Size/ total+=$2 END print total ' ftp.be.debian.org_debian_dists_sid_non-free_binary-amd64_Packages
4398812934

That is, the non-free repository contains just over 4G of software (after installation). Contrib is even smaller:

wouter@carillon:/var/lib/apt/lists$ awk -F':' 'BEGIN total=0 /^Size/ total+=$2 END print total ' ftp.be.debian.org_debian_dists_sid_contrib_binary-amd64_Packages 
306578636

While, combined, this is approximately the same size as some other distributions, free and non-free together, it pales in comparison with the "main" repository:

wouter@carillon:/var/lib/apt/lists$ awk -F':' 'BEGIN total=0 /^Size/ total+=$2 END print total ' ftp.be.debian.org_debian_dists_sid_main_binary-amd64_Packages
60025033622

i.e., the "main" repository contains just shy of 60G of software. Of course, whether non-free is large or not should not matter at all; the fact that it exists in the first place is a problem for free software advocates. But of those 4 gigabytes, what exactly is the non-free repository used for? For that, the Debian popularity contest has a page specifically for the non-free section. Looking at this page, and ignoring the packages that are listed as "Not in sid" (which either means they were installed from third-party suppliers and therefore cannot be attributed to Debian, or were only available in older distributions so should not be held against the Debian of today anymore), we see three main uses of non-free:

1. Various implementations of non-free algorithms (unrar, libfaac0, etc)
2. A lot of packages containing non-free firmware
3. A number of packages containing non-free documentation

The first package which does not fit any of those descriptions is iozone3, currently on the 64th position of the popularity contest (including packages "Not in sid"). The first item in this list is something which most free software advocates will try to avoid. After all, for all of those algorithms, alternatives exist, and so there is no real reason why one would want to use such software, other than "someone sent me a file encoded according to that algorithm, and I want to decode it". Debian gives you the option to do so, but a valid case could be made for not doing so. The second item in that list is really an alternate version of "the installer recommends non-free software", so I'll come back on that in the next section. The 3rd item in the above list deserves some special attention. Just over 10 years ago, Debian clarified its position where it clearly specified that documentation and programs (as both are interpreted as "software") should follow the same set of guidelines regarding freedom. The FSF disagrees with this position, and chose to release some of its documentation under the GFDL with invariant sections, which Debian decided is a non-free license in 2006. Since then, such GNU documentation would no longer be available were Debian to ditch the non-free repository.

Recommending non-free firmware As I said above, the FSF's page claims that the Debian installer recommends non-free firmware "in some cases", without going into much detail. Special care should be taken in understanding the word "recommend" here; wiktionary defines it as follows:

recommend (third-person singular simple present recommends, present participle recommending, simple past and past participle recommended)

1. (transitive) To bestow commendation on; to represent favourably; to suggest, endorse or encourage as an appropriate choice.

The board recommends Philips, given his ample experience in similar positions.

1. (transitive) To make acceptable; to attract favor to.
2. (transitive) To advise, propose, counsel favorably

The therapist recommends resting the mind and exercising the body.

1. (transitive, archaic) To commit, confide to another's care, confidence or acceptance, with favoring representations

A medieval oblate's parents recommended the boy for life to God and the monastery

However, according to the FSF, merely showing a firmware filename in unencrypted form counts as a recommendation. I've always found that use of the word "recommendation" to be dubious, at best. Ignoring that part, we can make the following observations: The debian-installer module ("udeb") which is responsible for configuring apt is called apt-mirror-setup (code in the apt-setup git repository). The actual debconf templates which allow the user to enable the non-free or contrib repositories are the apt-setup/non-free and apt-setup/contrib templates, and they look like this:

Template: apt-setup/non-free
Type: boolean
Default: false
# :sl1:
_Description: Use non-free software?
 Some non-free software has been made to work with Debian. Though this
 software is not at all a part of Debian, standard Debian tools can be used
 to install it. This software has varying licenses which may prevent you
 from using, modifying, or sharing it.
 .
 Please choose whether you want to have it available anyway.
Template: apt-setup/contrib
Type: boolean
Default: false
# :sl1:
_Description: Use contrib software?
 Some additional software has been made to work with Debian. Though this
 software is free, it depends on non-free software for its operation. This
 software is not a part of Debian, but standard Debian tools can be
 used to install it.
 .
 Please choose whether you want this software to be made available to you.

Note the lines Default: false, which imply that the default answer to these questions is to disable the non-free and contrib repositories. A question which defaults to false isn't a recommendation, nor do I read the text as any form of recommendation to install these repositories (quite the contrary, in fact). More important than whether the default is to enable or disable these repositories is whether this question is even shown to the user in normal installations. Whether this is the case isn't decided by any template; instead, it is decided by the code that calls the templates. That code currently looks like this:

STATE=1
while true; do
        case "$STATE" in
            1)
                db_input low apt-setup/non-free   true
                ;;
            2)
                # If they chose not to use non-free, ask about
                # contrib, with a default of no. If they chose to
                # use non-free, they get contrib too..
                db_get apt-setup/non-free
                if [ "$RET" = false ]; then
                        db_fget apt-setup/contrib seen
                        if [ "$RET" = false ]; then
                                db_set apt-setup/contrib false
                        fi
                        db_input low apt-setup/contrib   true
                else
                        db_fget apt-setup/contrib seen
                        if [ "$RET" = false ]; then
                                db_set apt-setup/contrib true
                        fi
                fi
                ;;
            *)
                break
                ;;
        esac
        if db_go; then
                STATE=$(($STATE + 1))
        else
                STATE=$(($STATE - 1))
        fi
done

Note the "low" in the db_input lines; it means the question is only shown at low priority, i.e., in "expert" mode. The result is that, unless you enable expert mode, you won't even see the question about the non-free repository, and it will be disabled by default. There won't even be any mention of it in your sources.list file. Of course, the above isn't about firmware files, which is what the FSF website claims. So, there is a second part to this story. If you run a system with hardware for which there is a non-free firmware package in the non-free repository, and you provided firmware for this hardware to the installer through various means, then the hw-detect module comes into play. This has code which currently looks like this:

# install cached firmware debs
if [ -d /var/cache/firmware ]; then
    for deb in /var/cache/firmware/*.deb; do
            if [ -f "$deb" ]; then
                    cp -a "$deb" /target/tmp
                    # TODO debconf passthrough
                    if ! in-target dpkg -i "/tmp/$(basename "$deb")"; then
                            # dpkg failed, force removal of package
                            in-target dpkg --force-depends --remove "$(deb_package "$deb")"   true
                    fi
                    rm -f "/target/tmp/$deb"
                    need_nonfree=1
            fi
    done
fi
if [ "$need_nonfree" ]; then
    db_set apt-setup/non-free true
fi

As can bee seen, if any firmware .deb files exist under /var/cache/firmware in the installer environment, then these will be installed on the target system and default answer to the apt-setup/non-free question will be changed to enable the non-free repository. How do such packages end up there? Well, that's done by the same hw-detect udeb, in a different script. The code there is a bit too long to quote in this already far too long blog post, but it boils down to:

• If the installer detects missing firmware, the user is asked whether (s)he wants to install firmware, with the following template:

  Template: hw-detect/load_firmware
  Type: boolean
  Default: true
  # :sl2:
  _Description: Load missing firmware from removable media?
   Some of your hardware needs non-free firmware files to operate. The
   firmware can be loaded from removable media, such as a USB stick or
   floppy.
   .
   The missing firmware files are: $ FILES 
   .
   If you have such media available now, insert it, and continue.
  

• If the user agreed to doing so (and to me, the above template does not read as a recommendation), any firmware .deb files that are provided to the installer are tested for the missing firmware files.
• If any of those files are found, these .deb files are copied to the /var/cache/firmware directory, and the firmware files are made available to the running kernel.

Conclusion In my opinion, while some of the claims on the FSF website are correct, others are at the very least misleading. Debian does not recommend the use of non-free software; and while providing non-free firmware to the installer isn't too difficult, it isn't documented very well; and as such installing a system with non-free enabled isn't something you can easily do by accident. Debian and the FSF clearly disagree on certain things, and this disagreement, not any policies by the Debian project, is the main reason why the FSF won't encourage Debian as a free distribution. I don't think the Debian project will easily agree to get rid of the non-free repository while the FSF still produces non-free documentation or while non-free firmware is still a fact of life in this world. However, I do think the following should happen:

1. The FSF should relicense its non-free documentation so it becomes truly free.
2. Once that has happened, Debian should rename its non-free repository to a "hardware-support" repository (or something along those lines), and remove everything that does not exist solely for the support of proprietary hardware (i.e., non-free drivers, non-free firmware, etc).
3. The installer could be clearer on the fact that if non-free firmware is found to be in use for the system, the user is using software that cannot be properly supported by the Debian project.

While the above wouldn't necessarily bring the opinions of the FSF and Debian 100% in line, it would at least bring their positions closer to eachother. Unfortunately, however, I doubt that this will happen; and that's a shame, because in the end, both Debian and the FSF want the same thing: a world where Free Software is the norm, not the exception.

Update

• The first version of this text said that providing firmware is not very easy. As was pointed out to me, it isn't too hard, but it's not very well documented and not part of the 'regular' installer images.
• Add contrib too, when we compare non-free and main
• Clarify when GFDL is non-free

Today I attended the Don't Spy On Us campaign's Day Of Action at Shoreditch Town Hall in London. I'm not sure how much actual action there was, but the talking was interesting. Retrospective Day of Action drinking game: drink every time you hear the phrase "If you have nothing to hide, you have nothing to fear." The spooks have a really good marketing department. I don't write a lot on the internet any more - something I regret, actually. It can't even be because there are alternative places I am writing, because over the last couple of years I have been closing most of my social media accounts. I just share much less of myself online. On the internet, nothing is ephemeral. Bruce Schneier says so. Choose your words carefully. The thing about blogging is that it's so public. It's often tied to the writer's real-life identity. One of the valuable things about social media services is that they supposedly let you restrict who can read your words - the trade-off being that you must also grant access to advertisers, spies, cybercriminals... Most memorable moments of the day:

• Alan Rusbridger contrasting how he felt as a journalist working on the Snowden stories when in the US versus the UK - when in the UK, it's like "you know something bad is going to happen".
• Being asked to leave a tiny meeting room, in order that I could show my ticket for the next talk, only to retake the seat where I had left my bag. Go bureaucracy!
• Shami Chakrabarti leaving in her sunglasses like a rockstar.

In general, there were lots of famous people walking around as if they were normal. I was in the same room as Cory Doctorow, Jimmy Wales and Bruce Schneier at the same time. Ahem, action (mostly for UK citizens):

• Sign the petition.
• Join the various groups who organized today (Open Rights Group, Liberty, Big Brother Watch, Privacy International, Article 19, English PEN).
• Ask to meet your MP to talk about privacy at their next surgery - you can phone the House of Commons.

I must echo John Sullivan's post: GPG keysigning and government identification. John states some very important reasons for people everywhere to verify the identities of those parties they sign GPG keys with in a meaningful way, and that means, not just trusting government-issued IDs. As he says, It's not the Web of Amateur ID Checking. And I'll take the opportunity to expand, based on what some of us saw in Debian, on what this means. I know most people (even most people involved in Free Software development not everybody needs to join a globally-distributed, thousand-people-strong project such as Debian) are not that much into GPG, trust keyrings, or understand the value of a strong set of cross-signatures. I know many people have never been part of a key-signing party. I have been to several. And it was a very interesting experience. Fun, at the beginning at least, but quite tiring at the end. I was part of what could very well constitute the largest KSP ever in DebConf5 (Finland, 2005). Quite awe-inspiring We were over 200 people, all lined up with a printed list on one hand, our passport (or ID card for EU citizens) in the other. Actwally, we stood face to face, in a ribbon-like ring. And, after the basic explanation was given, it was time to check ID documents. And so it began. The rationale of this ring is that every person who signed up for the KSP would verify each of the others' identities. Were anything fishy to happen, somebody would surely raise a voice of alert. Of course, the interaction between every two people had to be quick More like a game than like a real check. "Hi, I'm #142 on the list. I checked, my ID is OK and my fingerprint is OK." "OK, I'm #35, I also printed the document and checked both my ID and my fingerprint are OK." The passport changes hands, the person in front of me takes the unique opportunity to look at a Mexican passport while I look at a Somewhere-y one. And all is fine and dandy. The first interactions do include some chatter while we grab up speed, so maybe a minute is spent Later on, we all get a bit tired, and things speed up a bit. But anyway, we were close to 200 people That means we surely spent over 120 minutes (2 full hours) checking ID documents. Of course, not all of the time under ideal lighting conditions. After two hours, nobody was checking anything anymore. But yes, as a group where we trust each other more than most social groups I have ever met, we did trust on others raising the alarm were anything fishy to happen. And we all finished happy and got home with a bucketload of signatures on. Yay! One year later, DebConf happened in Mexico. My friend Martin Krafft tested the system, perhaps cheerful and playful in his intent but the flaw in key signing parties such as the one I described he unveiled was huge: People join the KSP just because it's a social ritual, without putting any thought or judgement in it. And, by doing so, we ended up dilluting instead of strengthening our web of trust. Martin identified himself using an official-looking ID. According to his recount of the facts, he did start presenting a German ID and later switched to this other document. We could say it was a real ID from a fake country, or that it was a fake ID. It is up to each person to judge. But anyway, Martin brought his Transnational Republic ID document, and many tens of people agreed to sign his key based on it Or rather, based on it plus his outgoing, friendly personality. I did, at least, know perfectly well who he was, after knowing him for three years already. Many among us also did. Until he reached a very dilligent person, Manoj, that got disgusted by this experiment and loudly denounced it. Right, Manoj is known to have strong views, and using fake IDs is (or, at least, was) outside his definition of fair play. Some time after DebConf, a huge thread erupted questioning Martin's actions, as well as questioning what do we trust when we sign an identity document (a GPG key). So... We continued having traditional key signing parties for a couple of years, although more carefully and with more buzz regarding these issues. Until we finally decided to switch the protocol to a better one: One that ensures we do get some more talk and inter-personal recognition. We don't need everybody to cross-sign with everyone else A better trust comes from people chatting with each other and being able to actually pin-point who a person is, what do they do. And yes, at KSPs most people still require ID documents in order to cross-sign. Now... What do I think about this? First of all, if we have not ever talked for at least enough time for me to recognize you, don't be surprised: I won't sign your key or request you to sign mine (and note, I have quite a bad memory when it comes to faces and names). If it's the first conference (or social ocassion) we come together, I will most likely not look for key exchanges either. My personal way of verifying identities is by knowing the other person. So, no, I won't trust a government-issued ID. I know I will be signing some people based on something other than their name, but hey I know many people already who live pseudonymously, and if they choose for whatever reason to forgo their original name, their original name should not mean anything to me either. I know them by their pseudonym, and based on that pseudonym I will sign their identities. But... *sigh*, this post turned out quite long, and I'm not yet getting anywhere ;-) But what this means in the end is: We must stop and think what do we mean when we exchange signatures. We are not validating a person's worth. We are not validating that a government believes who they claim to be. We are validating we trust them to be identified with the (name,mail,affiliation) they are presenting us. And yes, our signature is much more than just a social rite It is a binding document. I don't know if a GPG signature is legally binding anywhere (I'm tempted to believe it is, as most jurisdictions do accept digital signatures, and the procedure is mathematically sound and criptographically strong), but it does have a high value for our project, and for many other projects in the Free Software world. So, wrapping up, I will also invite (just like John did) you to read the E-mail self-defense guide, published by the FSF in honor of today's Reset The Net effort.

My last update about ongoing development of botch, the bootstrap/build ordering tool chain, was four months ago and about several incremental updates. This post will be of similar nature. The most interesting news is probably the additional data that bootstrap.debian.net now provides. This is listed in the next section. All subsequent sections then list the changes under the hood that made the additions to bootstrap.debian.net possible.

bootstrap.debian.net The bootstrap.debian.net service used to have botch as a git submodule but now runs botch from its Debian package. This at least proves that the botch Debian package is mature enough to do useful stuff with it. In addition to the bootstrapping results by architecture, bootstrap.debian.net now also hosts the following additional services:

• History of graph size shows how the dependency graph developed over time for a normal self-contained repository plus for both minimizing strategies, updated every five days
• Crossbuild dependency satisfaction gives an overview of reasons why the crossbuild dependency situation cannot yet be analyzed with bug numbers where applicable
• Crossbuild order modifies the metadata of a repository so that the crossbuild dependency situation can be analyzed and then outputs an overview page as it is done for every architecture on the main page
• Source package importance calculates the port metric for source packages on a daily basis

Further improvements concern how dependency cycles are now presented in the html overviews. While before, vertices in a cycle where separated by commas as if they were simple package lists, vertices are now connected by unicode arrows. Dashed arrows indicate build dependencies while solid arrows indicate builds-from relationships. For what it's worth, installation set vertices now contain their installation set in their title attribute.

Debian package Botch has long depended on features of an unreleased version of dose3 which in turn depended on an unrelease version of libcudf. Both projects have recently made new releases so that I was now able to drop the dose3 git submodule and rely on the host system's dose3 version instead. This also made it possible to create a Debian package of botch which currently sits at Debian mentors. Writing the package also finally made me create a usable install target in the Makefile as well as adding stubs for the manpages of the 44 applications that botch currently ships. The actual content of these manpages still has to be written. The only documentation botch currently ships in the botch-doc package is an offline version of the wiki on gitorious. The new page ExamplesGraphs even includes pictures.

Cross By default, botch analyzes the native bootstrapping phase. That is, assume that the initial set of Essential:yes and build-essential packages magically exists and find out how to bootstrap the rest from there through native compilation. But part of the bootstrapping problem is also to create the set of Essential:yes and build-essential packages from nothing via cross compilation. Botch is unable to analyze the cross phase because too many packages cannot satisfy their crossbuild dependencies due to multiarch conflicts. This problem is only about the dependency metadata and not about whether a given source package actually crosscompiles fine in practice. Helmut Grohne has done great work with rebootstrap which is regularly run by jenkins.debian.net. He convinced me that we need an overview of what packages are blocking the analysis of the cross case and that it was useful to have a crossbuild order even if that was a fake order just to have a rough overview of the current situation in Debian Sid. I wrote a couple of scripts which would run dose-builddebcheck on a repository, analyze which packages fail to satisfy their crossbuild dependencies and why, fix those cases by adjusting package metadata accordingly and repeat until all relevant source packages satisfy their crossbuild dependencies. The result of this can then be used to identify the packages that need to be modified as well as to generate a crossbuild order. The fixes to the metadata are done in an automatic fashion and do not necessarily reflect the real fix that would solve the problem. Nevertheless, I ended up agreeing that it is better to have a slightly wrong overview than no overview at all.

Minimizing the dependency graph size Installation sets in the dependency graph are calculated independent from each other. If two binary packages provide A, then dependencies on A in different installation sets might choose different binary packages as providers of A. The same holds for disjunctive dependencies. If a package depends on A C and another package depends on C A then there is no coordination to choose C so to minimize the overall amount of vertices in the graph. I implemented two methods to minimize the impact of cases where the dependency solver has multiple options to satisfy a dependency through Provides and dependency disjunctions. The first method is inspired by Helmut Grohne. An algorithm goes through all disjunctive binary dependencies and removes all virtual packages, leaving only real packages. Of the remaining real packages, the first one is selected. For build dependencies, the algorithm drops all but the first package in every disjunction. This is also what sbuild does. Unfortunately this solution produces an unsatisfiable dependency situation in most cases. This is because oftentimes it is necessary to select the virtual disjunctive dependency because of a conflict relationship introduced by another package. The second method involves aspcud, a cudf solver which can optimize a solution by a criteria. This solution is based on an idea by Pietro Abate who implemented the basis for this idea back in 2012. In contrast to a usual cudf problem, binary packages now also depend on the source packages they build from. If we now ask aspcud to find an installation set for one of the base source packages (I chose src:build-essential) then it will return an installation set that includes source packages. As an optimization criteria the number of source packages in the installation set is minimized. This solution would be flawless if there were no conflicts between binary packages. Due to conflicts not all binary packages that must be coinstallable for this strategy to work can be coinstalled. The quick and dirty solution is to remove all conflicts before passing the cudf universe to aspcud. But this also means that the solution does sometimes not work in practice.

Test cases Botch now finally has a test target in its Makefile. The test target tests two code paths of the native.sh script and the cross.sh script. Running these two scripts covers testing most parts of botch. Given that I did lots of refactoring in the past weeks, the test cases greatly helped to assure that I didnt break anything in the process. I also added autopkgtests to the Debian packaging which test the same things as the test target but naturally run the installed version of botch instead. The autopkgtests were a great help in weeding out some lasts bugs which made botch depend on being executed from its source directory.

Python 3 Reading the suggestions in the Debian python policy I evaluated the possibility to use Python 3 for the Python scripts in botch. While I was at it I added transparent decompression for gzip, bz2 and xz based on the file magic, replaced python-apt with python-debian because of bug#748922 and added argparse argument parsing to all scripts. Unfortunately I had to find out that Python 3 support does not yet seem to be possible for botch for the following reasons:

• no soap module for Python 3 in Debian (needed for bts access)
• hash randomization is turned on by default in Python 3 and therefore the graph output of networkx is not deterministic anymore (bug#749710)

Thus I settled for changing the code such that it would be compatible with Python 2 as well as with Python 3. Because of the changed string handling and sys.stdout properties in Python 3 this proved to be tricky. On the other hand this showed me bugs in my code where I was wrongly relying on deterministic dictionary key traversal.

Please stop recommending that checking government-issued ID is a good way to verify someone's identity before signing their GPG key. Have you been a US bartender before? Or held any other position where you've had to verify an ID? It's not an easy thing to do. People in those positions have books of valid IDs from different states. They have lights that show the security marks. They still get it wrong regularly. A very amateur fake ID, or borrowed real ID, will fool just about everyone in any informal context. What's even worse is that people have a habit of happily looking at passports from other countries than their own, and nodding knowingly. It's fun, but be honest, you have no idea what you're doing. How about just signing keys with people you would actually say you know well enough to trust? It's not the Web of Amateur ID Checking. ID checking is at best ineffective against the threats it's supposed to address, and is probably actually damaging to the Web of Trust because of the false sense of security. No idea what I'm talking about? Learn to encrypt your email by reading the FSF's new Email Self-Defense Guide, published in honor of today's Reset The Net effort.

Please stop recommending that checking government-issued ID is a good way to verify someone's identity before signing their GPG key. Have you been a US bartender before? Or held any other position where you've had to verify an ID? It's not an easy thing to do. People in those positions have books of valid IDs from different states. They have lights that show the security marks. They still get it wrong regularly. A very amateur fake ID, or borrowed real ID, will fool just about everyone in any informal context. What's even worse is that people have a habit of happily looking at passports from other countries than their own, and nodding knowingly. It's fun, but be honest, you have no idea what you're doing. How about just signing keys with people you would actually say you know well enough to trust? It's not the Web of Amateur ID Checking. ID checking is at best ineffective against the threats it's supposed to address, and is probably actually damaging to the Web of Trust because of the false sense of security. No idea what I'm talking about? Learn to encrypt your email by reading the FSF's new Email Self-Defense Guide, published in honor of today's Reset The Net effort.