The script isn't fully automated yet, I'm running it manually and watching for errors, but I do now have a simple script (~100 lines of perl) which runs edos-debcheck against each of the supported Emdebian architectures for unstable-grip and picks out those packages which are missing from the subset of packages which Emdebian monitors for Grip.

I was hoping to make these scripts into a normal Debian package but there are some limitations in that the scripts need to assume various pieces of Debian infrastructure. e.g. I started off using rsync to pull in the Packages files (along with Release and Sources) because apt has annoying behaviours with multiple architectures. (Emdebian Grip processes 7 architectures simultaneously on blavet.debian.org). Peter Palfrader kindly arranged for a local mirror to be available and now the entire Packages processing can be done using symlinks. Much, much better.

Actual package movement data comes directly from dak via projectb - something else which makes packaging these scripts for use on non debian.org boxes a bit hard.

Once again, I am indebted to the EDOS team (please, please keep working on edos-debcheck - it hasn't had uploads recently) because I tried to get this working with germinate but failed. (Colin - I'll be looking to borrow some of your time to work out what was going wrong.)

The dependency resolution script now means that I can meet release team requirements that when foo is updated in sid to depend on libbar2 but Emdebian Grip only has libbar1, the scripts will automatically pick up libbar2 and add it to Emdebian Grip. The current sid version gets pulled in and (the theory goes) will migrate into testing-grip as an almost automatically valid candidate. Processing of the dependency can happen within hours of the updated version of foo arriving in Emdebian Grip but as long as it happens within a few days I think everyone will be fine with it.

I've got a sync script too which can periodically run across the entire suite and check for packages which have slipped the net. I'll have to see how often that needs to run but once a week doesn't sound too painful.

Processing for Emdebian Grip is very fast. Most of the time is taken uploading:

2012-01-23 22:24:42 add sid-grip deb main armhf libfltk-images1.3 1.3.0-5em1

2012-01-23 22:24:52 add sid-grip deb main i386 libfltk-images1.3 1.3.0-5em1

2012-01-23 22:25:07 add sid-grip deb main mips libfltk-images1.3 1.3.0-5em1

2012-01-23 22:25:25 add sid-grip deb main powerpc libfltk-images1.3 1.3.0-5em1

2012-01-23 22:26:00 add sid-grip deb main amd64 libstdc++6-4.5-dev 4.5.3-12em1

2012-01-23 22:26:34 add sid-grip deb main armel libstdc++6-4.5-dev 4.5.3-12em1

(It was a lot longer, comparatively, when I was having to rsync and then dget instead of just read from the local filesystem.)

I've also been working on an init script which will gradually work through the subset of packages by order of Priority - approximately what a new architecture (like armhf) would do. Note that Emdebian Grip supports armhf already. The only slight concern is whether the init script needs to be throttled back to not flood the queues with hundreds of uploads an hour (which it could end up doing, at least for the first 24 hours).

Logs of package movements are available: http://www.emdebian.org/grip/logs.php

Packages can be searched too: http://www.emdebian.org/grip/search.php

More on Emdebian Integration on the Debian wiki: http://wiki.debian.org/EmdebianIntegration

Now all I need is a fix for #655919 FTP team?

When you shake the tree, things happen..:-). I sent targeted update requests for level 2 (mostly espeakup and grub2).

• Gujarati, Serbian and Tamil complete level 1
• Bulgarian, Esperanto, Japanese complete level 2 and are now full 100%
• Progress for Indonesian in level 1
• Icelandic completes level 2
• Simplified Chinese completes level 3 and is now full 100%
• Hebrew completes level 3
• First update for Wolof since ages

Status for D-I level 1 (core D-I files):

• 25 languages 100%: bg cs de eo es fr gu it ja kk km ko mr nb nl pl pt ru sk sr sv ta th uk zh_CN
• 5 languages 99%: bs fa hi si te
• 3 languages 97%: ar eu tr
• 2 languages 96%: be he
• 4 languages 95%: bn dz id zh_TW
• 4 languages 94%: ast da ga ro
• 4 languages 93%: el hu is lo
• 1 language 92%: sl
• 3 languages 91%: et pa vi
• others are 90% or below

Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

• 19 languages 100%: bg cs de eo es fa fr he is it ja kk nl pt ru si sk uk zh_CN
• 6 languages 99%: be da id sl sv th
• 4 languages 98%: ca eu pt_BR ro
• 1 language 96%: nb
• 2 langauges 95%: el fi
• 6 languages 94%: ar ast gl hr vi zh_TW
• 8 languages 92%: bn bs hu ko ne pl sr tr
• 10 languages 91%: dz ga gu ka km lt mk mr ta te
• others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

• 28 languages 100%: be bg bs ca cs de el eo es fa fi fr ga gl he is it ja kk nb nl pt ru sk sv th tr zh_CN
• 2 languages 98%: hu uk
• others are 90% or below

Full 100% completeness (hall of fame) for 14 languages: Bulgarian, Czech, German, Esperanto, Spanish, French, Italian, Japanese, Kazakh, Dutch, Portuguese, Russian, Slovak, Simplified Chinese

I gave a presentation on UEFI at LCA 2012 - you can watch it here, with the bonus repeat (and different jokes) here. It's a gentle introduction to UEFI, followed by some discussion of the problems we've faced in dealing with implementation bugs.

The fundamental problem is that UEFI is a lot of code. And I really do mean a lot of code. Ignoring drivers, the x86 Linux kernel is around 30MB of code. A comparable subset of the UEFI tree is around 35MB. UEFI is of a comparable degree of complexity to the Linux kernel. There's no reason to assume that the people who've actually written this code are significantly more or less competent than an average Linux developer, so all else being equal we'd probably expect somewhere around the same number of bugs per line. Of course, not all else is equal.

Even today, basically all hardware is shipping with BIOS by default. The only people to enable UEFI are enthusiasts. Various machines will pop up all kinds of dire warnings if you try to turn it on. UEFI has had very little real world testing. And it really does show. In the few months I've been working on UEFI I've discovered machines where SetVirtualAddressMap() calls code that has already been (per spec) discarded. I've seen cases where it was possible to create variables, but not to delete them. I've seen a machine that would irreparably corrupt its firmware when you tried to set a variable. I've tripped over code that fails to parse invalid boot variables, bricking the hardware. Many vendors independently fail to report the correct framebuffer stride. And those are just the ones that have ended up on hardware which crosses my desk, which means I haven't even tested the majority of consumer-grade hardware with UEFI.

The problems with UEFI have very little to do with its design or the ability of the people implementing it. After a few years of iterative improvements it stands a good chance of being more reliable and useful than BIOS. Increased commonality of code between vendors is a blessing and a curse - in the long term it means that these bugs can be shaken out, but in the short term it means that at least one hardware-destroying bug has ended up carried by multiple vendors. Right now we're still in the short term and it's likely that we'll find yet more UEFI bugs that cause all sorts of problems. The next few years will probably be a bumpy ride.

comments

Adam sent a new call for testing for the next point release of Debian Squeeze. Please test the packages in squeeze-proposed-updates on some machines running squeeze if possible, so that we don't screw up your production machines with bad updates in a week. The point release is scheduled for January 28th, i.e. next Saturday. Don't forget to copy the debian-release mailing list when you encounter regressions. Thanks for your efforts.

If you want to receive these notices by mail, please subscribe to the debian-stable-announce mailing list.

For my day job I build storage systems. A lot of what I do at present involves caring a lot about how different OSes deal with things like new LUNs being presented from a SCSI target, or errors along a subset of the available paths to a device. It will come as no surprise to you to discover that they all suck (for values of all equal to Linux, Solaris, Windows and VMWare). New LUNs are particularly annoying. I'm in the situation that creation and removal of a LUN is exceptionally easy. Hmmm. Maybe I need to back up here a bit first. SCSI has the concept of a target (think, device, eg hard drive). Each target can present multiple logical units. Each of these is assigned a number - a Logical Unit Number. Most devices (a hard drive, or a CDROM drive) will present a single LUN. A storage array will tend to present multiple LUNs; one for each volume that is exported to the host. At the host level each LUN really just looks like a separate device (for Linux /dev/sda and /dev/sdb may well be separate LUNs on the same array, rather than 2 separate arrays/hard drives, for example. At the block device level you don't care about the difference usually). Anyway. For various reasons I end up adding and removing LUNs quite often. And there are ways for the array to indicate that this has happened to the host (the UNIT ATTENTION/REPORT LUNS DATA CHANGED check condition seems to be favoured these days, as a complete Fibre Channel LIP can be disruptive). What I'd like to happen in that case is the host to pick up the check condition and drop and/or add the devices that have changed. Instead everything wants a manual rescan. rescan-scsi-bus tends to be simplest for Linux. Windows wants a manual refresh in Disk Administrator. VMWare a "Rescan HBAs" from vSphere. Solaris a "devfsadm -C" and possibly a "cfgadm -al" first. And all of these can be temperamental about picking up the changes. We've done a lot about hotplug for the desktop user experience, without doing the same level for the server experience. I appreciate that there are situations that you don't want your server to reconfigure things without being told to, but the current situation can be detrimental (for example Linux multipathing will hold a device open even after it's disappeared and is returning an "INVALID LUN" response; it would be much better if it could cleanly close that device and wait for it to return). Storage is capable of being much more than just a single block device these days, and it's a shame that nothing seems to deal fully with that fact. (Yes, yes, I should write and submit patches, but I appreciate that there's not always a simple answer, nor necessarily an answer that works for all situations automatically. Plus, y'know, not enough hours in the day and I hope you all appreciate I've taken a break from watching BSG to write this.)

Recently the upstream Linux kernel released a fix for a serious security vulnerability (CVE-2012-0056) without coordinating with Linux distributions, leaving a window of vulnerability open for end users. Luckily:

• it is only a serious issue in 2.6.39 and later (e.g. Ubuntu 11.10 Oneiric)
• it is only local
• it requires execute access to a setuid program that generates output

Still, it s a cross-architecture local root escalation on most common installations. Don t stop reading just because you don t have a local user base attackers can use this to elevate privileges from your user, or from the web server s user, etc. Since there is now a nearly-complete walk-through, the urgency for fixing this is higher. While you re waiting for your distribution s kernel update, you can use systemtap to change your kernel s running behavior. RedHat suggested this, and here s how to do it in Debian and Ubuntu:

• Download the am I vulnerable? tool, either from RedHat (above), or a more correct version from Brad Spengler.
• Check if you re vulnerable:

  $ make correct_proc_mem_reproducer
  ...
  $ ./correct_proc_mem_reproducer
  vulnerable
  

• Install the kernel debugging symbols (this is big over 2G installed on Ubuntu) and systemtap:
  • Debian:

    # apt-get install -y systemtap linux-image-$(uname -r)-dbg
    

  • Ubuntu:
    • Add the debug package repository and key for your Ubuntu release:

      $ sudo apt-get install -y lsb-release
      $ echo "deb http://ddebs.ubuntu.com/ $(lsb_release -cs) main restricted universe multiverse"   \
            sudo tee -a /etc/apt/sources.list.d/ddebs.list
      $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ECDCAD72428D7C01
      $ sudo apt-get update
      

    • (This step does not work since the repository metadata isn t updating correctly at the moment see the next step for how to do this manually.) Install the debug symbols for the kernel and install systemtap:

      sudo apt-get install -y systemtap linux-image-$(uname -r)-dbgsym
      

    • (Manual version of the above, skip if the above works for you. Note that this has no integrity checking, etc.)

      $ sudo apt-get install -y systemtap dpkg-dev
      $ wget http://ddebs.ubuntu.com/pool/main/l/linux/$(dpkg -l linux-image-$(uname -r)   grep ^ii   awk ' print $2 "-dbgsym_" $3 '   tail -n1)_$(dpkg-architecture -qDEB_HOST_ARCH).ddeb
      $ sudo dpkg -i linux-image-$(uname -r)-dbgsym.ddeb
      

• Create a systemtap script to block the mem_write function, and install it:

  $ cat > proc-pid-mem.stp <<'EOM'
  probe kernel.function("mem_write@fs/proc/base.c").call  
          $count = 0
   
  EOM
  $ sudo stap -Fg proc-pid-mem.stp
  

• Check that you re no longer vulnerable (until the next reboot):

  $ ./correct_proc_mem_reproducer
  not vulnerable
  

In this case, the systemtap script is changing the argument containing the size of the write to zero bytes ($count = 0), which effectively closes this vulnerability. UPDATE: here s a systemtap script from Soren that doesn t require the full debug symbols. Sneaky, put can be rather slow since it hooks all writes in the system. :)

2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

here's the list of RC bugs I've worked on during the last week:

• #590822 python-jaxml: "python-jaxml: cannot be bytecompiled with Python 2.7: SyntaxError: cannot assign to __debug__"
  sponsor (modified) NMU by Miguel Landaeta, upload to DELAYED/2
• #592382 src:liblunar: "liblunar: hardcoded dependency on python-support"
  use $ python:Depends in Depends [Jakub Wilk], upload to DELAYED/2
• #615765 pornview: "pornview: ftbfs with gold or ld --no-add-needed"
  add patch to pass -lm to linker, upload to DELAYED/2
• #641971 gsnmp: "Fix FTBFS with moved glib-2.0 headers."
  add patch from Ubuntu / Matthias Klose, upload to DELAYED/2
• #643050 src:autodir: "autodir: FTBFS: dpkg-buildpackage: error: dpkg-source -b autodir-0.99.9 gave error exit status 2"
  change config. sub,guess handling to a "3.0 (quilt)" compatible way, upload to DELAYED/2
• #648384 buildbot: "buildbot: FTBFS due to missing build-dependency on python-twisted-words"
  add patch from Ubuntu / Daniel T Chen, upload to DELAYED/2
• #653953 magicmaze: "magicmaze searches for Isabella.ttf in wrong place, crashes."
  add patch to search .ttf file in new location, upload to DELAYED/2
• #654257 src:newlib: "newlib: FTBFS: cannot find the library sys/linux/liblinux.la' or unhandled argument sys/linux/liblinux.la'"
  add patch from Ubuntu / Wookey & Steve Langasek, send to BTS
• #654911 python-lunar: "python-lunar: missing dependency on python-gtk2"
  add python-gtk2 to Depends [Jakub Wilk], upload to DELAYED/2

Somebody bought The Crying Tree by Naseem Rakha for Sarah as a present and she asked me to read it so we could talk about it, so I did. It's a novel about a family tragedy, Southern-state U.S.A. and the death penalty. A little boy is shot dead, a man is arrested for the crime and faces the death penalty. The book chronicles what happens to the family and to the killer during the 19 years or so that passes before the execution takes place. The book could be accused of having an anti-capital punishment agenda, but I have no problem with that. At least until about half-way through the book, the plot and happenings follow the synopsis with little in the way of surprise, but then there's a twist. I felt that the twist diminished the message of the book. Without trying to spoil what happens, I felt that you didn't need the core details of the story to be varied or expanded upon or U-turned to push that agenda. In my opinion, it's perfectly possible for someone to construct a story about a killer who faces death, who was unambiguously guilty of the crime they are charged with, and still leave the reader feeling remorse for the killer, and questioning the justice meted out. In fact I thought Rakha was doing a fine job of just that. However, a twist (or twists) occur, and the facts are called into question, and so you ask even further questions about the practise of killing criminals, in my view redundantly. It's a good book and worth a read, regardless of what your position is on the issue. It does a great job of describing the unbearable pain of losing a child, but also of the power of forgiveness for the victims.

Looks like installing iceweasel backport packages from

deb http://mozilla.debian.net squeeze-backports iceweasel-release

causes installation of backport of libcairo2 1.10 which breaks printing from GIMP, as described in this ubuntu bug. Does anybody know a fix/workaround? Update: looks like running gimp with LD_LIBRARY_PATH set to directory containing library files from squeeze libcairo2 package (1.8.10-6) workarounds this. But this story makes me think again that bundling libraries with applications is not that bad idea at least for users who need to get things done.

Bugs I've hit in GNOME 3, in the last 5 minutes:

• gnome-shell's "run" pop-up lost focus and I couldn't get rid of it. Nor could I launch another "run" dialogue, which is my normal route for starting software.
• I tried to coax my web browser into launching a terminal by browsing to file:///usr/bin/gnome-terminal. Instead, it downloaded a copy to ~/Downloads. Fine, no problem, because after another click it launches nautilus to handle the file. (This is not a bug).
• nautilus doesn't know what to do with the file. Apparently I haven't installed a handler for viewing executable files. I'm guessing the copy of gnome-shell isn't +x. nautilus does sensible things when another file (such as a picture) is marked +x, so why doesn't it do so for the opposite case?
• This is such a bizarre error message I try to take a screenshot of it. I take one, then take another (cropped). I try to save the second picture over the first one, but gnome-screenshot complains that it cannot find a file somewhere in /tmp.
• The flash plugin crashed in the middle of streaming an album. I'm almost not allowed to complain about the flash plugin crashing.
• After and around and inbetween all that, I downloaded an .amz file (an Amazon MP3 download file, in fact, of the album I'm failing to stream) which I hoped to run through clamz (a command-line, open source tool that can interpret AMZ files). nautilus offers clamz as a launch option in the right-click menu, but it's not the default handler. If I select "properties" to set the default handler, clamz is not in the list of available handlers.
• After sending the HUP signal to gnome-shell, I've got rid of the "run" pop-up, but now something has happened to it's $PATH and future "run" pop-ups can't find anything in the standard locations, like /usr/bin.
• since the flash plugin died, nothing else seems to be able to make any sound, so I can't play the album I downloaded anyway.

I didn't set out to bug-hunt this morning, I'm actually trying to get some work done. Whilst this is a particularly bad case of stacked-bugs, it's unfortunately, pretty representative of what it's like to try and use a Linux desktop at the moment. Every day I go through a similar experience. I've just had to give up and log out and back in again. The shell path bug above has re-surfaced in a new session. Great. I'm waiting on a delayed laptop at work. I'm very tempted to cancel the order and just use a Mac Air instead (we have one spare). It seems I'm walking the inevitable path of F/OSS desktop frustration and the end for many people seems to be the Mac perhaps I should spare myself the pain of the journey. In the mean time, It's bye-bye to GNOME and off to Xfce for me.

Today, six months have passed since the massacre I survived at Ut ya. It strangely feels simultaneously as if it happened ages ago, and as if it happened yesterday. I will write a longer post at some point, to try to sum up some thoughts, feelings and reflections I have made in the aftermath. But this post is actually intended to be somewhat, albeit weirdly, cheerful because today, of all dates, I was given a link to a web album by the Police where they post photographs of items they have recovered and cleared for re-release. And I got a kick out of this. Check it out!

I m getting my OpenVMS shirt back! Yay for conference T-shirts!

Something I've picked up from reading about economics is the concept of "the margin". It's a way of thinking about problems that more people ought to take into consideration. What is "the margin"? It's that space on a line, in the middle, between two extremes, where the transition from "yes" to "no" occurs. If I offered you a million dollars for the computer you're reading this on (for broad definitions of 'computing device'), you'd probably take me up on the offer. For 0 dollars, you would not. Somewhere, in the middle, is a number where you'd change your mind from "nope, won't sell" to "well... sure, what the heck". That is, loosly defined, a margin. As an example, when people debate about "intellectual property", they often use terrible examples: companies like Microsoft, or performers such as Lady Gaga. Those are bad examples because they are complete outliers, way off on one end of the curve. It's hard to disagree with "so what if Lady Gaga earns a bit less revenue from her music, she's got plenty to live on" when you talk about copyright being a means for artists to support themselves with. Thinking "at the margin" is about those bands that currently barely sell enough music to work professionally as musicians. In scenario A, they are able to work creating music, thus creating more, and likely better music than if they merely pursued it as a hobby. In scenario B, they fall on the other side of the margin and therefore have to get 'real jobs'. This means that their music takes a back seat, and they produce less of it. Now, copyright and company are a complex conundrum with many facets; my point is simply that when thinking about big changes, we should think what will happen at the margin, not what will happen to the outliers.

PHP 5.4.0 is around the corner, with RC6 released this weekend. With the courtesy of Ond ej Sur it s already available in experimental. Earlier this week, Raphael Geissert, tested some PHP extensions with the new version and reported 16 bugs (severity: important) against those who failed to build with PHP 5.4. As they will become RC bugs when 5.4.0 will enter unstable (probably around mid February), I preferred to handle them sooner than later. The result is 3 patches sent (one already uploaded) and 5 NMUs done. During the process I poked through a lot of upstream SCMs. In the way I found out a trivial change was done in PECL s SVN two years ago for all the extensions located there, which got me suspicious regarding some extensions we have. For non PECL hosted extensions, I had to track if and when a fix was done and use it for building or do a trivial fix myself. Packages using the 3.0 (quilt) source format were extremely easy to fix, and I was quite happy it generally went fast. In general I m a passive subscriber to the Debian PHP Maintainers mailing list, and finally could help actually and not just reply emails. For me it s also doing some Debian work after a few months of focusing on the coming LibreOffice 3.5 release. update [26/1/2012]: From the 16 issues, only 2 aren t fixed already or will be on the next upstream release. As their upstreams are dead, this is in the hands or the debian people, at least till they ll FTBFS on unstable (instead of experimental).
Filed under: Debian GNU/Linux, PHP

Hard to believe I've consumed all of 1981's Usenet posts now on olduse.net, and it's been running for 7 months already.

---

Last night, there was a "very long" post, describing nearly every node on usenet in 1982. There had been a warning about this post the day before, since it would take many sites half an hour to download at 300 baud. It was handily formatted as a shell script, which created per-node files. So, I ran this code nobody has run since 1982. It worked. I got files. I tossed them on the olduse.net wiki, and used some ikiwiki code TOVA contracted me to write just a few months ago, to make clickable links on my usenet map. The map data was contributed in another post a while back. By 1982, usenet is getting nearly impossible to map with 1982 technology of ascii art. I enjoyed throwing graphviz, git, wikis, and the web at it. So, we have a collaboration across time, me and "Mark" and a lot of people who described their usenet nodes and piles of technology that make creating a mashup easy. Awesome!

---

I blog about stuff I find on the olduse.net blog. It's an open blog; Koldfront also blogs there, and we welcome other bloggers. Some of the highlights for me have included: As the space shuttle program is winding down, reading the excitement about the first shuttle flights, and the play-by-play coverage of a launch, posted to net.columbia by a high school student borrowing his dad's account. (A usegroup name that's hard to read without remembering its fate). The announcements of the Motorola M68k, the IBM PC, and the CD-ROM. Reading the TCP-IP digest, and Postel's plans for launching IPv4 soon, while the world IPv6 launch is being planned now. (The nay-sayers are especially fun to read. Including the guy who was concerned about the address space size, in 1981!) Learning that nethack ascention tales have a history streching back 30 years, to rogue, and that the stories back then had much the same flavor as they do today. Various celebrity sightings. Dennis Ritchie teaching C and Unix. Bill Joy talking vi. RMS talking .. nuclear politics? The general development of usenet. B-news being rolled out, groups proliferating, many first inklings of what will be major problems and developments in 5 or 10 years. A shift in tone is already apparent, by now usenet is not only about announcements, there are already some flames. Still 9 years to go!

I really should write these a bit more often. Wow, I can't believe it was over 4 years ago that I started having occasional face to face meetings with the ISC DHCP folks. The entire ISC DHCP team (of 5) was in town for an all-hands meeting, and Larissa Shapiro, the Product Manager for DHCP (and BIND) suggested it would be a good opportunity for another catch up. Given the current (bad) state of DHCP 4.2 in unstable, I thought this was an excellent idea, and so we all had lunch on Tuesday. I pretty much set the agenda, and it was

• general state of 4.2.2 in Debian
• situation with GNU/Hurd and their patch to fix an FTBFS (#616290)
• the current FTBFS issues with kFreeBSD (#643569)
• the embedded BIND sources in the DHCP source
• removal of the RFCs from the embedded BIND source (#645760)

The general state of 4.2.2 in Debian In a nutshell, it's a bit of a mess. We've got release critical bugs, build failures, the whole cat and kaboodle. It makes me very sad, because 4.2.2 was the first 4.2 series that I had a chance to upload, and I was very excited to do so, because it contains the hotly desired LDAP patches merged upstream. Unfortunately, it's also got the beginnings of the BIND/DHCP merger that's going to be BIND 10, and that is all a bit of a mess. It's directly responsible for the kFreeBSD FTBFS and the introduction of the RFCs, which are both keeping 4.2.2 out of testing. I gave the ISC folks a high-level overview of how Debian development works, and the normal progression of packages from unstable to testing to stable, and the release process and whatnot, and impressed upon them the implications of the current release critical bugs. I also showed them how Ubuntu development fitted into the picture. Finally, I showed them the popcon statistics for DHCP. I think they found it useful. FTBFS issues on kFreeBSD This was a good segue to #643569. The issue is actually with the embedded BIND sources. I'd already forwarded this bug upstream when it first happened, but I don't know what had happened to it. They seemed to act as if this was the first they'd heard of it. I'm hoping that they can get this fixed in 4.2.3, which is due around the end of the quarter. Embedded BIND sources Since we were already talking about an issue caused by the embedded BIND sources, we moved on to talking about #645760 and the existence of the embedded BIND sources in general. It should be pretty straightforward for them to strip the RFCs out of the source. They've already done it in the past for the DHCP sources, so I'm also hopeful that this will get resolved in 4.2.3. The issue of the embedded BIND sources is apparently a bit more complicated, although the day before our meeting, Michael Gilbert filed #643569 and #645760, so I hope that the ISC folks can take a look at these patches and see if it's feasible to adopt them. Patches for GNU/Hurd Finally we talked about #616290, which I know is near and dear to the GNU/Hurd porters' hearts. We probably spent the most time talking about this. The DHCP developers have concerns about accepting a patch for an OS that they do absolutely no testing on, and also questioned the viability of the OS in general. They stressed that they're fairly thin in numbers relative to what they have on their plate to achieve this year, and so pushed back pretty firmly on accepting the current patch. I relayed the frustration that the Hurd folks were having about a lack of dialogue around the patch (most of the interaction has been via an ISC support person). There was actually a bit of a split between the developers, with one of them appreciating that the Hurd was unlikely to go anywhere as a platform without a working DHCP client, so in some regards, they were condemning the platform by taking the position they were taking. They're going to go away and take another look at the patch and try to come back with some actionable feedback on what needs to change to make it more acceptable to them, so we'll see what comes of this. I'm not particularly optimistic that anything acceptable to the GNU/Hurd folks is likely to happen any time soon, but maybe if the patch gets cleaned up a bit more, I'll just bite the bullet and start applying it to the Debian package. BIND 10 One of the guys is more involved in BIND 10 than DHCP, and asked if I could help out with the packaging of a build dependency for BIND 10. It seemed like #578387 was languishing so I offered to pick it up. I've not packaged a library before, mainly because the library packaging guide has scared me off it (I feel I lack the deep C fu that seems necessary), but I figured that this would be a good learning opportunity, so I'm going to dive in.

So the site http://megaupload.com/ has been taken offline, amidst allegations of knowingly conducting in piracy. There are probably a lot of legitimate users who have lost access to their uploaded files, even if they were offsite backups you can imagine a user owning a website which now has a million dead-links. This reminds me of a conversation I overheard on Jon Dowlands blog - the summary is that he'd written a (useful) tool to extract attachments from Maildir folders and was wondering how to store and access those attachments. The upshot seemed to be magical URLs of the form:

• https://file.example.com/sha1/509c2fe2eba509e93987c3024a74d74583c274bd

The comments covered an alternative which was hash:///sha1/xxxxxxxxxxxxxxxx, which then becomes close to the magnet:// schema. I've not yet thought things through, but I can't help thinking that with the redundency already present in the internet we should be looking at non-server-specific links. Yes there are times right now when you might want to address a specific file on a specific server - but otherwise? Wouldn't it be nice if you could just access a file from "anywhere" which happened to have the right contents? Already my nonporn-but-definitely-adult-site makes its images available as /img/$md5sum.jpg - and similarly the storage at the back-end of my random image upload site uses SHA1 hashes to store the actual files. To make this more complete what we need is something that crawls the internet to find files by hash; then add support in browsers. Obviously this must be async and could introduce timing issues, but fundamentally it seems like a reasonable approach to the problem of a single host going offline. (Consider what happens if imgur.com disappears. All those links would die, yet 99% of the images would still be available somewhere.) I'm tempted to suggest microformat format but I need to consider the matter. Right now I'm going to immediately update my current image hosts to use, at the very least:

 <a href="/foo" rel="sha1:xxxxx md5sum:xxxx">
  <img src="foo.jpg" alt="img name">
 </a>

The unfortunate thing is you cannot have a 'rel="xx"' attribute for an image. So you either have to encode it in the parent link, or add it to the alt attribute which is suboptimal. ObQuote: "Now, they tell me I paid my debt to society." - Oceans Eleven (2001)

For the sake of anyone else who s fighting their screens, I ve got a script that will do this (but very very slowly) setup script

xrandr --output VGA1  --auto
xrandr --output LVDS1 --auto
xrandr --output VGA1  --rotate left
eval " monconf "

monoconf script

#!/usr/bin/env python
import subprocess
output = subprocess.check_output(["/home/tag/.bin/mon.ls"])
output = [ item.split() for item in output.split("\n") ]
mons =  
for mon in output:
    if len(mon) > 0:
        if mon[1] == "connected":
            res=mon[2]
            res=res.split("x")
            for item in res[1].split("+"):
                res.append(item)
            res.pop(1)
            mons[mon[0]] = res
BOTTOM_LEFT = "LVDS1"
TOP_RIGHT="VGA1"
LEFT_WIDTH  = mons[BOTTOM_LEFT][0]
LEFT_HEIGHT = mons[BOTTOM_LEFT][1]
RIGHT_WIDTH  = mons[TOP_RIGHT][0]
RIGHT_HEIGHT = mons[TOP_RIGHT][1]
print "xrandr --output %s --pos 0x%s" % (
    BOTTOM_LEFT,
    (int(RIGHT_HEIGHT) - int(LEFT_HEIGHT)))
print "xrandr --output %s --pos %sx0" % (
    TOP_RIGHT,
    int(LEFT_WIDTH)
)

and the mon.ls command

xrandr -q   grep "^[^\ ].*"   grep -v Screen

All of this is super hacky, but it works.

There s been a lot of weather here in Seattle this week. It looks like the Tukwila DC s power blipped this morning. It gave me an excuse to update some configs on a few of the OpenVPN endpoints, re-configure some of the shorewall6 stuff, and enter a bunch of hosts and IPs into the BIG-IP s node list. It gives me a quick way to know what needs fixing first when things start breaking and is a lot easier to configure than NAGIOS. If I really wanted to get fancy, I could even write me some hot external monitoring scripts in perl. I am growing fond of this machine.

Kim Jong-Il is gone. That said, he continues to live on, looking at things, on the popular blog Kim Jong-Il Looking At Things which continues to be updated with new content from the archives. It is now joined by Kim Jong-Un Looking At Things. I think I agree with Jo o Rocha, creator of the original, that the younger Kim seems to be missing some hard-to-pin-down quality that made the original work well.

Debtags for derivative distributions Sometimes I do cool stuff and I forget to announce it. Ok, so I recently announced a new Debtags website. I forgot to say in the announcement that the new website does not only know of Debian packages: see for example this page, at the very bottom it says: "Distributions: oneiric, precise, sid, testing". This means that already, here and now, debtags.debian.net can be used to tag packages from both Debian and Ubuntu, and can easily be extended to cover the entire Debian ecosystem. If you are a package maintainer, you will notice that your maintainer page shows your packages from everywhere. If you want to filter things a bit, for example hide obsolete packages from an old Debian Stable or Ubuntu LTS, just click on the "Settings" link on the top right to configure the page. How it works The magic is in this mergepackages script, which is run daily, and exports merged Packages files at dde.debian.net. The debtags.debian.net concept of Packages and Sources files are just those all-merged.gz and all-merged-sources.gz. The merging is simple: that rebuild script processes files in order, and the first version of a package that is found is chosen as the base for the one that will go in the merged Packages file. Some fields like "Description" are just taken from this pivot package, others like Architecture or dependencies are merged into it. It's arbitrary, but works for me: the result has all the packages with all their possible architectures and dependencies, and is ready to be indexed with apt-xapian-index. At the moment I pull data from Debian and Ubuntu, but you can see that the script can easily be extended to pull data from any Debian-style ftp archive, so any Debian derivative can go in. I've already started negotiations with the Derivatives Census on how to add any Debian derivative and keep the list up to date. How to export tags for your own distribution I'll use Ubuntu as an example since the data is already available. The way you add Debtags to the Ubuntu packages file is just this one:

1. Get the full reviewed tag database
2. Optionally filter out those packages that you are not interested in
3. Tweak this script to build an overrides file.
4. Give the overrides file to your favourite ftp archive building tool.

The make-overrides is a bit rusty: if you improve it, please send me your changes. That is it, nothing else required, no excuses, it's ready, here, now! Hitches and gotchas This merged Packages file is a bit of a hack, and suffers from name conflicts across distributions, where two different softwares are packaged in two different distributions with the same name. Ideally, name conflicts should not happen: if a derivative decided to package kate and call it gedit, they deserve to have it tagged uitoolkit::gtk. I think it's rather important that the whole Debian ecosystem works as much as possible with a single package namespace. However, that reasoning fails if you take time into account: packages get renamed, like git and chromium, and may mean completely different things, for example, if you compare Debian Stable with Debian Sid. This last is a problem caused by debtags only working with package names but not package versions. I have a strategy in mind based on being able to override the stable tag database using headers in debian/control; it still needs some details sorted out, but I'm confident we will be able to address these issues properly soon enough. Why stop at the Debian ecosystem? Why indeed. I'm clearly trying to use FOSDEM, and the CrossDistribution devroom as the venue to discuss just that.