This is just a diary of a random day sort of post. Nothing all that particularly unusual, but then perhaps if I only blog about the unusual, the typical gets lost. This week started out with temperatures as high at 99F. Today we didn t even reach 60, and tonight will be cold almost to freezing. It s Kansas, so while we remark on the strange weather, we re not particularly surprised by it. This morning, Jacob and Oliver went over to play across the street. Jacob then went with Terah to a birthday party for one of his classmates, while Oliver helped plant some trees with his grandparents. I was at home this morning, which gave me a small but welcome break from the past week: we had something going on 6 out of the last 7 evenings, and little time to do anything else. So I got out the Kindle and read from War and Peace. I started at the kitchen table while eating some granola, and then sat on the couch. The day was cold, windy, and overcast, and I like to use natural light, so there were only a few places to sit with enough light. I sat there reading the account of Napoleon s capture of Moscow in 1812, the characters lives uprooted by those events, and stories of violent mobs forming. And, of course, Tolstoy s essays on the mistake of overstating the importance of famous figures in history were common. I am now about 3/4 through War and Peace, which means I ve read 1,044 pages and have (only) 348 to go. As I got up to try to find slippers or a blanket for my cold feet at one point, I remembered a conversation I had while eating lunch this week. I was reading on the Kindle while reading, and a person I knew happened to walk past. She asked what I was reading. I said simply, War and Peace. There was this stunned silence, so I added, for fun. It s great! Which is true. She said, Oooookay, John, you just enjoy that then and went on her way. This isn t the first time I ve had a conversation like that about this book. It seems to be held up as some example of long, dry, literature that people are made to read but don t want to. I think that only long is accurate here. Anyhow, pretty soon Oliver arrived and, much to my surprise, said Doh! (meaning no ) when I asked him if he was ready for lunch. Instead he stood there looking at me sweetly. Taking a guess, I said, Oliver, would you like to watch train videos? Yeah yeah yeah yeah! So we did. He particularly enjoyed the steam engines today, and eventually was ready for lunch at 12:30. Jacob and Terah got home just as he was finishing. Oliver got his nap, and I played with Jacob for a bit. Ever since he had heard of the British wedding in preschool, he s been interested in weddings, so I showed him pictures from Terah and me at our wedding. This was a big hit. Then he went with Terah to decorate a Sunday School room at church, so I read a bit more Tolstoy. Tolstoy s characters sometimes are prone to not just changes in mood but changes in outlook and worldview changing from cold, logical, and uncaring to warm and loving, or from despair at the vanity and shallowness of humanity to elation at the simple honesty of a peasant and renewed trust in the human spirit. At first I marveled at this tendency in certain characters, but on reflection, and in consideration of the timeline of the novel, which spans several years, it didn t seem quite so strange. Things can happen to change the mood of individuals, or even a country; I can remember the national mood changing after 9/11, and again after Obama s election, for instance. I also felt a contrast with some of the characters who, due to their decisions or circumstances, lost things that mattered while chasing things that weren t as important. I felt fortunate to be at this place, with Jacob, Oliver, and Terah, and decided that we all ought to do something fun in the evening, but hadn t quite figured out what. And about that, the clouds started to break and Oliver woke up from his nap. I went up to his room, and much to his delight, played peekaboo with him for quite awhile. Eventually he was done with that game, but spotted my phone, so I helped guide his finger to launch Youtube and watch a train video over the wifi. This was certainly a Big Deal to him. After Terah and Jacob got back, I figured it was time figure out something fun to do. And what could be more fun, I decided on the spur of the moment, than a drive into town to pick up some cans of Silly String for the boys, neither of which had ever used it before. So we went to a restaurant, then bought the surprise toy (I hadn t told them what it would be beforehand), then went to an ice cream place, and finally back home. Oliver loved it. He needed my help to press the nozzle to make it come out, but still had a great time and shrieked with delight. Jacob then tried out his can, and stood in one place spraying his bright green strings all over the place. Then he took to running around the yard spraying string randomly in every direction until he finally completely emptied out the can. And after that, he gathered up some of the different colors of string and rolled them up into a ball which he was very proud of. A nice day.

There is actually some buzz around Gnome 3 and the new release of Ubuntu switching to Unity. I thought this was a good occasion to make this small post to tell that I still uses FVWM whose latest version was released a month ago. FVWM is a very old window manager. So old that nobody really knows what the F stands for. It comes with a ugly configuration by default and can be configured to look even uglier. However, FVWM is powerful and highly configurable. Moreover, it is pretty lightweight.

My FVWM configuration My configuration of FVWM is available on GitHub. It requires the use of fvwm-crystal. If you are running Debian or Ubuntu, there is a package for this. This is a set of powerful configuration files for FVWM which add a lot of goodies and themes. Unfortunately, the project site seems to have disappared. You can still grab the sources from your nearest Debian mirror. While those configuration files were great, I did not like to have such a complex desktop with all those icons and menus. However, what is great with fvwm-crystal is you can overlay your own configuration. If a file is present in your home directory, it will take precedence over the one installed on the system. So, I just wrote a new recipe. What you may want to steal from my configuration is the ability to alter the configuration depending on the fact that you are using one or two screens. With one screen, you get:

• an application panel (a launcher bar) on the upper left corner,
• an icon manager (which will contained iconified windows) on the upper right corner,
• a systray on the left of the icon manager
• a pager (which shows various desktops) just below it,
• two icon managers (which act as a taskbar) on the bottom, one for regular applications and the other one for terminal windows.

With two screens, the organization is different. The two icon managers at the bottom are on their own screens. The pager is between the two screens and the other elements are on the upper left of the right screen. If I am using a single screen and I plug another one, I do something like this to get my setup updated:

1 2	$ xrandr --output VGA-1 --auto --output DVI-I-1 --auto --right-of VGA-1 $ killall trayer ; FvwmCommand Restart

I also uses conky, a light-weight system monitor. It is started in my .xsession, not through FVWM.

Building wallpapers for dual-screen One difficulty with a multi-head setup is how to get the background set properly? . I have made a small script for generating one. Use it like this:

1	$ ./build-wallpaper.py -d ~/.fvwm/wallpapers -c -t ~/tmp/mywallpaper.png

It will randomly select two wallpapers and put them together to look nice on your dual-screen setup. It will also correct the aspect ratio, either by cropping the image or centering the image. You need the following dependencies to make script work: python-xpyb and python-imaging. UPDATE: Thomas Adam pointed to Nitrogen, a graphical wallpaper utility supporting multi-head setups. It is a great tool if you want to set manually your wallpaper but it does not allow to select a random one. Nowadays, I grab most of my wallpapers from InterfaceLIFT.

You know, GNOME Shell and I are not really strangers to each other for a long time now. I have been using it almost daily as my main desktop since late 2009, when I started shipping it to Debian experimental. That means that I had ample opportunity to both get used to it and witness the huge improvements it had with every new release. My general feeling towards GNOME 3 is this: . Yes, I love it! I love the new themes, I love the window borders, I love the top panel, the overview, the dynamic workspaces, I love the Me menu, I love the clock and the calendar, the system indicators with the beautiful symbolic icons, being able to search for applications in such a nice way, the window animations, the multi-screen support, the new nautilus, the dash, looking glass. It s hard for me to even express how thankful I am and how much admiration I have for the awesome folks who helped bring this to life. Thanks so much! After all this time, there are only two things I can say I dislike about GNOME3, apart from some minor wishlists: the alt-tab behaviour and the message tray. Let me expand on those. Message Tray Of the very few things I dislike, there is only 1 I hate and cannot see myself living with: the accordion animation in the message tray. No, really, it s such a terrible, terrible idea. Every single time I try to use that thing I overshoot while moving the mouse to the left, then overshoot again moving the mouse to the right because the frigging icon has moved. It s no good knowing that I can click in the text or in the empty area to its right, it feels wrong. It s terrible that my actual target is moving at all. Every single time I use it is a small frustration for me it s as if the message tray was playing games with me, laughing at me for not having good enough mouse pointer driving skills even more than the infamous sub-menus used to. And I had to go through that penance whenever I wanted to find the person I was chating with to resume the conversation. I wrote an extension that disables the accordion animation by simply not showing the title at all when you hover the icon, and I patched GNOME Shell s CSS to make the icons a bit bigger, so that it s easier for me to hit them with the mouse. It s clearly not ideal, and you still have to click the various people icons to figure out which of your friends who were lazy enough to not add a picture to their IM profiles is that one, but it s still much better than chasing the (smaller) ones around to figure that out. Perhaps we should have the icons be bigger and always have the title bellow them? I don t know, I trust the awesome designers who designed the awesomess that s everywhere else will come up with a great idea. Alt Tab The number 1 feature of workspaces for me has always been locality being able to not see all of the other applications and windows that are open elsewhere. This lowers the amount of noise when I m trying to find something. The overview is very nice in this matter only windows in the current workspace are shown, and even when you have an extra screen, the windows in there appear in that screen. The alt-tab behaviour, on the other hand, of showing all windows and apps, even with the separator, bothers me. It s really useful to have when you want to go to a specific window no matter where it is (I usually use the dash for that, though), but it adds noise when you want to go to a specific window _in this_ workspace, which is the most common use case for my usage. So I copied the alt-tab code over to an extension and modified it so that only windows in the current workspace would be considered. In addition to that, with the windows in the extra screen always being there no matter what workspace you re in (which I think is an awesome idea), they are effectively in all workspaces, so they add constant noise even with my extension. It s also weird to have to look at the main screen to switch to a window in the extra screen. There s a huge discontinuity. What I would prefer is having alt-tab to follow the mouse regarding screens only show windows in the extra screen if you hit alt-tab in there, making sure the selector thing appears in there as well.

From the java point of view Recently I had to get some Scala Tool working correctly. Unfortunately there are basically no packages in the Debian Archive at all so I had to use maven to install these (or download + install manually). Being a highly paranoid person downloading and executing code from the internet without any cryptographic verification at all one after the other practically drove me nuts. Looking a bit deeper I noticed that some of the software in maven's repository have some signatures next to them -- signed by the author or release manager of this specific project. Why secure sources matters With my experience in mind I got some Input from other people. One of the things I was told is that some scala tools just aren't security critical -- they're only installed and used as the current user. In my opinion this is, for my desktop system, totally wrong. The important things on my private Computers are my GPG and SSH keys as well as my private data. For messing with these no super user access is needed at all. Comparing to the Common Lisp situation Being a Common Lisp fan of course I noticed basically the same problem for installing Common Lisp libraries. Here the situation in Debian is quite a bit better -- and I'm working in the pkg-common-lisp Team to improve this even more. Common Lisp has some maven-alike tool for downloading and installing dependency trees called quicklisp -- without any cryptographic verification as well. However there's light at the end of this tunnel: There are plans to add GPG verification of the package lists really soon. Comparing the maven and the quicklisp model So there are basically two different approaches to be seen here. In maven the software author confirms with his signature the integrity of his software while in quicklisp the distributor confirms all users get the same software that he downloaded. Now the quicklisp author can't and won't check all the software that is downloadable using quicklisp. This won't be doable anyway as there's way to much software or a single person to check. Now in some kind of perfect World the maven way would be vastly superior as there's a End-To-End verification and verification of the full way the software takes. However there's a big problem: I don't know any of these Authors personally and there's no reason I should just trust any of them. Now comparing this to the Distribution / quicklisp model. Here I would just have to trust one person or group -- here the quicklisp team -- to benefit from the crypto which might be possible based on karma inside the using community. However here I don't gain the possibility that the software is integer. However idealized if some of these pieces of software was forged between upstream and the quicklisp team and attacker would also intercept me downloading the software from the same address so I get the source from upstream matching the checksum from quicklisp -- assuming the quicklisp team does indeed know the correct website. Additionally I get the confirmation that all other quicklisp users get the same source (if the quicklisp guys are fine of course) so no-one inside the community complaining is a good indication the software is fine. For this to work there's of course a relevant user-base of the distributor (quicklisp) necessary. Relevance for Debian So how do conventional Linux Distributions like Debian fit in here. Ideally we would have maintainers understanding and checking the software and confirming the integrity using their private key or at least know their upstreams and having at least a secured way getting the software from upstream and a trust relationship with them. Of course that's just illusionary thinking of complex and important software (think libreoffice, gcc or firefox for example). Maintainers won't fully understand a lot simpler pieces of software. And loads of upstream projects don't provide a verified way of getting the correct source code though that's a bit better on the real high-impact projects where checksums signed by the Release Manager are more common than in small projects. A misguided thought at the end As I'm a heavy emacs user I like to have snapshots from current emacs development available. Fortunately binary packages with this are available from a Debian guy I tend to trust who is also involved upstream so adding the key from his repository to the keyring apt trusts. Now my first thoughts were along the lines "It would be really nice if I could pin that key to only the emacs snapshot packages" so this guy can't just put libc packages in his repository and my apt would trust them. Now thinking of it again a bogus upload of the emacs snapshot package could just as well put some binary or library on the system at some place in front of the real on in the system path which would be rather similar bad. b

Some months ago I wished for a tool to create virtual disk images, something similar to debootstrap except it would create a disk image instead of a directory. It would be used like this:

sudo ./vmdebootstrap --image test.img --size 1G \
--log test.log --mirror http://mirror.lan/debian/

debootstrap is wonderful for creating a basic Debian system for use with chroot. In the modern world, something similar but for virtual machines (such as KVM) would be nice to have, I thought. Particularly if it is something that is not tied to a particular virtualization technology, and does not require running in an actual virtual machine (for speed and simplicity). I could imagine using a tool like this for automatically building appliance-like system images, for example for FreedomBox. Being able to do that in a fully automatic manner would make development simpler and more easy for others to reproduce, and would allow for things like automatic integration testing: build image, boot it in a virtual machine, run automatic tests, report results. There are a bunch of tools that almost do this, but not quite. For example, live-build builds a disk image, but it's one that is aimed for live CDs and such. This brings in some limitations. By default, the image is not persistent. You can make a persistent image, but that requires you to provide a second partition (or disk) that holds the modified parts, using copy-on-write. It also means that if the bootloader or initramfs needs to be updated, then the read-only image needs to be updated. That's fine for live CDs, but not so good if you want a real installed system that you can upgrade to all future releases of Debian. (Ben Armstrong kindly helped me use live-build the right way. Thanks!) There's also grml-debootstrap which seems to work on a real system, but not installing onto a virtual disk image (and the difference is crucial to me). Based on feedback I received back then, and some experimentation I've done the past couple of days, I wrote a very rudimentary version of the tool I want. The code is on gitorius. (You need my cliapp library to run it.) Be warned: it really is a very rudimentary version. For example, it provides not flexibility in choice of partitioning, filesystem type, what packages get installed, etc. It does not configure networking, and the root password is disabled. Some of that flexibility would be quite easy to add. It also uses extlinux as the bootloader, since I utterly failed to get grub2 to install onto the virtual disk. (I did, however, manage to overwrite my develpment machine's bootloader.) The command above works, and results in a RAW format disk image that boots under KVM. It might boot off a hard disk or USB flashdrive too, but I haven't tried that. I now have several questions:

• Does something like this already exist?
• Does anyone else find this useful?
• Is another approach better? Perhaps running a fully preseeded debian-installer in a virtual machine might be better.
• If no other tool exists for the same job, what should I do next with vmdebootstrap?

After a bit of work, today I decided to start using mpm, the mancoosi package manager, to upgrade my laptop. My first use of it on a production system - until now I run all my experiments in throw-away virtual machines - and it works ! Not rocket science here. During the last month David Kalnischkies (of APT fame) visited our offices in Paris and together with zack worked out a communication protocol between apt-get and the mancoosi cudf solvers (EDSP). I guess somebody is going to announce all details about this endeavor soon. This cooperation enabled us to advance in the integration of apt-get and the mancoosi technology. Reusing the same protocol and backend I developed to translate the apt problem to cudf (and to call a suitable solver), I've re-wrote large part of mpm and added the possibility to generate the installation plan before calling dpkg and really installing the selected packages. Below notice the intermediate calls as Inject Model , Simulate, Compare Models . These are at the moment stubs that are going to call the simulation framework developed at mancoosi. The food : Reg the installation plan, this is an xml file that will be passed to a simulator developed by the university of L'Aquila to make sure that the installation script (well, a model of them), will not cause any problem during the installation. The format is a very simple xml file as follows : Soon APT will ship a patch to use the very same infrastructure of mpm (yeii !!!). This will on one hand make mpm useless as package manager on its own. It is a simple hack in python and I never tough to compete with its big brothers. On the other hand I think it will stand as a nice workbench to experiment with new ideas, to prototype new features and to make it easier for poeple that are not c++ experts to play with the APT library (thanks to python-apt) in a semi structured environment. The code is in the mancoosi svn if you want to have a look.

After a bit of work, today I decided to start using mpm, the mancoosi package manager, to upgrade my laptop. My first use of it on a production system - until now I run all my experiments in throw-away virtual machines - and it works ! Not rocket science here. During the last month David Kalnischkies (of APT fame) visited our offices in Paris and together with zack worked out a communication protocol between apt-get and the mancoosi cudf solvers (EDSP). I guess somebody is going to announce all details about this endeavor soon. This cooperation enabled us to advance in the integration of apt-get and the mancoosi technology. Reusing the same protocol, and backend I developed to translate the apt problem to cudf (and to call a suitable solver), I've re-wrote large part of mpm and added the possibility to generate the installation plan before calling dpkg and really installing the selected packages. Below notice the intermediate calls as Inject Model , Simulate, Compare Models . These are at the moment stubs that are going to call the simulation framework developed at mancoosi. The food : Reg the installation plan, this is an xml file that will be passed to a simulator developed by the university of L'Aquila to make sure that the installation script (well, a model of them), will not cause any problem during the installation. The format is a very simple xml file as follows : Soon APT will ship a patch to use the very same infrastructure of mpm (yeii !!!). This will on one hand make mpm useless as package manager on its own. It is a simple hack in python and I never tough to compete with its big brothers. On the other hand I think it will stand as a nice workbench to experiment with new ideas, to prototype new features and to make it easier for poeple that are not c++ experts to play with the APT library (thanks to python-apt) in a semi structured environment. The code is in the mancoosi svn if you want to have a look.

Executive summary: Don t use btrfs on Debian Squeeze.
Longer summary: Don t use btrfs RAID with the kernel Debian Squeeze comes with. About six months ago, I set up a new server to handle this web site, mail, and various other things. The system and most services (including web and mail) was set to use an MD RAID 1 array across two small partitions on two separate disks, and the remaining space was setup in three different btrfs file systems:

• One btrfs RAID 0 for shared data I wouldn t mind having offline while fixing issues on one disk
• One btrfs RAID 1 for shared data I would mind having offline while fixing issues on one disk
• One last btrfs RAID 0 for entirely throwable things such as build chroots

Three days ago, this happened:

May 10 10:18:04 goemon kernel: [3545898.548311] ata4: hard resetting link
May 10 10:18:04 goemon kernel: [3545898.867556] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
May 10 10:18:04 goemon kernel: [3545898.874973] ata4.00: configured for UDMA/33

followed by other ATA related messages, then, garbage such as:

May 10 10:18:07 goemon kernel: [3545901.28123] sd3000 d]SneKy:AotdCmad[urn][ecitr
May 10 10:18:07 goemon kernel: 4[550.821 ecio es aawt es ecitr i e)
May 10 10:18:07 goemon kernel: 6[550.824     20 00 00 00 00 00 00 00 <>3491225     16 44 <>3491216]s ::::[d]Ad es:N diinlsneifrain<>3491216]s ::::[d]C:Ra(0:2 00 03 80 06 0
3491217]edrqet / ro,dvsb etr2272
May 10 10:18:07 goemon kernel: 3[550.837 ad:sb:rshdln etr2252
May 10 10:18:07 goemon kernel: 6[551214]s ::::[d]Rsl:hsbt=I_Kdiebt=RVRSNE<>3491215]s ::::[d]SneKy:AotdCmad[urn][ecitr
May 10 10:18:07 goemon kernel: 4[550.833 ecitrsnedt ihsnedsrpos(nhx:<>3491216]    7 b0 00 00 c0 a8 00 00 0

Then later on:

May 10 12:01:18 goemon kernel: [3552089.226147] lost page write due to I/O error on sdb4
May 10 12:01:18 goemon kernel: [3552089.226312] lost page write due to I/O error on sdb4
May 10 12:10:14 goemon kernel: [3552624.625669] btrfs no csum found for inode 23642 start 0
May 10 12:10:14 goemon kernel: [3552624.625783] btrfs no csum found for inode 23642 start 4096
May 10 12:10:14 goemon kernel: [3552624.625884] btrfs no csum found for inode 23642 start 8192

etc. and more garbage. At that point, I wanted to shutdown the server, check the hardware, and reboot. Shutdown didn t want to proceed completely. Btrfs just froze on the sync happening during the shutdown phase, so I had to power off violently. Nothing seemed really problematic on the hardware end, and after a reboot, both disks were properly working. The MD RAID would resynchronize, and the btrfs filesystems would be automatically mounted. It would work for a while, until such things could be seen in the logs, with more garbage as above in between:

May 10 14:41:18 goemon kernel: [ 1253.455545] __ratelimit: 35363 callbacks suppressed
May 10 14:45:04 goemon kernel: [ 1478.717749] parent transid verify failed on 358190825472 wanted 42547 found 42525
May 10 14:45:04 goemon kernel: [ 1478.717936] parent transid verify failed on 358316642304 wanted 42547 found 42515
May 10 14:45:04 goemon kernel: [ 1478.717939] parent transid verify failed on 358190825472 wanted 42547 found 42525
May 10 14:45:04 goemon kernel: [ 1478.718128] parent transid verify failed on 358316642304 wanted 42547 found 42515
May 10 14:45:04 goemon kernel: [ 1478.718131] parent transid verify failed on 358190825472 wanted 42547 found 42525

Then there would be kernel btrfs processes going on and on sucking CPU and I/O, doing whatever it was doing. At such moment, most file reading off one of the btrfs volumes would either take very long or freeze, and un-mounting would only freeze. At that point, considering the advantages of btrfs (in my case, mostly, snapshots) were outweighed by such issues (this wasn t my first btrfs fuck up, but by large, the most dreadful) and the fact that btrfs is just so slow compared to other filesystems, I decided I didn t want to care trying to save these filesystems from their agonizing death, and that I d just go with ext4 on MD RAID instead. Also, I didn t want to just try (with the possibility of going through similar pain) again with a more recent kernel. Fortunately, I had backups of most of the data (only problem being the time required to restore that amount of data), but for the few remaining things which, by force of bad timing, I didn t have a backup of, I needed to somehow get them back from these btrfs volumes. So I created new file systems to replace the btrfs volumes I could directly throw away and started recovering data from backups. I also, at the same time, tried to copy a big disk image from the remaining btrfs volume. Somehow, this worked, with the system load varying between 20 and 60 (with a lot of garbage in the logs and other services deeply impacted as well) But when trying to copy the remaining files I wanted to recover, things got worse, so I had to initiate a shutdown, and power cycle again. Since apparently the kernel wasn t going to be very helpful, the next step was to just get other things working, and get the data back some other way. What I did was to use a virtual machine to get the data off the remaining btrfs volume. The kernel could become unusable all it wanted to, I could just hard reboot without impacting the other services. In the virtual machine, things got interesting . I did try various things I ve seen on the linux-btrfs list, but nothing really did anything at all except spew some more parent transid messages. I should mention that the remaining btrfs volume was a RAID 0. To mount those, you d mount one of the constituting disks like this:

$ mount /dev/sdb /mnt

Except that it would complain that it can t find a valid whatever (I don t remember the exact term, and I threw the VM away already) so it wouldn t mount the volume. But when mounting the other constituting disk, it would just work. Well, that s kind of understandable, but what is not is that on the next boot (I had to reboot a lot, see below), it would error out on the disk that worked previously, and work on the disk that was failing before. So, here is how things went:

• I would boot the VM and mount the volume,
• launch an rsync of the data to recover, which I d send onto the host system,
• observe, from the host system, what was going on I/O wise,
• at some point (usually after something like 10 to 50 files rsync ed), after throwing a bunch of parent transid error messages, the VM would just stop doing any kind of I/O (even if left alone for several minutes), at which point I d hard shutdown the VM and start over.

Ain t that fun? The good thing is that in the end, despite the pain, I recovered all that needed to be recovered. I m in the process of recreating my build chroots from scratch, but that s not exactly difficult. It would just have taken a lot more time to recover them the same way, 50 files at a time. Side note: yes, I did try newer versions of btrfsck ; yes I did try newer kernels. No, nothing worked to make these btrfs volumes viable. No, I don t have an image of these completely fucked up volumes.

LightDM's a from-scratch implementation of an X display manager, ie the piece of software that handles remote X connections, starts any local X servers, provides a login screen and kicks off the initial user session. It's split into a nominally desktop-agnostic core (built directly on xcb and glib) and greeters, the idea being that it's straightforward to implement an environment-specific greeter that integrates nicely with your desktop session. It's about 6500 lines of code in the core, 3500 lines of code in the gtk bindings to the core and about 1000 in the sample gtk greeter, for a total of about 11,000 lines of code for a full implementation. This compares to getting on for 60,000 in gdm. Ubuntu plan to switch to LightDM in their next release (11.10).

This is a ridiculous idea.

To a first approximation, when someone says "Lightweight" what they mean is "I don't understand the problems that the alternative solves". People see gtk and think "Gosh, that's kind of big for a toolkit library. I'll write my own". And at some point they have to write a file dialog. And then they need to implement support for handling remote filesystems. And then they need to know whether the system has a functioning network connection or not, and so they end up querying state from Network Manager. And then they suddenly have a library that's getting on for the size of gtk, has about the same level of complexity and has had much less testing because why would you want to use a lightweight toolkit that either does nothing or is 90% of the size of the alternative and crashes all the time.

Adding functionality means that code gets larger. Given two codebases that are of significantly different sizes, the two possible conclusions are either that (a) the smaller one is massively more competently written than the larger one, or (b) the smaller one does less. The gdm authors have never struck me as incompetent, even if some people may disagree with some of their design decisions, and the LightDM authors don't seem to have argued on that basis either. So the obvious conclusion is that LightDM does less.

And, indeed, LightDM does less. Part of this is by design - as the proposal to the Gnome development list shows, one of the key advantages of LightDM is claimed as it not starting a Gnome session. And from that statement alone, we can already see that there's been a massive failure of understanding the complexity of the problem.

Let's go back to the comparisons of code size. LightDM's simple GTK greeter is about 1000 lines of code. gdm's greeter is almost 20,000. Some of this is arbitrary shiny stuff like the slidy effects that occur, but a lot of it is additional functionality. For example, some of it is devoted to handling the interface with AccountsService so gdm can automatically update when users are created or deleted. Some of it is providing UI for accessibility functionality. Some of it is drawing a clock, which I'll admit may be a touch gratuitous.

But if your argument is that your software is better because it's doing less, you should be able to ensure that you can demonstrate that the differences aren't important. And the differences here are important. For example, one of the reasons gdm starts a local gnome session is that it wants gnome-power-manager to be there to handle power policy. Closing the lid of my laptop should suspend the system regardless of whether it's logged in or not. LightDM takes a different approach. Because there's no session, it has to take care of this kind of thing itself. So the backend daemon code speaks to upower directly, and the greeters ask the daemon to implement their policy decisions.

This is pretty obviously miserable. Now you've got two sets of policy - one at the login screen, and one in your session. How do I ensure they're consistent? The only sane solution is to ignore the functionality the backend provides and have my greeter run gnome-power-manager. And now how about accessibility preferences? Again, if I want to have the same selection of policy, I need to run the same code. So you end up with a greeter that's about as complex and large as the gdm one, and unused functionality in the backend. Lighter weight through code duplication. We have always been at war with Eurasia.

The entirety of LightDM's design is based on a false premise - that you can move a large set of common greeter functionality into a daemon and just leave UI presentation up to the greeter code. And if you believe that, then yes, you can absolutely implement a greeter in 1000 lines of code. It'll behave differently to your desktop - the range of policy you can implement will be limited to what the daemon provides, even if your desktop environment has a different range of features. It'll have worse accessibility for much the same reason. And eventually you'll end up with a daemon that's absolutely huge in order to attempt to provide the superset of functionality that each different desktop makes use of.

The only real problem LightDM solves is that it makes it easier to write custom greeters, and if you're really seeking to differentiate your project based on your login screen then maybe your priorities are a little out of line. I'm sure that Ubuntu will release with a beautiful 3D greeter that has a wonderful transition to the desktop. It's just a shame that it won't do any of the useful things that the existing implementations already do.

And if you think that when LightDM finally has the full feature set of gdm, kdm and lxdm it'll still be fewer lines of code and take less memory - I hear the BSD kernel is lighter weight than Linux. Have fun with it.

Is there something a little odd about the Deaf Society of NSW having a website with a frontpage, and other rather crucial bits like course information lookup forms, that are built entirely using flash. Given that, as far as I know, screen reading software and flash aren't best of friends? I guess they figure that as long as people are catering to their disability, everyone else can fend for themselves. A slightly more charitable interpretation might be that blind people aren't going to be real keen to learn sign language, but I would have thought that a bit of consideration might be shown, for the purposes of solidarity or something.

IPv6 adoption is increasing, and along with it come a new set of behaviours and defaults that system administrators and users must learn and become familiar with. I was recently caught out by Linux s handling of IPv6 router advertisements (RAs) when forwarding is also enabled on the interface. It took me a while to figure out and searching for obvious terms (such as those in the first half of the title of this post) didn t immediately yield useful answers, so here is my attempt to help shed some light on the subject. By default Linux will ignore IPv6 RAs if the interface is configured to forward traffic. This is in line with RFC2462 which states that a device should be either a Host or a Router. If you re forwarding packets you re a router and you re therefore expected to be sending RAs, not receiving them. This policy does make a certain amount of sense but there are obviously situations where it can be useful to accept RAs and still forward packets over the interface[0]. The confusing part is that the Linux IPv6 stack allows the accept_ra sysctl to be set to 1 (enabled) at the same time as the forwarding sysctl is set to 1, yet all incoming RAs are ignored with no hint as to why. If you re not aware that the default behaviour is to ignore RAs when forwarding is enabled it looks very much like autoconfiguration has simply broken. The key piece of information is that makes everything as clear as mud is realising that the forwarding and accept_ra sysctl s are not simple boolean enabled/disabled flags like many of their brethren. There are instead three possible values for each, all clearly documented in sysctl.txt, when you take the time to read it. Ironically the documentation states the type of the values as BOOLEAN even though they re not at least it helped me to feel better about my hasty assumption that the sysctl s were boolean values.

accept_ra BOOLEAN
Accept Router Advertisements; autoconfigure using them. Possible values are:
0 Do not accept Router Advertisements.
1 Accept Router Advertisements if forwarding is disabled.
2 Overrule forwarding behaviour. Accept Router Advertisements
even if forwarding is enabled. Functional default: enabled if local forwarding is disabled.
disabled if local forwarding is enabled.

The documentation for forwarding is similar, but much longer, so you can refer to the link above to see it. Conclusion: If you want to autoconfigure IPv6 addresses on an interface that you re also forwarding IPv6 traffic over, you need to set accept_ra to 2. No doubt there are more IPv6 quirks and defaults like this waiting to trap me in the future [0] Arguably you really don t want to be autoconfiguring addresses on your router ever, but that s a philosophical debate that isn t really relevant to this post.

Two weeks ago, PostgreSQL announced the first beta version of the new major 9.1 version, with a lot of anticipated new features like synchronous replication or better support for multilingual databases. Please see the release announcement for details. Due to my recent moving and the Ubuntu Developer Summit it took me a bit to package them for Debian and Ubuntu, but here they are at last. I uploaded postgresql-9.1 to Debian experimental; currently they are sitting in the NEW queue, but I m sure our restless Debian archive admins will get to it in a few days. I also provided builds for Ubuntu 10.04 LTS, 10.10. and 11.04 in my PostgreSQL backports for stable Ubuntu releases PPA. I provided full postgresql-common integration, i. e. you can use all the usual tools like pg_createcluster, pg_upgradecluster etc. to install 9.1 side by side with your 8.4/9.0 instances, attempt an upgrade of your existing instances to 9.1 without endangering the running clusters, etc. Fortunately this time there were no deprecated configuration options, so pg_upgradecluster does not actually have to touch your postgresql.conf for the 9.0 9.1 upgrade. They pass upstream s and postgresql-common s integration test suite, so should be reasonably working. But please let me know about everything that doesn t, so that we can get them in perfect shape in time for the final release. I anticipate that 9.1 will be the default (and only supported) version in the next Debian release (wheezy), and will most likely be the one shipped in the next Ubuntu LTS (in 12.04). It might be that the next Ubuntu release 11.10 will still ship with 9.0, but that pretty much depends on how many extensions get ported to 9.1 by feature freeze.

My previous website was generated (dynamically) with mod_autoindex, a module from Apache targeted at rendering an index page for the files contained in a directory. Pretty basic (but themable). Here is a new revamped web site using Hyde, a static web generator written in Python.

Static or dynamic ? It has been a long time since almost everybody is creating dynamic websites. Most often, they are powered by some blog engine written in PHP like DotClear or WordPress. Infrequently, they are crafted with bare hands using one s preferred language. Nowadays, hosting a dynamic website is pretty easy. There are a lot of cheap virtual servers solution that allow you to get a root access. However, dynamic websites have some drawbacks:

• Security need to be monitored tightly, especially security upgrades; some platforms are more often vulnerable than others.
• Dynamic websites may have some problem to handle increased load.

With a static website, security problems are easier: there are far less components to monitor and most of them (for example, the web server) are pretty solid and well audited. A site powered by nginx can handle several thousands of new connections per second without any special tuning. Getting this kind of performance with a dynamic website is far more difficult. Scaling is also easier. Just add more webservers and copy the content. No database, no synchronisation mechanism needed. Hosting can even be done on simple platforms like Amazon S3, even if there is now more appropriate platforms for this like Amazon CloudFront. Moreover, even a static website can receive comments using a system like Disqus which leverages the use of Javascript to add comment system to any static page.

Hyde Hyde is a static website generator written in Python. There exists a lot of those beasts. On StackOverflow, a question about this got some interesting answers. You will also find a fairly complete list on this blog. If you like Ruby, I assume that nanoc and Webgen are good candidates. Let s go back to Hyde. It was rewritten recently and the documentation is a bit incomplete. You will need to read the source code or look at examples (like the shipped one or this site) to understand how things are handled. Everything is handled by a plugin system. I hope it will not become a mess in one year (I play a large part in this). Hyde takes your files, apply a lot of filters on them and generate the final website. It uses Jinja as a template system, Markdown for markup syntax, Less to write CSS in a more efficient way (and to minify them), UglifyJS to minify Javascript, etc. It is also possible to add metadata to files (title, author, creation date, etc.). But since everything comes as a plugin, you can use any technology you like. In the end, you can tune all knobs but you need to do most of the work (HTML, CSS, Javascript), as well. Conveniently, there is a small web server that rebuild modified files on the fly. This comes handy but this is a common feature with this kind of generator.

New website Here is a new website crafted with Hyde. It is translated both in French and in English. This is done with the help of a small plugin plugin. I am quite happy about the result. Technically, there is one minified CSS file, one minified Javascript file and the dependencies (like jQuery) are loaded with the help of yepnope.js which will become part of Modernizr. I don t want to support Internet Explorer 6, but I plan to support Firefox 3, Internet Explorer 7 and recent versions of Chrome and Safari. For the design, I have always been undecided. I am pretty impressed by some raw website but also by some others more elaborate. Since I am not a talented graphic designer, I have tried to keep things plain and simple. If you are interested, sources are available on GitHub !

I started doing daily builds of the Debian Installer in February 2004. (Before me, Martin Sj gren did them for a while.) Seven years of building d-i every single day, twice a day in recent years, and also twice a day for armel. Well over ten thousand builds total. That would have been tedious, but thanks to cron it was instead seven years of keeping the machines running and upgraded, dealing with things when they broke, and getting highlighted on IRC whenever someone mentioned "people.debian.org/~joeyh/d-i". Still a bit tedious, but those bits were used to install a lot of machines. The rest of the d-i daily builds moved onto the autobuilders a while back, but I guess there was some reluctance to mess with my institution. I finally convinced them to take over my builds too. :) http://d-i.debian.org/daily-images/

I'm playing with btrfs to get a feel for what's coming up in linux filesystems. To be daring, i've configured a test machine using only btrfs for its on-disk filesystems. I really like some of the ideas put forward in the btrfs design. (i'm aware that btrfs is considered experimental-only at this point). I'm happy to report that despite several weeks of regular upgrade/churn from unstable and experimental, i have yet to see any data loss or other serious forms of failure. Unfortunately, i'm not impressed with the performance. The machine feels sluggish in this configuratiyon, compared to how i remember it running with previous non-btrfs installations. So i ran some benchmarks. The results don't look good for btrfs in its present incarnation. UPDATE: see the comments section for revised statistics from a quieter system, with the filesystems over the same partition (btrfs is still much slower). The simplified test system i'm running has Linux kernel 2.6.39-rc6-686-pae (from experimental), 1GiB of RAM (no swap), and a single 2GHz P4 CPU. It has one parallel ATA hard disk (WDC WD400EB-00CPF0), with two primary partitions (one btrfs and one ext3). The root filesystem is btrfs. The ext3 filesystem is mounted at /mnt I used bonnie++ to benchmark the ext3 filesystem against the btrfs filesystem as a non-privileged user. Here are the results on the test ext3 filesystem:

consoleuser@loki:~$ cat bonnie-stats.ext3 
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
loki          2264M   331  98 23464  11 10988   4  1174  85 39629   6 130.4   5
Latency             92041us    1128ms    1835ms     166ms     308ms    6549ms
Version  1.96       ------Sequential Create------ --------Random Create--------
loki                -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  9964  26 +++++ +++ 13035  26 11089  27 +++++ +++ 11888  24
Latency             17882us    1418us    1929us     489us      51us     650us
1.96,1.96,loki,1,1305039600,2264M,,331,98,23464,11,10988,4,1174,85,39629,6,130.4,5,16,,,,,9964,26,+++++,+++,13035,26,11089,27,+++++,+++,11888,24,92041us,1128ms,1835ms,166ms,308ms,6549ms,17882us,1418us,1929us,489us,51us,650us
consoleuser@loki:~$ 

And here are the results for btrfs (on the main filesystem):

consoleuser@loki:~$ cat bonnie-stats.btrfs 
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
loki          2264M    43  99 22682  17 10356   6  1038  79 28796   6  86.8  99
Latency               293ms     727ms    1222ms   46541us     504ms   13094ms
Version  1.96       ------Sequential Create------ --------Random Create--------
loki                -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1623  33 +++++ +++  2182  57  1974  27 +++++ +++  1907  44
Latency             78474us    6839us    8791us    1746us      66us   64034us
1.96,1.96,loki,1,1305040411,2264M,,43,99,22682,17,10356,6,1038,79,28796,6,86.8,99,16,,,,,1623,33,+++++,+++,2182,57,1974,27,+++++,+++,1907,44,293ms,727ms,1222ms,46541us,504ms,13094ms,78474us,6839us,8791us,1746us,66us,64034us
consoleuser@loki:~$ 

As you can see, btrfs is significantly slower in several categories:

• writing character-at-a-time is *much* slower: 43K/sec vs. 331K/sec
• reading block-at-a-time is slower: 28796K/sec vs. 39629K/sec
• all forms of file creation and deletion are nearly an order of magnitude slower
• Random seeks are almost as fast, but they swamp the CPU

I'm hoping that i just configured the test wrong somehow, or that i've done something grossly unfair in the system setup and configuration. (or maybe i'm mis-reading the bonnie++ output?) Maybe someone can point out my mistake, or give me pointers for what to do to try to speed up btrfs. I like the sound of the features we will eventually get from btrfs, but these performance figures seem like a pretty rough tradeoff.Tags: benchmarks, bonnie, btrfs, ext3

Almost a year ago I put together a simple picture sharing site, which was designed to accept uploaded images via OpenID logins and then share them. It was created because none of the existing image sharing sites allowed OpenID logins - and because frankly I rarely trust sites I don't control. (Especially when they're free sites, with no obvious revenue stream. We can reference recent compromises if we wish to be topical, but even without them it is all about trust.) I opened up the site to allow others to use it, but assumed it would remain Steve-only - After all why would somebody choose to use my site rather than imgur, or similar? Over the year I've spotted several minor problems with the site. The built-in support for comments was never used, primarily because I didn't have any other users, and some of the tagging aspects were hard to use. Since I've started using twitter I again found myself wanting a picture-hosting site, one with "short links". I started using twitpic.com, but wasn't happy with it. So the plan was formed; build another picture sharing site. But this time with short URLS. Sadly my first choice of http://picc.ie was denied to me - I couldn't register it as I don't live in Ireland. But my second choice http://linkti.me was free. As a challenge to myself this site uses Redis for storing all user-details, all image details, and all session data. It hasn't been difficult to implemenet a multi-user site, with logins, file uploads, and tagging. I guess I cheated as a lot of the code is borrowed from the previous site, but this time it looks prettier and there have been many cleanups due to my previous experience. This time round I was more focussed upon short links, so each image is allowed an incrementing number, which is then encoded using base 36 math - which is the alphabet you get if you use [0-9A-Z]. I designed three shareable links for each image to keep it simple:

• http://linkti.me/v/S - Show a HTML page with the image embedded.
• http://linkti.me/i/S - Show a raw image.
• http://linkti.me/r/S - Show a "thumbnail" of the image.

Why "/r" for the thumbnail? Well because it is a random thumbnail. To cope with difference sizes, and allow the recent image display to look regular, I wanted each thumbnail to be 150x150 pixels. My thumbnail generation simply picks a random X,Y co-ordinate from the image and then cuts out a 150x150 square block starting at that offset. The results of a random segment of an uploaded image are sometimes very cool, altough equally often they are misleading. If I upload naked people there is the possability that we'll just get a nipple or a cock - but I'm actually quite smug about the way it turned out. Simpler to code than to resize, scale, or similar and surprisingly random when you see 9x9 random images (as you do on the front page). Anyway that's all for now. I was tempted to talk about implementation or migration, but I've written too much for an early morning already. Some other time? ObQuote: "He's just trying to freak us out. " - 30 Days of Night.

I was amazed this morning that it's only been a month since I moved house. So much has happened since. For reasons that I won't go into, I've had to move house. Since I like living in Mechelen, not too far from work, however, I decided to search for something close by. And I did find something close by; I moved to "just around the corner". Which is a bit silly, but then blame the bureaucrats. At any rate, I'm starting to feel at home in the new spot. It's a bit smaller and a bit more expensive than the old place, but it looks a lot better, and that more than makes up for it. And it's not like I needed the extra space in the old place -- I kept having this big empty spot in the middle of my living room, anyway. To make matters more interesting, I've been on stage three times and out of the country once in this past month: One wedding, two concerts, and a work trip to Cork. I guess that's why it seemed like so much more time. But that's over now, and recently I finally managed to make some time working on other projects. Today it was incorporating some patches (which turned out not to work entirely) and doing a release for nbd. That release was overdue by quite a bit; there were a few fairly serious bugs outstanding that I had commits for in the git repository already, I just had to get them out there. Oh well. A few weeks back I've also managed to work a bit on ipcfg some more. The reason it has been so long since the last time I worked on it was that I had started to realize there were several things wrong with it, and I wasn't sure how to best move forward. But then not so long ago I had an epiphany, and I've now started to reimplement the broken bits which is, well, most of it. But hey. Moving isn't fun. Glad it's over. Almost.

GBrowse, a genome browser, is packaged for Debian since three weeks. It is the result of a work started four years ago, and I am especially grateful to Olivier Sallou for having unblocked and finished the project while I was starting to go round in circles, and to Aaron M. Ucko for repairing the new package, that was not compatible with our build farm. GBrowse and similar software, like IGV, are used for graphical representation of a chromosome and its annotations, such as the gene positions and markers of their activity, and in particular the result of experiments using high throughput DNA sequencing. A Debian Med task is actually dedicated to gather programs for that topic. In a couple of years, it may be possible that any person who wants can sequence his chromosomes, that is, his genome. This will revolution medicine and perhaps break some founding myths. Even if software for the general public will look different, packages like gbrowse are a first step towards a better access of the public to his medical data. Work continues on GBrowse, with the goal of being able to install it with a reference version of the human genome with a command like apt-get install human-genome. The next step will be to update BioPerl, used by GBrowse, to version 1.6.9.

DebConf11 program, the FOSS way I'm going to DebConf11 and, once more, I'm thrilled at idea of living for about 2 weeks immersed in a very very very, very Debian-ic life. DebConf10, DebConf edition of last year, has been for me one of the most intense DebConfs ever, it would be hard to match it. But one of the nice things about DebConf, no matter the edition, is that it's very "open source" and do-ocratic in they way it is organized: everyone could contribute into making it even better than the previous edition. For one thing, there is no "event organizer" company behind DebConf. Rather, most of the hard work that goes into making a DebConf possible is done by the DebConf team, a volunteer team formed by a mixture of Debian contributors, local team members, and DebConf enthusiasts. But while they take care of most of the work and hence deserve the thanks of all DebConf attendees for that they do not account for all that makes a DebConf a great DebConf. For instance, having a great program is up to DebConf participants and the event proposals they submit: talks, BoF, panel discussions, film projections, contests, you name it. Today, in deadline frenzy mode (read on), I've submitted for review the following events for DebConf11:

• bits from the DPL, the traditional "state of the union"-like address
• Test Driven Development BoF, about DEP-8 and autopkgtest (co-authored with Tom Marble)
• sprint and money BoF, to review one year of the Debian sprint program and further discuss how we use donations to the Debian project
• Debian dErivatives eXchange BoF, about DEX and other initiatives to improve collaboration with Debian derivatives
• DebConf in Debian BoF: one year later the "DebConf vs Debian" BoF, what's the state of integration of DebConf in Debian? (spoiler: good!) how can we improve it further? (co-authored with Gunnar Wolf, Holger Levsen, and Moray Allan)
• DPL wannabe BoF: inspired by a discussion during this year DPL campaign, an attempt to explain what the "DPL job" is about and, who knows, collect "training" material for wannabe DPLs
• rolling BoF: let's summarize the monster thread about Debian rolling for those we lost half-way through and present the advancements of more recent and consensual proposals (this event is actually not by me: I've just prodded some of the rolling proponents on IRC and Raphael Hertzog has kindly agreed to submit and organize the BoF)

That's it from me, how about you?
Are you going to DebConf and willing to attend a conference with a great program? Great, then start submitting a great event yourself! Anything you are working on in Debian or you are excited about qualifies. And don't worry if you've never attended DebConf either, video archives from past DebConfs can give you more than a feeling of typical DebConf events. While the first deadline for events submission expires today, submissions will be open all the way to the end of the conference, but you should better hurry up as the early birds get the best scheduling slots (and missing the deadline, you'll miss the round of the review for the main program ... but you still have a few hours left :)). Good luck and see you in Banja Luka.

I've just uploaded a new KVIrc SVN snapshot (r5829) to Unstable. This one took a little longer than the "normal" ten days because the transition of the previous version waited for some other packages to migrate first. This transition might also take longer than we hope for, due to #625607 in EGLIBC, which leads to a segfault in CMake on SPARC. The stable update to (more or less) upstream's 4.0.4 is also almost ready. We hope for an upload soon.