Various people observed in a long thread on debian-devel that the grub2 package was in a bit of a mess in terms of its release-critical bug count, and Jordi and Stefano both got in touch with me directly to gently point out that I probably ought to be doing something about it as one of the co-maintainers. Actually, I don't think grub2 was in quite as bad a state as its 18 RC bugs suggested. Of course every boot loader failure is critical to the person affected by it, not to mention that GRUB 2 offers more complex functionality than any other boot loader (e.g. LVM and RAID), and so it tends to accumulate RC bugs at rather a high rate. That said, we'd been neglecting its bug list for some time; Robert and Felix have both been taking some time off, Jordi mostly only cared about PowerPC and can't do that any more due to hardware failure, and I hadn't been able to pick up the slack. Most of my projects at work for the next while involve GRUB in one way or another, so I decided it was a perfectly reasonable use of work time to do something about this; I was going to need fully up-to-date snapshots anyway, and practically all the Debian grub2 bugs affect Ubuntu too. Thus, with the exception of some other little things like releasing the first Maverick alpha, I've spent pretty much the last week and a half solidly trying to get the grub2 package back into shape, with four uploads so far. The RC issues that remain are:

• upgrade-from-grub-legacy problems (#547944, #550477):
  I think this has just been traditionally undertested. I'm setting up a KVM image now with GRUB Legacy which I can snapshot just before and after running upgrade-from-grub-legacy, and I should be able to unpick the bugs this way.
• LVM snapshots break GRUB's LVM module (#574863):
  Sean has been working on this and seems to be nearly there. Yay.
• RAID metadata version 1.x not supported (#492897):
  This became rather more of an issue recently since mdadm switched its default from the old 0.90 format which GRUB understood. Felix put together a branch implementing the hard parts of this a while back, and I've been trying to finish it off. The hard bit is dealing with device naming, especially as the new-format and rather more useful names under /dev/md/ don't show up during d-i after creating RAID volumes; I think this is because we always create them as /dev/md0 etc. It's looking tractable, though.
• Another odd problem probing RAID (#548648):
  Not sure about this one, and I'll need to work with Josip on it as soon as I get a chance.
• Stable device naming #554790) and consequential problems due to grub-install not being properly run (#557425 and many other sub-RC bugs):
  Ubuntu's been carrying a patch to rearrange device presentation in the postinst, which Robert OKed in principle ages ago and so I've been intending to merge it for a while, but there are a few known problems with it that I need to fix first. One known unfixable problem is that it will have to ask some people which devices they want GRUB to be installed on, even if they'd answered that question before: this will be one-time, and it's because it recorded the answer using unstable device names and so has in some sense forgotten. Simple cases (e.g. single-disk) can be handled without needing to ask again, though.
• Alignment errors on SPARC (#560823):
  I have no idea what's going on here, I'm afraid. I'll try to trace it, but may have to downgrade it at some point since after all we don't install GRUB by default on SPARC yet.
• Fonts not shown in gfxmenu (#564844):
  Apparently fixed upstream, but I couldn't find the responsible commit so I want to make sure I can get gfxmenu working before closing this.
• Sensitivity to out-of-date device.map files (#575076 and other sub-RC bugs):
  We're trying to get rid of device.map in general. It was fine in the 1990s but it's hopeless now. Unfortunately there are still a small number of problems with running entirely without one, and one of my patches to help is controversial upstream, so we probably won't get to that for squeeze. In the meantime we'll probably just need some extra sanity-checking and robustness in the event that there's an incorrect or out-of-date device.map lying around, which we may just be able to do in the maintainer scripts or something if necessary.
• Seriously weird failures to load initramfs (#582342):
  If anyone can produce a reproduction recipe for this, that would really help me out. There are too many reports to discount as user error, but I haven't seen this myself yet.
• Build failure on sparc (unfiled):
  We've been discussing this upstream, but for the time being I'm just going to stop building grub-emu on sparc as a workaround.

If we can fix that lot, or even just the ones that are reasonably well-understood, I think we'll be in reasonable shape. I'd also like to make grub-mkconfig a bit more robust in the event that the root filesystem isn't one that GRUB understands (#561855, #562672), and I'd quite like to write some more documentation. On the upside, progress has been good. We have multiple terminal support thanks to a new upstream snapshot (#506707), update-grub runs much faster (#508834, #574088), we have DM-RAID support with a following wind (#579919), the new scheme with symlinks under /dev/mapper/ works (#550704), we have basic support for btrfs / as long as you have something GRUB understands properly on /boot (#540786), we have full info documentation covering all the user-adjustable settings in /etc/default/grub, and a host of other smaller fixes. I'm hoping we can keep this up. If you'd like to help, contact me, especially if there's something particular that isn't being handled that you think you could work on. GRUB 2 is actually quite a pleasant codebase to work on once you get used to its layout; it's certainly much easier to fix bugs in than GRUB Legacy ever was, as far as I'm concerned. Thanks to tools like grub-probe and grub-fstest, it's very often possible to fix problems without needing to reboot for anything other than a final sanity check (although KVM certainly helps), and you can often debug very substantial bits of the boot loader - the bits that actually go wrong - using standard tools such as strace and gdb. Upstream is helpful and I've been able to get many of the problems above fixed directly there. If you have a sound knowledge of C and a decent level of understanding of the environment a boot loader needs to operate in - or for that matter specialist knowledge of interesting device types - then you should be able to find something to do.

Joey Hess wrote on Debian-devel about the problem of init scripts not doing adequate checks before using the data from a PID file under /var/run to determine which process to kill [1]. Unfortunately that still doesn t quite solve the problem, there is still the issue of a race condition causing a process to die while you are doing the checks and then be replaced by another process. Below I have included the source code to a little program that will repeatedly fork() until it finds a particular PID and then have it s child call sleep(). So you can run a command such as kill -9 1234 ; ./a.out 1234 and then have this program take over the PID 1234. From testing with this it seems that when you have a shell with it s current working directory as /proc/1234 then once process 1234 is killed the current directory is empty, ls -l returns 0 entries. This isn t surprising, it s the standard Unix behavior when the working directory is removed. So if a program (or even a shell script) changes directory to /proc/1234 it can then verify all attributes of the process (it s CWD, it s root directory, the executable used to run it, it s UID, GID, supplemental groups, it s SE Linux context, and lots of other things all atomically. The only possibility for confusion is that a process might execute a SETUID or SETGID program or a program that has a label which triggers a SE Linux domain transition. It might also change some attributes without executing a new process, for example by using the setuid(), setgid(), setgroups(), or other similar system calls. For the purposes of killing a process I don t think that the possibility of it changing it s own attributes or executing a new program are serious problems, if you want to kill a process then you probably want to kill it after it has called setuid() etc. It seems to me that it would be useful to have a file named /proc/PID/signal (or something similar) to which you could write a number and then have the kernel send the signal in question to that process. So the commands kill -9 1234 and echo 9 > /proc/1234/signal would give the same result. But you could run the command cd /proc/1234 and then some time later you could run echo 9 > signal and know that you would kill the original process 1234 if it was still running and not some other process that replaced it. What do you think? Is this worthy of adding a new feature to the proc filesystem? The Source Run the following program with a single parameter which is the PID that you want it to take. It will keep looping through the PID space until it gets the one you specify, if the one you specify isn t available (EG you give it 1 as a parameter) then it will run forever. It will take some time to get a result on a slower system. On my P3-900MHz test system it took up to 72 seconds to get a result. #include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h> int main(int argc, char **argv)

if(argc != 2)

printf("Specify the desired PID on the command-line\n");
return 1;

pid_t wanted = (pid_t)atoi(argv[1]);
int rc;
int status;
while((rc = fork()) != wanted && rc != 0)

if(rc == -1)

printf("fork error\n");
return 1;

if(wait(&status) == -1)
printf("wait error\n");

if(rc == 0)

if(getpid() == wanted)

printf("Got pid %d, sleeping\n", wanted);
sleep(200);

return 0;

wait(&status);
return 0;

• [1] http://lists.debian.org/debian-devel/2010/05/msg00067.html

It seems I ve missed quite a few weeks in my countdown to our Mini DebConf. Fortunately, it s not too late for you to join even if, right now, you don t know why you should! So what s the status? Well, we ve had out problems in the beginning to start the whole thing but we managed. There are a few minor issues left, I admit. If you happen to have a bit time on 10th or 11th to hold the camera for an hour or so, contact us! Or if there is *anything* you would wanna do to help us out a bit, don t be shy. You don t need to be a known Debian contributor, you don t need to know us. Arguably, you should know Debian but then, if you re able to spell the name correctly, you re probably in. :-) Apart from that, we really think it s gonna be awesome. Not only will there be a few Debian Developers from outside Germany, like our project leader zack who we have the honor of listening to for he has the first talk on Thursday. There will also be quite a few German faces that some of us haven t met yet have you met Sebastian, the chief of our beloved debianforum.de, yet? We are going to have a great time and you re a fool not to join. Thus, a last time for all of you who still don t know what I m talking about: Next week there ll be the famous LinuxTag in Berlin from Wednesday to Saturday. Right in the middle, on Thursday and Friday, we, the Debian community, will have a Mini DebConf right there. There will be talks about different Debian related things. And many active developers will be there happy to talk to you about whatever you always wanted to know or say about Debian. And then, not to forget, we ll have a hacklab where you can easily plug in your notebook and start hacking with us. Have an idea to improve Debian? Always wanted to know how to fix a bug? You want to make Debian Squeeze the best Debian release ever? Join us it seems we re fighting on the same side. See you there, guys! PS: The latest and hottest news about it.

Two years ago, while working on awesome, I joined the Freedesktop initiative to work on XCB. I had to learn the arcane of the X11 protocol and all the mysterious and old world that goes with it. Now that I've swum all this months in this mud, I just feel like I need to share my thoughts about what become a mess over the decades. When I was unborn

the Toto band were releasing their song "Africa" and some smart guys were working on a windowing system: the X Window System (this is its full name) which therefore has a (too) long history. The latest version of its protocol, the 11th one, has been designed in the 80's. You can learn more about the history in the Wikipedia article about X. In 2010, we still listen disco music and we still use various protocols designed in the 80's and even before X. Music have evolved, protocols have evolved, and so did X11. The problem is that X11 did not evolve that well. The guys at MIT-and-some other-places-with-very-smart-people-in-it created X version 1 in 1984, and updated it until X version 11 (the one we're still using) in 1987. Eleven version in 3 years, that was following the "release early, release often" model. But I don't know why, it just stopped to happen for the last 23 years¹. I don't know what changes have been made in the first 11 major versions of the X protocol, but I'm rather sure we should have deserve a couple of major version updates this last 2 decades. In my humble opinion, X11 was not designed to live 23 years. But hey, I'm not blaming anyone here: I was 4 years old and playing Lego when they released this latest version of the X protocol, so there is little chance I'd have done something better.

1. That's not totally true: they added (and then deprecated) many extensions. We won't fix. We'll work-around.

That is probably one of the guideline of the X protocol for the last years. And don't misread me: I'm not bashing anyone thereafter. Since the X11 protocol was aging, the X guys started to add extensions. They added tons of them over the years. This, in application of one of the early principles of X:

It is as important to decide what a system is not as to decide what it is. Do not serve all the world's needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.

All of them with no exception were added because, bad luck, the X11 protocol did not anticipated the things that happened in the last 23 years, like video, OpenGL, multiple monitors, or the pleasure to draw oval windows. Some of this extensions are still in use, while some of them have been dropped. While this is not a bad thing to extends the protocol, it seems like a bad thing to try to fix the protocol with for example the XFixes extension, even with all the good intentions Keith Packard might have in his greatness. Actually it's even worst than you think

The X11 protocol (without extensions) defines about 120 types of requests: create a window, move a window, etc. Nowadays, there's at least 25 % of them which are useless: usage of server-side font, or the drawing of squares and polygon, are unused by any modern application or toolkit. All of this is superseded by requests from extensions, like the XRender one. The handling of multiple monitors displays has totally been screwed up. X11 has been designed to work in Zaphod mode (independent monitors). But Xinerama, and nowadays XRandR have replaced it up: recent X servers (released after ~2007) does not support Zaphod mode anymore, even if it's a core piece of the X11 protocol. Worst: on many requests, there's limitation or design flaws, like described in this document: Why X Is Not Our Ideal Window System by DEC researchers. We'll add more broken standard on top of that

Following its early principle, X does not define policies but only mechanisms, which seems like a good thing, Consequently, people started writing specifications to determine a number of stuff and dogmas: ICCCM. That was 22 years ago in 1988. It's useless to add that many things in this specification are now obsolete, useless, or that it misses many modern stuff. I was not the only one to think that. The people from what will be the major desktop environments, KDE and GNOME, saw that too in the 90's while I was learning to count. So they wrote EWMH, another standard that comes on top of ICCCM and extends it with nifty features like maximization, full screen mode, etc. The problem is that this standard has also been written by narrow-minded people who at that time, were working on GNOME or KDE (and maybe others). This desktop environments were having and still have some strong concepts of how should work a desktop: "it should have work-spaces", "a window is only on one workspace", "we only see a workspace at a time", "you do not have multiple screens", etc. Dude, we don't care: we have toolkits!

This vision of how the desktop should work have now been written in marble in all applications and libraries implementing EWMH, like GTK+ or Qt. Nowadays, everybody forgot about all of this standards. Toolkits have implemented this, circumvented the X11 protocol limitation and flaws, and nobody wants to look back. Like all standards, obviously some people implemented them badly. This had some side effects, like OpenOffice acting like a pager. We don't look back? Worst, we forgot where we came from!

With all these layers of bad designed standards, the desktop continued to evolve for more than a decade. They continued to add more standard, the more recent ones being based on D-Bus like the Desktop Notification Specification or the latest Status Notifier Specification developed by KDE. The Status Notifier is a new implementation of the good old system tray based on XEmbed, using D-Bus instead of the X11 mechanisms, and adding the possibility to show the system tray with something else than icons. This specification draft saw an important issue and design flaw raised by Wolfgang Draxinger in this thread on the XDG mailing-list. What Wolfgang points out, is that X is network-oriented, and D-Bus is not. Therefore, making the Status Notifier specification to use D-Bus to pass system tray messages around is a bad idea, since running a X application from host A on host B will draw the system tray on the wrong host! Apparently, reading the thread, this does not fear some of the KDE people:

of course this is a bizarre corner case not worth much thought. at least that's what you'll think until you actually run into it yourself (be it because you are testing something or because you are setting up some weird kiosk environment).

What Oswald describes as a corner case is an actual common use case for many of us. Of course, YMMV. From my point of view, this is a step back in the wrong direction. But we can conclude that the network part of X is now worthless, to at least KDE. I used to believe in XCB

When I joined Freedesktop, it was to work on XCB, the X C Binding. XCB is a nice, clean, 21st century technology based API to play with the X11 protocol. Its code is auto generated based on XML file describing the protocol. In comparison, Xlib is 80's obfuscated code with almost no comments and hard-coded things. Only a few people can understand some of its corner like its i18n or XKB implementations. And all its code is synchronous. For people not knowing it yet, X is a network protocol where you send request (like a GET in HTTP) and then get a response. Xlib forces the application to wait for the reply to its request, so the application is blocked until the X server sends the reply to the request. XCB on the other hand does not block and allows the application to send a batch of requests, do some other stuff in the mean time, and then gets the replies. It's like your Web browser would send one request at a time to a Web server, and would wait until you downloaded all the images one by one to display the page. In cases where X and all its clients are on the same host, the latency is small and not really visible, therefore the gain for XCB to be asynchronous is small. On slow network however, the gain can be huge, as proved in the rewrite of xlsclients with XCB by Peter Harris. One of the long standing goal of the XCB folks is to kick-out Xlib, to increase speed and hides latency in X11 applications. That requires to port many libraries, because almost none of them (Cairo being an exception) supports XCB. From where I stand, I don't really see if the work is worth it now. The desktop world is trusted by GNOME and KDE, meaning GTK+ and Qt. It seems none of this toolkits are interested to work on XCB, neither on the X protocol. They probably put hard effort in bypassing X limitation and flaws, and they now sit on top of crap of workarounds and broken-by-design-standard implementation. It seems to me they don't want to go back in the layers and improves things. They're too high to go back down and they don't see what the gain would be. Enlightenment with its EFL was the first toolkit to have an XCB back-end with the work of Vincent Torri. Unfortunately, the back-end is not maintained and nobody cares about it. Last time I tried it, it did not compile at all. X12 ?

There's a page called X12 on the Freedesktop wiki, listing all the things that should be fixed some days. Unfortunately, the list continues to grow up an no one talks about working on X12. On the other hand, there's a handset of people trying to work when they will have time on XKB2, the second version of the "let's-try-to-fix-up-the keyboard-part-of-the-protocol-we-wrote-23-years-ago-a-second-time" extension. To me, it does not seem X12 will happen in the next decade neither. Alternative ?

Do we got alternative to X ? There's Wayland, but it's far from being usable. There's DirectFB, but that's not very portable. None seems candidate to replace X some days to me. Anyhow, none of the main toolkits around support this alternative. GTK+ once supported DirectFB, but as far as I know, it is not supported nor works nowadays, as stated by Josselin Mouette. This is why recent versions of the Debian installer have migrated to X for the graphic part, thanks to Cyril Brulebois work. Conclusion

XCB has been around for more than half-a-decade, and very few people showed interested in it. As far as I can see, nobody is interested to use the X protocol and everybody tries to encapsulate it in some higher-level API as soon as possible to stop seeing it. This leads to poorly written application and toolkits, with a lot of ugly hack. All of that also means that starting to write applications and graphical toolkits based on XCB would be a very interesting project, but that would lead to spend too much time learning to circumvent the X protocol flaws, things that have been done in years by predecessors like Qt and GTK+. Major toolkits implementations have almost nothing to win in going back in the dark water of X. I guess most of their folks prefer to work on shiny 3D effects based on your GPS location, rather than redefining better basis for everyone. The manpower available in the X world is very small. Debian lacking of X maintainers is just the summit of that. There is very smart and very competent and skilled guys in the X world, as you can see by simply reading blog posts on Planet Freedesktop for example (me excluded). Unfortunately, there's not enough of them to cover all the things involved in X: input devices, graphics devices, new protocol extension specification and so on. The X server is really late, and it seems most of the developers prefers to work on the server itself than on the protocol behalf. Which is understandable. I'm curious to see where all of that will lead in the upcoming years. I've been walking in the X world hallways for about 3 years now, and I feel desktop alternatives to KDE and GNOME will all die sooner or later. The time were you could choose between a dozen "modern" window managers has passed away. After all, maybe that is simply Darwinism applied to computer software.

Last week was very busy for me. Unfortunately, I spent most of my time no my study duties. An exam session is fast approaching, so in next 3 weeks the situation will not be better. But as I promised, during second part of week, I am working full time on my project.

So, here are missing mockups of future Aptitude Qt GUI:

Changes Preview tab:

Only one "Preview Tab" can be opened at the same time. Changes will be live updated here.

Resolver:

Perform Changes tab:


This time I am not too innovative. I think that current Aptitude GTK design for this tabs is quite intuitive and good-looking, so I have added only minor tweaks.

To sum up mockups topic, below are links to all presented mockups (with some comments which raises during my recent private discussions). Now, I am open for any suggestions and comments about current design. At the end of next week, second iteration will be presented. Depending of the level of required changes there could be third iteration, before sending mockups to Piotr "Patpi" Pe zowski, who is the person behind Kadu Usability Project. I am glad that he agreed to spend his spare time and help me with this task :)

Packages tab:

This tab in second iteration will have only small changes. I was told that will be nice to select multiple categories at the same time. I agree, so categories list will support multi selection. There will be also an option in configuration window (There is no mockup now. Currently, I do not know what option will be configurable and it is not the most important thing at this stage. The window will be patterned upon KDE configuration windows) to move this list on the right side of this tab.

It will be possible to have multiple "Packages tabs" opened at the same time. It will not be allowed to close first "Package Tab". Ctrl + T shortcut, action in main menu and toolbar (I have forgotten about toolbar in this iteration - maybe it is not necessary?) will open new tab. There will be also an option that each text search will do this. I will try to add shortcuts wherever is it possible to allow using the GUI without mouse.

"Show changes" button will be change to "Resolve dependencies" when necessary.

Updates widget:

Most of the comments from "Packages tab" apply here too. In the table showing updates will be also "Current Version" column.


Package Informations tab:

Only one clarification: multiple "Packages Informations" tabs for the same package will not be allowed.

I am looking forward to read any comments regarding presented design. Please leave comment below this post or contact me directly (see About Me page).

On Monday/Tuesday I will write first weekly report.

Cheers.

Today, I have finished only one mockup. Designing "Package Details" tab was much harder than I had thought it would be. But here is it:

Package details:

It is not finished and requires more work (as all mockups). There are two main mockups left (Preview and Resolver tabs). They will have been finished till the end of Wednesday. Fortunately, there is still some time to finish and evaluate mockups. As I wrote before, firstly I will be coding low level classes.

Today (it is past midnight in my timezone ;) ) is also official start of coding period (It has not been announcement yet). For the next month, I will have been sharing my time between GSoC and my normal study duties. Due to that fact, most of my work will be done in second part of each week. I will write weakly updates on this blog.

So, please stay tuned and wish me good luck :)

as a short reminder that there are still a few RC bugs waiting to be fixed here's a short list of the ones I've worked on in the last time:

• #571116 debsums: "debsums: overwrites /etc/apt.conf.d/90debsums without asking"
  upload NMU prepared by Giovanni Mascellani to DELAYED/2
• #578458 src:libcoro-perl: "libcoro-perl: FTBFS: E: Caught signal 'Terminated': terminating immediately"
  modify package to get more info (pkg-perl)
• #578638 libpam-ldap: "libpam-ldap: needs versioned dependency on libpam-runtime"
  make dependecy versioned (again), upload to DELAYED/2
• #581194 libpoe-component-irc-perl: "libpoe-component-irc-perl: Insufficient stripping of CR/LF allows arbitrary IRC command execution"
  upload new upstream release (pkg-perl)
• #581327 src:libpoe-component-sslify-perl: "libpoe-component-sslify-perl: FTBFS: tests failed"
  new upstream release + additional dependency (pkg-perl)
• #583351 libtime-y2038-perl: "libtime-y2038-perl: FTBFS"
  adjust build dependency to please sbuild (pkg-perl)
• #583418 src:libnet-z3950-zoom-perl: "libnet-z3950-zoom-perl: FTBFS: Nonexistent build-dependency: libyaz3-dev"
  correct build dependency (pkg-perl)
• #583462 src:libnet-https-any-perl: "libnet-https-any-perl: FTBFS: tests failed"
  fix test suite in debian/rules (pkg-perl)
• #583467 src:libdist-zilla-perl: "libdist-zilla-perl: FTBFS: tests failed"
  upload new upstream version (pkg-perl)
• #583751 src:ufraw: "ufraw-batch: fails to install"
  new RC bug reported

Today I received a mail from the DebConf Travel Sponsorship Team and they informed me that I may get partially (or fully) sponsored. Ok that s a bit vague for now, but that also means that I may come to Debconf 10 yeah! I also applied for this Debconf Newbies thingy, as this will be the first DebConf for me, but unfortunately there where too many applicants and not enough money and they had to decline my request.

Yesterday finally my GuruPlug Server Plus arrived. Took longer than expected, but as it seems Globalscale had some issues with the power supplies and they were replaced before shipping the GuruPlugs.

The GuruPlug Basically the GuruPlugs are an enhanced version of the well known SheevaPlugs, the biggest difference is probably the need for an external JTAG/UART<>RS232 board to access the serial console. Good thing is that the board comes with a normal JTAG connector and an additional RS232 connector and 2.5V DC power outlet, so it will be useful for other devices, too. Globalscale could have chosen different connectors with less wiggly cables, though. As I was not able to find a useful howto about installing Debian on the Guruplug, I've written down what I did to install Debian unstable on a micro SD card using the Debian installer for the GuruPlug. I did not have a look who modified the Debian installer to work on the plug, but thanks for that! The instructions below are based on Martin Michlmayer's awesome SheevaPlug documentation, the hints from oinkzwurgl.org/guruplug and various forum posts in the plugforum.

Preparations Please note that I'm not resposible for whatever you're doing with your plug. If you follow this tutorial and end up with a brick, it is your fault, not mine. To install the GuruPlug you need the JTAG Board. Connect the UART port to the GuruPlug and the JTAG board to your computer, it should show up as FTDI (thanks for using good chips!) USB<>Serial converter. Serial port settings are 115200, 8-N-1, no hw/sw flow control. The other thing you should prepare is a working tftpd, I'm using aftpd.

apt-get install aftpd

Recent versions share files from /srv/tftp/, in case you're running Lenny /var/lib/tftpboot/ should be the place to drop your files. When everything is connected properly the boot process should show up in minicom, make sure to press some key to enter uBoot. The first thing you should do is to save the original uBoot environment in case you want to restore the factory settings later. Run

printenv

and save the output somewhere.

Upgrading uBoot Unfortunately the uBoot version on the GuruPlug is pretty old and seems to have some issues in booting from USB devices, so the first thing you should do is to upgrade it. You might want to investigate if there is even a better, more recent uBoot version available somewhere, or build one on your own, but I didn't bother and took the uBoot.guruplug.bin from here. The main issue with that is that booting from USB still seems to be buggy (even for FAT partitions) and that ext2load is still not supported. Otherwise it works well :-). I'm mainly following Martin Michlmayer's tutorial again. Download the uBoot.guruplug.bin and drop it into the tftpd directory. Make sure you always set the plug's IP address (ipaddr) and your server's IP address (serverip) properly. I'll use 192.168.121.253 for the plug and 192.168.121.2 for the server in all examples, make sure to change that for your own needs. Stop if something goes wrong, especially when the tftp download failed.

setenv ipaddr 192.168.121.253
setenv serverip 192.168.121.2
tftp 0x6400000 uBoot.guruplug.bin
nand erase 0x00000000 0x0100000
nand write 0x6400000 0x0000000 0x80000
reset

Enter uBoot again after the reset.

Preparing the installer Download uImage and uInitrd to your tftpd directory. Although I've heard that setting mainlineLinux/arcNumber in the uBoot environment is not necessary anymore for very recent kernel, lets set them to make sure the Debian kernel works:

setenv mainlineLinux yes
setenv arcNumber 2097
saveenv
reset

Again, enter uBoot after the reset.

Running the installer Run the following in the uBoot console:

setenv ipaddr 192.168.121.253
setenv serverip 192.168.121.2
tftpboot 0x01100000 uInitrd
tftpboot 0x00800000 uImage
setenv bootargs console=ttyS0,115200n8 base-installer/initramfs-tools/driver-policy=most
bootm 0x00800000 0x01100000

You should see the installer starting now. You might want to follow the following hints:

• Before configuring the network, go back and set the debconf priority to low, Then continue. While chosing the Debian mirror, chose sid as the Debian version to install. If you don't have sid as choice, use a different mirror. The kernel in testing does not boot on the GuruPlug, you need 2.6.32-13 from sid.
• You might want to load the 'network console' installer component and continue via ssh. Makes things fater and colourful.
• Suggested partitioning: I've installed Debian to an 8GB micro SDcard. The SDcard reader is connected via USB and shows up as /dev/sdb (/dev/sda should be the internal NAND and not shown by the installer). I've used 150MB ext2 for /boot and the rest of the space for /, using ext4. You might want to use the noatime option on both filesystems to avoid unnecessary write access to the SDcard. You might choose to add a swap partition, but SDcards are so slow, so I've skipped that.

When you continue the installation, you will hit the following problem:

• The installer might fail on 'Make the system bootable' (not sure if it does for you, it did so with the kernel from testing).
• The uBoot is not able to boot from your /boot anyway. USB support is buggy and ext2load missing.

We'll work around these issues by writing the kernel and initrd into the plug's NAND. To do so, enter a shell in the installer and chroot into the install target. We'll then create the necessary uImage.bin and uInitrd and scp them to our tftpd directory:

chroot /target /bin/bash
cd /boot
mkimage -n vmlinuz -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -d /boot/vmlinuz uImage.bin
mkimage -n 'vmlinuz initrd' -A arm -O linux -T ramdisk -C gzip -d /boot/initrd.img uInitrd
scp uI* root@192.168.121.2:/srv/tftp

Now leave the shell, finish the installation and reboot, enter uBoot again.

Make the plug bootable To write kernel and initrd to the NAND memory, we have to transfer it via tftp first, then erase the NAND area we want to write to and then write it to the NAND. The values I've chosen here should be fine for the current Debian kernel, but you might need to change the necessary size for the initrd. To do so have a look at the output while transferring the initrd - the transferred bytes are displayed. They have to fit into the amount of bytes you write (the last option to nand write.e).

setenv ipaddr 192.168.121.253
setenv serverip 192.168.121.2
tftp 0x6400000 uImage.bin
nand erase 0x100000 0x400000
nand write.e 0x6400000 0x100000 0x400000
tftp 0x6400000 uInitrd
nand erase 0x500000 0x1fb00000
nand write.e 0x6400000 0x500000 0x600000

Now we need to set the necessary boot options. Make sure to change the root device if you've chosen a different layout from that I've suggested above, or if you're not using a SDcard.

setenv bootargs_debian 'console=ttyS0,115200 root=/dev/sdb2'
setenv bootcmd_nand 'nand start; nand read.e 0x00800000 0x100000 0x400000; nand read.e 0x01100000 0x500000 0x600000'
setenv bootcmd 'setenv bootargs $(bootargs_debian); run bootcmd_nand; bootm 0x00800000 0x01100000'
saveenv
run bootcmd

Finish Your GuruPlug should boot your new Debian installation now. Have fun! I'll try to keep the howto updated for changes in uBoot and the installer, but I might not have the time to so quickly. Patches and comments are welcome!.

root@guruplug:~# uname -a
Linux guruplug 2.6.32-5-kirkwood #1 Fri May 21 05:44:29 UTC 2010 armv5tel GNU/Linux
root@guruplug:~# cat /proc/cpuinfo 
Processor   : Feroceon 88FR131 rev 1 (v5l)
BogoMIPS    : 1192.75
Features    : swp half thumb fastmult edsp 
CPU implementer : 0x56
CPU architecture: 5TE
CPU variant : 0x2
CPU part    : 0x131
CPU revision    : 1
Hardware    : Marvell GuruPlug Reference Board
Revision    : 0000
Serial      : 0000000000000000
root@guruplug:~# 

It all started when I tried to typeset my documents with the Utopia typeface, which is Free Software, since Adobe liberated it to the TeX Users Group. There are some nice packages for changing fonts in LaTeX with Utopia, like mathdesign and fourier. But I wanted to play with a new toy, XeTeX. Then, seeing as small caps was missing from the fonts that Adobe donated, I tried to just fake them with a program that I had never heard of before: fontforge. My first attempt wasn t that very successful, but, after reading a little bit of the code, the reason for the problem was soon discovered. At the same time, I wanted to see what I had done with a PDF reader. The options aren t many here: xpdf was already removed from Debian s unstable (which is the distribution that I use), trying to update it proved to be unsuccessful and upstream isn t that active (which, by the way, led to the creation of the poppler project). Unfortunately, some of the Free PDF viewers seem to be limited to a maximum zoom of 400%, while some documents seem to demand more zooming (especially those with little, tiny letters, and for those people, like myself, that don t have good eyesight). Patching the programs to eliminate those hardcoded limits isn t that hard, but the results with evince and epdfview can be frightening: a very simple document, when zoomed with epdfview to, say, 1600% uses so much memory that one needs to watch out for the Out Of Memory killer and so much swapping that it is hard to believe. Don t believe my words. Just try it. But, hey, I just want to see a document without having to suffer too much. And 400% isn t enough for many documents that I have. Especially when I want to see how a given font has been rendered on a document. I have not yet investigated the real cause of the huge memory consumption, but I would trust the words of the developers, that evince does everything in a big piece of memory. On the other hand, if one compiles poppler right from the sources, its demo program for the glib backend can display files in a zippy manner, even with very large zooms. xpdf, though, works very well with higher zoom levels, without making the whole computer grind to a halt. Seeing as Martin Pitt has stripped xpdf to be able to use the poppler backend, I was animated to have the best of both worlds: the frequent updates of poppler and the lightness of xpdf. Martin s port of xpdf doesn t compile with a recent poppler, though: the library has changed a lot since 2006. And, in many respects, for good reason, since the original code is not that clean. But it makes harder for luses like me to use code based on fast-changing libraries. Well, to cut a long story short, after two afternoons hunting down a version of poppler that would allow me to compile Martin s xpdf version, I got bored and tried to attack the problem. Now I can use this alternative version of xpdf with the poppler backend, with very little dependencies and with little memory usage on my computer. I m keeping it for my private use, but if anybody else is interested, please let me know.

Mark Zuckerberg apparently recently promised

we will add privacy controls that are much simpler to use.

Dear Facebook. I believe your users want more privacy, not just prettier controls. They don't want you to give away their data by default, track them across web sites (that embed the "Like" buttons) and auto-connect them to various things.In particular, they want opt-in, not opt-out. And they also don't want to be auto-opted in by you either. Make the defaults all opt-out!They don't want you to index and publish their data by default like if you were a paparazzi and they were a pop star. Sure, they'd like to be a pop star, but without the paparazzi stuff, you know.There is an essential difference between actually caring about privacy, and just making it look nice (which unfortunately is what Facebook currently does, pretend it's all fine, because you could turn it off, if you go to all the sites you don't want and turn it off on each single one ...).

As a part of my Google Summer of Code project, I should take a look at criterion, a benchmark library for Haskell. As I d have to use it, and it seems to be a useful library for a lot of people, I packaged it to Debian, with all its dependencies. The haskell-criterion package is waiting on the NEW queue. Ok, so now I had to test it with something. I came up with a test related to my reading about the Haskell execution model. I noticed that I never have played with multi-threaded Haskell code to achieve better performance, but only to make things that should be in parallel work. So I thought this was a good oportunity to try both. I wrote this code: import Control.Applicative ((<$>), (<*>))
import Control.Concurrent.MVar (newEmptyMVar, putMVar, takeMVar)
import Control.Exception (evaluate)
import Criterion.Main (Benchmark, bench, bgroup, defaultMain, whnf, whnfIO)
import GHC.Conc (forkIO, par)
import System.Environment (getArgs) value :: Int
value = 32 threadsL :: [Int]
threadsL = [1, 2, 3, 4, 8, 16, 32, 64] main :: IO ()
main
= defaultMain
[bgroup "fib" $ [bench "fib" $ whnf fib value]
, bgroup "pfib" $ map pbench threadsL
, bgroup "ifib" $ map ibench threadsL] pbench :: Int -> Benchmark
pbench i = bench (show i) $ whnf (pfib i) value ibench :: Int -> Benchmark
ibench i = bench (show i) $ whnfIO $ ifib i value fib :: Int -> Int
fib n
n < 2 = n
otherwise = fib (n 1) + fib (n 2) pfib :: Int -> Int -> Int
pfib threads n
n < 2 = n
threads < 2 = fib n
otherwise = par n1 $ seq n2 $ n1 + n2
where
n1 = pfib (threads 1) $ n 1
n2 = fib $ n 2 ifib :: Int -> Int -> IO Int
ifib threads n
n < 2 = evaluate n
threads < 2 = evaluate $ fib n
otherwise
= do
f1 <- newEmptyMVar
f2 <- newEmptyMVar
forkIO $ ifib (threads 1) (n 1) >>= putMVar f1
(+) <$> evaluate (fib $ n 2) <*> takeMVar f1 My point is to benchmark the na ve implementation of the Fibonacci function. I ve done three implementations:

fib

which is the simple sequential algorithm

pfib

which call par threads times

ifib

which call forkIO trheads times

I have chosen the value to be found to 32, because it took long enought to make the algorithms show a difference, and short enough to allow me to wait for the benchmarks. I built with:

ghc --make -threaded fib

And runned with:

./fib --summary=fib.csv +RTS -N

The results were surprising for me:

Name,Mean,MeanLB,MeanUB,Stddev,StddevLB,StddevUB
"fib/fib",1.7197746428132055,1.5104790505051613,2.031735854971409,1.2921598378638277,0.9514102096387,1.7188536150257019
"pfib/1",1.246479478704929,1.1805571945786482,1.3496261318802836,0.41570319371618925,0.29446316107198395,0.5823241217769125
"pfib/2",0.709402781355381,0.6850755818963056,0.7553855475068096,0.1650893842889147,0.10136189180677138,0.26853312094492815
"pfib/3",0.6876571162819864,0.6694172366738321,0.7332831247925761,0.136836636420341,5.5142145935712775e-2,0.2581805596992922
"pfib/4",0.6940202649712561,0.6684220250725745,0.746035416948795,0.17779646229098522,9.15862482237111e-2,0.2804203635281818
"pfib/8",0.5887631162285805,0.5625207790017128,0.637939236986637,0.17967127569396815,0.10516397607491562,0.2680488710546616
"pfib/16",0.6578513725876809,0.6107277926087382,0.7372924216866497,0.3070809683902726,0.20679305873839168,0.44571891526087154
"pfib/32",0.8900135167717932,0.8123333438515664,0.9942556460976597,0.45942103747727,0.36818132854060703,0.6233813540278427
"pfib/64",0.6894888337731359,0.6386897596001626,0.7766118057847021,0.33329115700425754,0.22490082647959764,0.5323965317477072
"ifib/1",1.3290104445099833,1.2501559790253634,1.4700443919777875,0.5277373265779313,0.35208278898795436,0.9179652474421286
"ifib/2",0.7626163347840308,0.7239342554688452,0.8381309326767921,0.26785654248385327,0.1679981089775395,0.44853454282169086
"ifib/3",0.7440206035256386,0.6907905062317847,0.8396386941552162,0.3549675945489581,0.22865300516283724,0.536883746220313
"ifib/4",0.6503480085015301,0.6130640967965126,0.711405030119419,0.239522235814317,0.1712396398224678,0.3616636557596486
"ifib/8",0.6300301631569865,0.6085795434594155,0.699028070795536,0.17776228649867923,6.103918021418795e-2,0.39508078831864085
"ifib/16",0.6735970553040505,0.6393461784005164,0.7405966480851172,0.2366561170051713,0.13725890735483465,0.38273830961617655
"ifib/32",0.6162510784745214,0.5983195599198344,0.6736035832047466,0.14678572822644564,5.335959472427259e-2,0.3231734735693349
"ifib/64",0.7217859562516212,0.6820346101403237,0.7857911285042766,0.2534368199920431,0.18073576275895306,0.37636526159654315

I m only analyzing the first column. How come pfib/1 and ifib/1, that is, pfib and ifib called with just one thread, got faster than fib/fib? All they do is calling fib! There s certainly something wrong here. So I reran the tests in a single user environment (recovery-mode on GRUB). And also learned that I should not trust blindly these results in my usual user environment. The new tests were also surprising:

Name,Mean,MeanLB,MeanUB,Stddev,StddevLB,StddevUB
"fib/fib",0.4979443445852824,0.4978961411169597,0.49802434593949996,3.1186229071682967e-4,2.1763919616793833e-4,4.8569259617552764e-4
"pfib/1",0.4980979338339395,0.4980578795126507,0.49816361099992496,2.58930038115918e-4,1.7011790805380493e-4,4.0161943492611696e-4
"pfib/2",0.30187882572923386,0.3018276229551859,0.3019332328489849,2.7088498930449747e-4,2.3482715228451473e-4,3.21989842698397e-4
"pfib/3",0.3025411692312786,0.3024852743795938,0.3026018276861736,2.975260476849551e-4,2.6907713057828055e-4,3.3200184242397705e-4
"pfib/4",0.3045588889769145,0.3024707928350993,0.31463312775407504,2.0236423610294995e-2,3.5108499279216634e-4,4.8255009076253054e-2
"pfib/8",0.27702674538408,0.27158569247041425,0.2846977105787821,3.2790225595837955e-2,2.5215205278321072e-2,4.6554023411818295e-2
"pfib/16",0.26909642369065967,0.2650135436705178,0.2758063927343913,2.615724515115467e-2,1.8039283932648264e-2,4.0956608339390285e-2
"pfib/32",0.2726816359213419,0.26952718407426557,0.27636309534822184,1.7444576031246224e-2,1.4227057669027803e-2,2.2935079362453085e-2
"pfib/64",0.28010295302186694,0.27646669537339885,0.28658743054185587,2.4301648337333875e-2,1.3360828345387364e-2,3.9546300072208655e-2
"ifib/1",0.4984497705153056,0.4984218493155071,0.4984962406805583,1.8040276986757106e-4,1.242720342516858e-4,2.870577387900989e-4
"ifib/2",0.3603615990332194,0.34868544012818997,0.3741170039824078,6.496311767153333e-2,5.652406802832432e-2,7.448312251646773e-2
"ifib/3",0.31676718384538366,0.3109451881102153,0.32481522709642136,3.4887028192202074e-2,2.744363202200484e-2,4.8060797483303504e-2
"ifib/4",0.31541500956330976,0.30635046393190096,0.32643843323503213,5.109215651574664e-2,4.339074296796736e-2,5.995395740457435e-2
"ifib/8",0.32055945069108693,0.3132960501364299,0.3302640953711101,4.2886292109838166e-2,3.422708481199234e-2,5.4196317284934535e-2
"ifib/16",0.32268204123292654,0.31607536703859057,0.33141186625276287,3.858016286486453e-2,3.0959817731033304e-2,4.801572113400926e-2
"ifib/32",0.32020341069017116,0.3151469627073833,0.32664997250352584,2.92266723352992e-2,2.3352577028246314e-2,3.5913110786596904e-2
"ifib/64",0.3183670273474284,0.3136345234564373,0.3248856058767864,2.834147907711714e-2,2.1803978149062524e-2,3.8088053144371074e-2

I didn t thought the improvement in performance on a single-user would be so dramatic. The execution of the program in a single-user environment used 41% of the time to run in my usual environment. About the comparative results, they were more close to what I d expect than the old ones. fib/fib, pfib/1 and ifib/1 had a very similar execution time. What surprised me was that I was expecting to have the best result in the benchmarks with 2 threads, since I ran the tests in a Core 2 Duo T5750 @ 2.00 GHz with 2 cores. But in pfib the execution time has reduced from 4 threads to 8, and in ifib, from 2 to 3. I can t think of where these improvements come from. Unfortunately the creation of charts is currently broken in GHC 6.12, so no graphics by now.

A year ago, i wrote about how New York City's Summer Youth Employment Program (SYEP) requires the use of Internet Explorer to apply online (and it even appears to require IE just to download the PDF of the application!) Sadly, the situation has not changed, a year later. Today, I'm writing to Dan Garodnick, Chair of The City Council's Committee on Technology (and the rest of the committee members), Carole Post, Commissioner of DoITT (the city's Department of Information Technology and Telecommunications), and Jeanne B. Mullgrav, Commissioner of DYCD (the Department of Youth and Community Development, which runs SYEP). Here's what i wrote:

For the last two years at least, the DYCD's Summer Youth Employment Program
(SYEP) has been only available to users of Internet Explorer:
 https://application.nycsyep.com/
Internet Explorer (IE) is only made by Microsoft, and is only available for
people running Microsoft operating systems.  Users of other operating systems,
such as GNU/Linux, Macintosh, or others cannot access the SYEP application
process.  Even users of Windows who care about their online security or simply
desire a different web browsing experience might prefer to avoid Internet
Explorer.
Not only is the online form inaccessible from browsers other than IE, even
retrieving a copy of the PDF to print out and fill in manually is unavailable
for web browsers other than IE.
What is the city's policy is on access to government sites?  Is it city policy
to mandate a single vendor's software for access to city resources?  Should NYC
youth be required to purchase software from Microsoft to be able to apply
for the Summer Youth Employment Program?
The sort of data collection needed by such an application is a mainstay of the
standards-based web, and has been so for over 15 years now.  There is no reason
to require particular client on an open platform.  I can point you toward
resources who would be happy to help you make the system functional for users
of *any* web browser, if you like.
I raised this issue over a year ago (see nyc.gov correspondence #1-1-473378926,
and a public weblog posted around the same time [0]), and got no effective
remedy.  It's worrisome to see that this is still a problem.
Please let me know what you plan to do to address the situation.
Regards,
	--dkg
[0] https://www.debian-administration.org/users/dkg/weblog/47

Feel free to send your own message to the folks above (especially helps if you live in or near NYC) Finally, Carole Post, the head of DoITT will also be present at a panel tonight in Soho, which i'm unfortunately be unable to attend. If you go there, you might ask her about the situation.Tags: policy

Digging into the fabulous world of Emacs and Lisp, I wanted to use it to build my personal Web site and my blog. I already moved from ikiwiki to Emacs Muse for my HTML pages some weeks ago. Muse provides an extension to maintain a journal, called muse-journal. Unfortunately, it was far to fulfill all my needs, and I decided that it would be a good exercise to write a better extension. Consequently, I started to wrote my own extension, which I named muse-blog. And this is now what is used to build this blog. :-)

After more than a year of using ikiwiki to run bzed.de I thought it would be a good time replace the darkish-brown style by something bright. Also I wanted to get righ of the massive changes I had to do on the template files to make the old layout work. Unfortunately I hit one of the - in my opinion - major problems in the ikiwiki templates again: You can't rely on all <div>s being available on all pages, which is quite annoying when you need them to style the page with CSS. So I had to make a tiny change to page.tmpl:

diff --git a/templates/page.tmpl b/templates/page.tmpl
index 8a9911f..dbf78a0 100644
--- a/templates/page.tmpl
+++ b/templates/page.tmpl
@@ -111,8 +111,8 @@
 <TMPL_VAR CONTENT>
 <TMPL_IF HTML5></section><TMPL_ELSE></div></TMPL_IF>
-<TMPL_IF COMMENTS>
 <TMPL_IF HTML5><section id="comments"><TMPL_ELSE><div id="comments"></TMPL_IF>
+<TMPL_IF COMMENTS>
 <TMPL_VAR COMMENTS>
 <TMPL_IF ADDCOMMENTURL>
 <div class="addcomment">
@@ -121,8 +121,8 @@
 <TMPL_ELSE>
 <div class="addcomment">Comments on this page are closed.</div>
 </TMPL_IF>
-<TMPL_IF HTML5></section><TMPL_ELSE></div></TMPL_IF>
 </TMPL_IF>
+<TMPL_IF HTML5></section><TMPL_ELSE></div></TMPL_IF>
 <TMPL_IF HTML5><footer id="footer" class="pagefooter"><TMPL_ELSE><div id="footer" class="pagefooter"></TMPL_IF>
 <TMPL_IF HTML5><nav id="pageinfo"><TMPL_ELSE><div id="pageinfo"></TMPL_IF>

Everything else is a bit of CSS in local.css and some images. There are still various things which could be optimized, but there are more important things to do now :).

I wrote a bit ago a blog post about what debian kde is missing of distribution specific things Some of the more important things includes:

• A Qt based debian installer
• Tools to manage 3rd party modules and firmware and such
• Debconf frontend that fits in
• Report bug interface

Installer
I don t think it is the most important thing. Wether or not the graphical installer is using gtk or qt is not that important. I would love to see it happen, but it is not something I feel like putting my time in. Others are most welcome.
It will give the advantage of giving the installer the possibility to use the framebuffer directly. Modules and firmwares and such
Someone is saying that ubuntu has something called jockey that does this exact thing, with a KDE and a Gnome frontend. Unfortunately, it is python, so it is something I will really avoid. I m hoping that me mentioning it here will make someone into python&debian pick it up and bring it to debian.
It is apparantly some nice magic around discover-data. Debconf frontend that fits
A lot of work has been put into proper perl-qt bindings and they will hopefully be ready for kde4.5, which is unfortunately a series too late for Squeeze :/. But when that has happened, we just need some perl guy to adapt the old frontend to debconf. Reportbug interface
There is already a tool called reportbug-ng that is a qt interface to reporting bugs. All in all, it looks like we are quite far already. We just need to get the last bits put together. Someone: pickup jockey. And note that the comment field isn t a place to report bugs. They will be removed.

Recently, I ve been working on an exciting project with JaNet(UK) on a project to bring federated authentication to non-web applications. I ve worked on authentication projects a lot, although this is the first federation project I ve worked on. The big difference appears to be an emphasis on credential independence: the subject (person trying to authenticate) and service will not share a common credential type. Within their organization, the subject and their identity provider share a credential. Then, the federation has some credential mechanism such that the user s organization and the service share some (probably completely different) type of credential. The other emphasis is on providing personalization. For web applications, there are a lot of options to achieve this: Information Card, Open ID, SAML, and OAuth all provide solutions in this space. However there are not good options for non-web applications. If you out-source your mail and chat infrastructure but want to use your own chat client or IMAP client, then you will not get the same federation benefits you can get with the web. If you re using usernames and passwords and don t mind the potential problems with your out-sourcing provider being able to impersonate all your users, you can simply synchronize usernames and passwords. Within an enterprise, you can do better using Kerberos. JaNet(UK) runs the UK Access Federation, which is a SAML-based web single-sign-on federation. In order to better meet the needs of their customers they d like to expand this offering to non-web applications. This demand is apparently shared across the European academic community. I suspect there is also some demand in the US academic community and in enterprise situations. With the web, it turns out that you have a convenient platform for interactions with the identity provider: you can simply direct a web browser to the identity provider and need not specify the user interaction with the web browser at all. This is seen as a significant branding and usability advantage. With other environments, it becomes necessary to specify the interaction with the identity provider. Consider an automated client that wishes to examine a mail box and provide advanced mail sorting or aggregation. That automated client cannot directly use a web browser. OAuth solves this issue with an enrollment step that does typically involve a web browser that produces a consumer key and an authentication step that does not. However for non-web clients it seems like avoiding reliance on the browser authentication will be important. It turns out we already have widely used technologies that do this: the Extensible Authentication Protocol (EAP) mediates the interaction between a subject and identity provider for obtaining network access. It also turns out that we have fairly good technologies for abstract security services within non-web applications: thanks to Kerberos and Active Director, many application protocols and a fair number of applications support GSS-API. JaNet(UK) proposes to combine these technologies with SAML in order to produce a solution for federation beyond the web. I prepared a feasibility analysis of this proposal. At a technical level, the proposal is sound. There s a lot of standardization and implementation work, but there appears to be sufficient motivation to form the seeds of a standards activity and put together a proof-of-concept implementation. However, the big question is Will anyone use it? In particular, to be useful beyond fairly small communities, support from client vendors and application framework vendors will be needed. It s taken massive money and around 20 years to get Kerberos support to a point where it is effective within an enterprise. Moonshot can leverage that work to a large extent, but moonshot may also have greater usability and penetration goals. It s interesting that I m advocating EAP for application layer authentication. When I was a Security AD, I made a strong statement that EAP must only be used for network access. I ve been fairly consistent about that since then. I think there are two huge problems with using EAP for application authentication. The first is that EAP only authenticates the home realm; it does not authenticate what service you re going to. So you might try to connect to your e-mail and end up giving something access to your stored files and pictures instead. That is, EAP has a phishing exposure in the federated context. If the only thing you can get by using EAP is network access, that exposure is only moderate. However in a fully federated environment that is a huge exposure. Moonshot will address this problem by using EAP channel binding and by doing the necessary work to make that a viable solution. The second concern is that interoperability is reduced when you have multiple authentication approaches for the same problem. If EAP is going to be used for application authentication, we need to understand how it relates to the rest of the application authentication metasystem. Moonshot proposes such a relationship, addressing my objection. Moonshot is designed to work well with the objectives of the Identity Metasystem and its laws of identity. It uses a different technology, but does have an approach for dealing with claims-based identity and hopefully will have a user experience very similar to the identity metasystem. It uses a different underlying technology. However one of the main beliefs behind the identity metasystem is that is the user experience and universal interoperability that is important, not any specific technology. In its domain, the technologies Moonshot selects will be a better fit than a web services stack. It s strange not to be working on Kerberos; Moonshot uses some Kerberos technology, but its core is definitely not today s Kerberos. In some ways it is fun to be working on something new. There s one aspect of Kerberos I really miss: Moonshot has nothing like tickets. There s no place to remember state or exchange to directly involve the client in what the server learns. My analysis talks about ways to make Moonshot more like Kerberos; there are some potential advantages, but so far, the tradeoffs do not justify changes. We re hoping to have a bar BOF at IETF 77 and a BOF in the summer at IETF 78.

I find it surreal to have people guessing at my motivations when they could just ask me. On top of all that, I find it weird that people spend so much time with in-fighting. I just want my system not to suck. Some time ago (during in the Ubuntu Karmic development cycle maybe in September 2009), I started having giant problems with my build system. All I/O would start to stall, wait times would surge, and usually my entire system would just go unresponsive with the disk light on solid. This scared the crap out of me, and it wasn t entirely obvious what was triggering it. No one else seemed to be seeing it. I managed to start tracking things using latencytop , and saw stuff like liferea going crazy. As I eliminated more and more things, I eventually settled on it being a problem with umount, and I reported an Ubuntu bug. It seemed to look like an upstream bug that no one else but the reporter could reproduce either. Since no one else was seeing this issue, and it seemed related to LVM snapshots, I migrated off of snapshots, and started using aufs overlays for my builds. For a while, it seemed like things went away. It didn t, and I started hitting it again. I opened a new (now famous) bug in Ubuntu, since now snapshots weren t in the picture, and I didn t want to confuse the earlier history. I managed to find a relatively minimal test-case too. A few other people commenting on the bug were seeing the problem now too, but it was less pronounced for them. As an aside, this wasn t a just wait a few seconds longer kind of issue on my system. A single umount would last 30-40 minutes. And when I m doing parallel builds of security updates, this would turn into my system being unavailable for hours at a stretch. Since none of the kernel developers I was in contact with were able to track down the root cause, I asked Ted Ts o in email if he could just quickly peek in on this for me, since I figured he d be in a good position to confirm or deny it. I didn t want to start wasting upstream time with this if it wasn t reproducible (see earlier upstream kernel bug). To my great relief, Ted found a few minutes to check it, and was able to immediately confirm it and give me a viable work-around ( sync; umount ) for the time being. I confirmed the work-around, and went off to do other things. A while later, Ted came back to deliver a bit of a rant, the purpose of which was not clear to me, but I ultimately ignored it I didn t seem directed at me. I just wanted my system operating normally, and he d done me a favor to check in on it and got me a work-around. More time passes, and I eventually get caught in another I/O-wait melt-down. On investigation, it seemed that the in-kernel work-around in the Ubuntu kernel totally back-fired on me in some cases, rendering even the user-space work-around useless. After investigating the Ubuntu-specific work-around, I re-read Ted s rant in the course of researching what had happened during this bug s triage. It seemed that Ted was basically saying:
this is an upstream problem
RedHat hasn t run into it and he didn t know why I figured I should confirm for myself if Fedora was affected, so downloaded and installed Fedora to double-check there. Since I was able to reproduce it there, I opened an upstream bug, linking back to the original Ubuntu bug, and then went to open a bug in the Fedora tracker, linking back to upstream. And it seems to be these actions that everyone has jumped on. I will now bore you with the reality of my motivations: I wanted to fix the bug so no one would end up experiencing the same pain I d been through over the last 6 months. The bug was, from my perspective, a serious issue. Since I d managed to reproduce it in another distro, it was my duty as a Free Software developer to report it to them. And, in what I felt was an unambiguous gesture, I made sure to include the link to the upstream kernel bug. Reproducing it in Ubuntu, in Fedora, and with a stock kernel had me confident that it was an upstream issue. While Ted did correctly suspect the issue was upstream, I really didn t want to just open an upstream bug and have it be ignored. I wanted some additional proof of reproduction, which I got when I tested it on Fedora. So, I m rather saddened that so many people spent so much time questioning my motivations, making fun of Canonical, or doing anything other than trying to just simply solve this problem. I m totally disinterested in inter-distro fighting. Instead, I continue to assume we re all on the same team, fighting a philosophical battle against close-source software. And in that regard, I think it s still true. If I ignore the rants and jeering, I come away thankful for all the people that spent time trying reproduce the issue at Canonical, at RedHat, and in the larger community. I m hugely thankful that Ted made some time to let me know I wasn t crazy, and there was actually a problem. I m thankful for having some work-arounds, and I m thankful that the root cause was eventually ferreted out, with some possible solutions. I m even thankful that some people on the LWN thread saw that, far from malicious, I was trying to be helpful with the bug. I just wanted my filesystem not to eat my computer. And I was hoping other people could maybe help me, since I m not a filesystem expert. The drama around this bug is pathetic, and now by talking about it for almost 1000 words, I m just as guilty.

I recently read an introductory article in an IT magazine on the increased usage of new currencies. It briefly mentioned the work of a researcher called Arthur Brock in using those currencies within open source communities. While I did not find more details on the research, it led me to think again about Debian s relation with money. The reason why we have troubles using money to pay the time spent by our developers is that money is scarce and it s thus next to impossible to be fair in the way money is spent among us. So why not invent a new currency that is not scarce and that would encourage the kind of work that we really value? Apparently there is free software out there to build new currencies: see metacurrency.org or openmoney.org. Let s call the new currency swirly/swirlies for the sake of the examples below. There are then multiple ways to create a small economy within the community and/or even create bridges with the national currencies:

• we could have auctions (priced in swirlies) to redistribute goods among us (say I have this unused laptop and I want to give it away to someone who could make use of it within Debian)
• Debian sponsors/partners could offer discounts codes on their products and Debian would exchange them against the Debian-specific currency (inspired by community way);
• swirlies could be used to get funding to attend debconf and/or other meetings;
• we could donate our swirlies as bounties on important projects that we want to see implemented, we could even grant swirlies to release managers to help them drive the project towards a release (they would not end up in their accounts but they could use them to motivate people to work on release blockers);

I m sure we can come up with many similar ideas. Feel free to share yours in the comments. Partagez cet article / Share This

I spent last week at the Ubuntu Developer Summit in Belgium, where we kicked off the 10.10 development cycle. Due to our time-boxed release cycle, not everything discussed here will necessarily appear in Ubuntu 10.10, but this should provide a reasonable overview of the direction we re taking. Presentations While most of our time at UDS is spent in small group meetings to discuss specific topics, there are also a handful of presentations to convey key information and stimulate creative thinking. A few of the more memorable ones for me were:

• Mark Shuttleworth talked about the desktop, in particular the introduction of the new Unity shell for Ubuntu Netbook Edition
• Fanny Chevalier presented Diffamation, a tool for visualizing and navigating the history of a document in a very flexible and intuitive way
• Rick Spencer talked about the development process for 10.10 and some key changes in it, including a greater focus on meeting deadlines for freezes (and making fewer exceptions)
• Stefano Zacchiroli, the current Debian project leader, gave an overview of how Ubuntu and Debian developers are working together today, and how this could be improved. He has posted a summary on the debian-project mailing list.

The talks were all recorded, though they may not all be online yet. Foundations The Foundations team provides essential infrastructure, tools, packages and processes which are central to the development of all Ubuntu products. They make it possible for the desktop and server teams to focus on their areas of expertise, building on a common base system and development procedures. Highlights from their track:

• Early on in the week, they conducted a retrospective to discuss how things went during the 10.04 cycle and how we can improve in the future
• One of their major projects has been about revision control for all of Ubuntu s source code, and they talked last week about what s next
• We re aiming to provide btrfs as an install-time option in 10.10
• In order to keep Ubuntu moving forward, the foundations team is always on the lookout for stale bits which we don t need to keep around anymore. At UDS, they discussed culling unattended packages, retiring the IA64 and SPARC ports and other spring cleaning
• There was a lot of discussion about Upstart, including its further development, implications for servers, desktops and the kernel, and the migration of server init scripts to upstart jobs
• After maintaining two separate x86 boot loaders for years, it looks like we may be ready to replace isolinux with GRUB2 on Ubuntu CDs

Desktop The desktop team manages both Desktop Edition and Netbook Edition, on a mission to provide a top-notch experience to users across a range of client computing devices. Highlights from their track:

• A key theme for 10.10 is to help developers to create applications for Ubuntu, by providing a native development environment, improving Quickly, improving desktopcouch, making it easier to get started with desktopcouch, and enabling developers to deliver new applications to Ubuntu users continuously
• With more and more touch screen devices appearing, Ubuntu will grow some new features to support touch oriented applications
• The web browser is a staple application for Ubuntu, and as such we are always striving for the best experience for our users. The team is looking ahead to Chromium, using apport to improve browser bug reports, and providing a web-oriented document capability via Zoho
• Building on work done in 10.04, we will aim to make simple things simple for basic photo editing
• Security-conscious users may rest easier knowing that the X window system will run without root privileges where kernel modesetting is supported

Server/Cloud The server team is charging ahead with making Ubuntu the premier server OS for cloud computing environments. Highlights from their track:

• Providing more powerful tools for managing Ubuntu in EC2 and Ubuntu Enterprise Cloud infrastructure, including boot-time configuration, image and instance management, and kernel upgrades
• Improving Ubuntu Enterprise Cloud by adding new Eucalyptus features (such as LXC container support, monitoring, rapid provisioning, and load balancing. If you ever wanted to run a UEC demo from a USB stick, that s possible too.
• Providing packaged solutions for cloud building blocks such as hadoop and pig, Drupal, ehcache, Spring, various NOSQL databases, web frameworks, and more
• Providing turn-key solutions for free software applications like Alfresco and Kolab
• Making Puppet easier to deploy, easier to configure, and easier to scale in the cloud

ARM Kiko Reis gave a talk introducing ARM and the corresponding opportunity for Ubuntu. The ARM team ran a full track during the week on all aspects of their work, from the technical details of the kernel and toolchain, to the assembly of a complete port of Netbook Edition 10.10 for several ARM platforms. Kernel The kernel team provided essential input and support for the above efforts, and also held their own track where they selected 2.6.35 as their target version, agreed on a variety of changes to the Ubuntu kernel configuration, and created a plan for providing backports of newer kernels to LTS releases to support deployment on newer hardware. Security Like the kernel team, the security team provided valuable input into the technical plans being developed by other teams, and also organized a security track to tackle some key security topics such as clarifying the duration of maintenance for various collections of packages, and the ongoing development of AppArmor and Ubuntu s AppArmor profiles. QA The QA team focuses on testing, test automation and bug management throughout the project. While quality is everyone s responsibility, the QA team helps to coordinate these activities across different teams, establish common standards, and maintain shared infrastructure and tools. Highlights from their track include:

• There was a strong sense of broadening and deepening our testing efforts, mobilizing testers for specific testing projects, streamlining the ISO testing process by engaging Ubuntu derivatives and fine-tuning ISO test cases, and reactivating the community-based laptop testing program
• In support of this effort, there will be projects to improve test infrastructure, including enabling tests to target specific hardware and tracking test results in Launchpad
• There is a continuous effort to improve high-volume processing of bug reports, and two focus areas for this cycle will be tracking regressions (as these are among the most painful bugs for users) and improving our response to kernel bugs (as the kernel faces some special challenges in bug management)

Design The design team organized a track at UDS for the first time this cycle, and team manager Ivanka Majic gave a presentation to help explain its purpose and scope. Toward the end of the week, I joined in a round table discussion about some of the challenges faced by the team in engaging with the Ubuntu community and building support for their work. This is a landmark effort in mating professional design with free software community, and there is still much to learn about how to do this well. Community The community track discussed the usual line-up of events, outreach and advocacy programs, organizational tools, and governance housekeeping for the 10.10 cycle, as well as goals for improving the translation of Ubuntu and related resources into many languages. One notable project is an initiative to aggressively review patches submitted to the bug tracker, to show our appreciation for these contributions by processing them more quickly and completely.