Since my new laptop features shiny Intel graphics (and not the proprietary stuff from ATI or Nvidia), I can comfortably run compiz on it. None of my ATI or Nvidia machines did. Face it: compiz is a window manager. It does that job okay. I somewhat still like openbox better, but compiz has it's bright sides, too. I'll probably add devilspie again, though. Too bad they removed the XML syntax and went to a lispy thing. Openbox had a nice XML syntax and had all the window matching functions I needed. Now most people expect compiz to just magically add some 3D effects to their desktop. That's not entirely accurate. 3ddesktop worked like that, years ago. It worked with most window managers, by taking screenshots of your desktops. Maybe some of it's layouts could be added to compiz. Compiz however functions different, and thats why it IS a window manager in itself (which has "outsourced" window decoration to e.g. gtk-window-decorator to achieve a metacity-like look). From my understanding, it basically has all windows placed off-screen and uses GL to draw them on the real screen. This will result in some speedups, actually, whereas 3ddesktop came only with slowdowns. Anyway, so much for the theory. Here are my firsthand experiences:

• Performance is excellent on my i945 integrated graphics
• 3D drawing errors are minimal (playing with the cube settings and increasing the number of desktops to 8 caused some that eventually went away)
• Effects are much less annoying than I though. Some are actually quite pleasant, I find the "fading" visual bell useful. Cube switching at 8 desktops is more of a slide, and with a faster speed it's also nice, at least for adjacent desktops. I'd like to have a similar smooth transition for arbitrary desktops; no need to flash me the interim desktops.
• Expose-like overview is of course great. Except I use desktops to avoid any window overlap anyway.
• Wobbly windows still rather annoy me. I tried tuning them down to a faster and lighter setting, but failed. Any other parameters just made things worse, I found it hard to pick good parameters.
• the bad: every now and then, a window isn't drawn properly. Most often these are dialog boxes; there is no obvious pattern to when it works and when not. Today, the gnome-screensaver password prompt is invisible. Other dialogs are empty until minimizing them with the "show desktop" button and restoring them.

On overall, compiz doesn't distract from work that much; I had feared I would spend a lot of time just playing with the effects, but it's okay. (Try setting visual bell to shiver, start vim and press arrow up some time, and watch your window violently shake. Reduces stress when not finding a bug.) So it likely is less harmful for the Debian etch release than Frozen Bubble 2.0 (which has a new 6-player mode, and is feared to have a similar impact as tetrinet...) Whoever released frozen bubble 2.0 just now must have some evil plan of delaying the etch release. [Yes, I'm aware of this compiz fork; but I'm sticking to Debian packaged software, so my system will upgrade itself. I don't think it's packaged yet?]

I am familiar with the Xanadux project which is working on porting Linux to various WinCE based smartphones. However I'm not aware of anyone doing the same for Nokia or SonyEricsson phones. Are there no such projects or have I just not found them? I'd have thought phones running Series 60 were likely candidates to have other neat stuff ported. Is this not being done because it's harder to get into Symbian, or is it not being done because Symbian is much more functional than WinCE?

I don't often post about work, but it's always with some trepidation that I switch my main computers to the new distro.

While I moved from the Distro team to leading the Support team last October, I still retain my Ubuntu core-dev status, and keep current by looking at toolchain issues occasionally and hacking on our SCC ports, ia64 and hppa. This means that not only do I keep my programming sk1llz from getting rusty through disuse, but I also have reason to keep in touch with other members of the distro team who are both work colleagues and friends.

As one of the people who cares the most about glibc in Ubuntu, I try to make sure that I'm running it wherever possible and making sure that I'm likely to find at least the obvious bugs before others.

So last night I pointed my sources.list to feisty.

This morning there were already two updates: base-files, and debootstrap. linux-libc-dev (the kernel headers) will come along shortly after the new kernel doesn't FTBFS. At the beginning of next week, there will be binutils, libc*, and gcc-4.1. And after that, the merge.

This will be the fourth release in a row that I've ridden from the first files uploaded. At times it's been exciting (leaving for a management meeting in Spain with a non-working X or wireless, not to mention broken cell phone and airplane. All of which were solved by the time I got to Mallorca.)

Wouter Verhelst kindly replied to my comments on his calculation of Steve's net pay :

"The point wasn't getting a "correct" number; only Steve could produce those, if he wanted to. The point was proving that if you get 6000 for one month of work as a business, then those 6000 aren't what you can spend on your personal life. Whether the exact number is 2400 or 3400 is, thus, totally besides the point." "But, yeah, thanks for the clarification :-)"

I agree that 6000 gross works out as a lower net figure. So what? That's obvious to anyone who works and has noticed their gross pay isn't what they actually get to use, even if these "people have never had to run a business". (And I think running a business is a choice, not something anyone ever "had to" do.) If whether it's 2400 or 3400 net wasn't the point, why use inflated figures to advocate paying 6000 gross? My main point was: Dunc-Tank is not buying in a free market, so how can we tell what the fair rate is? Brian M. Carlson explained the lack of VAT:

"States in the US have either an income tax, a sales tax, or both. In the general case, non-profit corporations are exempt from taxation, as long as they have the proper paperwork. Also, certain essentials, such as necessary foods (cheese, but not soda) are exempt from sales tax where it exists." "Since Steve is almost certainly not a non-profit, he will have to pay income tax to both the state and federal governments; the rate depends on his annual income. He will have to pay FICA (social security) taxes, which are approximately 7% of the income. Any amount over $90,000 is not taxable under FICA."

Thanks for the explanation. I hope it'll help other non-US people. Of course, the actual payment rate is minor to me, compared to the problems of dunc-tank being designed as a market for lemons and being driven through by brinkmanship.

I woke up this morning and found the following scene outside the window: It’s first snow and I think it is on time. First snow rarely lasts, though; it is likely this snow will have melted away by midweek, if not earlier. Permanent (well, one that lasts until spring) snow generally comes a month later. Last year permanent snow came just before Christmas.

Tweaking my dspam setup I was recently disappointed with my dspam setup: while it was catching lots of spam and it would never misclassify ham, I would still get like 50 spams a day in my inbox. In a moment of enlightenment, I realised that it could be because I'm subscribed to loads of mailing lists many of which I read only every so often. This means that I don't feed dspam those spam messages that manage to go through those lists, and so dspam gets to figure out that they are innocent, and therefore ends up with too broad an idea of innocence. The solution was heroically simple: plug dspam only after sorting mailing lists in their own folders, and only on the remaining personal mail. Not so simple was the implementation, since I had dspam integrated with postfix and it would necessarily process all the mail I get. This is what I did:

1. I removed dspam hooks from postfix
2. I hooked dspam into procmail:

   :0fw
     /usr/bin/dspam --user=enrico --stdout --deliver=innocent,spam
   :0
   * ^X-DSPAM-Result: spam
   .spam/
   

3. I figured out that dspam run from postfix in that way would not make use of the server, not even if I use --client. I thus disabled the dspam server in /etc/default/dspam. This is a shame, but luckily I don't have many users and the server load won't suffer much.
4. I fixed the privileges under /var/spool/dspam so that the users can read their dspam data but not the data of other users. Here's what I conjured:

   drwxrwxr-x dspam  dspam /var/spool/dspam/
   drwxrwxr-x dspam  dspam /var/spool/dspam/data
   drwxrwxr-x dspam  dspam /var/spool/dspam/data/local
   drwxrwx--- enrico dspam /var/spool/dspam/data/local/enrico
   drwxrwx--- enrico dspam /var/spool/dspam/data/local/enrico/enrico.sig
   -rw-rw---- enrico dspam /var/spool/dspam/data/local/enrico/enrico.*
   drwxrwx--- pippo  dspam /var/spool/dspam/data/local/pippo [...]
   drwxrwx--- pluto  dspam /var/spool/dspam/data/local/pluto [...]
   drwxrwxr-x dspam  dspam /var/spool/dspam/opt-in
   drwxrwxr-x dspam  dspam /var/spool/dspam/opt-in/local
   -rw-rw-r-- dspam  dspam /var/spool/dspam/opt-in/local/enrico.dspam
   -rw-rw-r-- dspam  dspam /var/spool/dspam/opt-in/local/pippo.dspam
   -rw-rw-r-- dspam  dspam /var/spool/dspam/opt-in/local/pluto.dspam
   -rw-rw---- dspam  dspam /var/spool/dspam/system.log
   

   the opt-in files are actually empty files and dspam seems to only stat() them, so they are probably ok even if they're not readable by users. This setup of permissions should allow dspam to run as the various users while fully preserving their privacy towards each others.
5. In /etc/dspam/dspam.conf I commented out the Delivery options, since I let procmail take care of the delivery. I'm also not using ParseToHeaders anymore. The rest of the dspam configuration did not change.
6. Instead of ParseToHeaders, I set up postfix aliases properly running dspam as the dspam user. The trick is to create an alias file owned by the dspam user and the dspam group, and tell postfix to use it:

   alias_maps = hash:/etc/aliases, hash:/etc/postfix/aliases-dspam
   alias_database = hash:/etc/aliases, hash:/etc/postfix/aliases-dspam
   

   /etc/postfix/aliases-dspam in turn contains:

   spam-enrico:" /usr/bin/dspam --user enrico --class=spam --source=error"
   notspam-enrico:" /usr/bin/dspam --user enrico --class=innocent --source=error"
   spam-pippo:" /usr/bin/dspam --user pippo --class=spam --source=error"
   notspam-pippo:" /usr/bin/dspam --user pippo --class=innocent --source=error"
   spam-pluto:" /usr/bin/dspam --user pluto --class=spam --source=error"
   notspam-pluto:" /usr/bin/dspam --user pluto --class=innocent --source=error"
   

   since I took care of leaving everything in /var/spool/dspam writable by the dspam group, this works perfectly. It also allows me to bounce a mail to the report address instead of forwarding it. With ParseToHeaders I had to forward because the bounced messages would not have the report address in their headers, and so dspam could not identify them. By the way, here's the mutt macro I use to report spam:

   macro index,pager <F12> "bspam-enrico@xxx.xx\n\ns=.spam\n" 'Report as spam'
   

This is it. It took a while as dspam isn't always straightforward to set up, but now I've finally stopped having to remove 20 spam messages from my inbox for every 1 legitimate mail I receive.

It is currently rumored that, despite great efforts from the release managers and the dunc-tank board, the etch release will not make it on time. Some say this is because of Developers being upset by some random and obscure experiment. Some others think this is because there are too many RC bugs and that we won't be able to fix them on time. Some might say this is because of a surrealist conflict with the Mozilla corporation. Guys. You're all wrong. The main reason for the delay will be the same as that of the woody release. Developers will be too busy testing the brand new Frozen-bubble 2.0.0 release extensively. Instead of hacking, they will spend their nights fighting with each other in up to five players LAN and internet games, enjoying the shining new graphics. Debian unstable packages are available in incoming and tonight on your favorite mirror.

Telling gpg not to use the key in the card So, I created the subkeys for the OpenPGP card, and it works. Now I'd like to upload some Debian packages, but the uploads fail because my new subkeys aren't yet known to the Debian keyring. I tried to push my subkeys to keyring.debian.org, but uploading afterwards still was rejected. Maybe it takes some time for propagation, maybe there's some other procedure to follow, I don't know. I didn't manage to figure out what is the procedure for getting a new subkey in the Debian keyring. I wish to replace this paragraph with proper details if I'll ever find out. Now, failing to use the subkeys, I had to convince gpg to use my good old main key. The quick and dirty way was to make a backup of the keyring, delete the subkeys, sign and upload. Seconds after hours of searching terminated in the above crude hack, as it normally happens, someone (Holger in this case) suggested the correct way to do it: use --default-key and append an exclamation mark at the end of the key ID. This was in the gpg manpage, but nowhere near the documentation of --default-key:

Note that you can append an exclamation mark (!) to key IDs or fingerprints.
This flag tells GnuPG to use the specified primary or secondary key and not
to try and calculate which primary or secondary key to use.

So, now I'm happy:

$ gpg --sign  --default-key '797ebfab!'
You need a passphrase to unlock the secret key for
user: [...]
$ gpg --sign
gpg: signatures created so far: xx
Please enter the PIN
[sigs done: xx]

I spent the past couple of days at LinuxWorld UK, mostly on the Debian stand. I'm not sure I'd go as a punter these days; I don't think a lot changes year on year. However I do think it's good for us to turn up and show our faces so people can come and talk to us and realise we're wonderful people. Plus it's always good to see people I haven't seen in ages. We sold a lot of T-Shirts (even the red one with yellow writing that looks like something Phil would come up with after too much vodka [I bought one of these myself]). We didn't sell a lot of DVDs, but that's been happening since broadband became more pervasive. There were a reasonable number of people who just stopped to chat (shouting "Use Debian, it's great!" at people as they walked by was surprisingly effective). Lots of questions about how we felt about Ubuntu. Not really many about why we hadn't released yet which is a change! I also found a new toy I think I want; the Nokia E70. Steve has one of these and it looks pretty sweet; 3G+WLAN, a keyboard and a screen that, while not that big, manages a pretty good PuTTY session in 80x43 or so. And all of this in something not that much larger than a normal phone. That's appealing. I'd been sort of lusting after an HTC TyTN, but reports of poor reliability have put me off. And Simon has put me off the 9300i by saying it's quite slow as a phone. Finally, courtesy of Trexy, I present a new, work safe, goat sex: And now I will spend the next week sleeping I think.

upstart is a replacement for the init daemon, the process spawned by the kernel that is responsible for starting, supervising and stopping all other processes on the system. The existing daemon is based on the one found in UNIX System V, and is thus known as sysvinit. It separates jobs into different “run levels” and can either run a job when particular run levels are entered (e.g. /etc/init.d/rc 2) or continually during a particular run level (e.g. /sbin/getty). The /etc/init.d/rc script is also based on the System V one (and is in the sysv-rc package), it simply executes the stop then start scripts found in /etc/rcN.d (where N is the run level) in numerical order. Why change it? Running a fixed set of scripts, one after the other, in a particular order has served us reasonably well until now. However as Linux has got better and better at dealing with modern computing (arguably Linux’s removable device support is better than Windows’ now) this approach has begun to have problems. The old approach works as long as you can guarantee when in the boot sequence things are available, so you can place your init script after that point and know that it will work. Typical ordering requirements are:

• Hard drive devices must have been discovered, initialised and partitions detected before we try and mount from /etc/fstab.

• Network devices must have been discovered and initialised before we try and start networking.

This worked ten years ago, why doesn’t it work now? The simple answer is that our computer has become far more flexible:

• Drives can be plugged in and removed at any point, e.g. USB drives.

• Storage buses allow more than a fixed number of drives, so they must be scanned for; this operation frequently does not block.

• To reduce power consumption, the drive may not actually be spun up until the bus scan so will not appear for an even longer time.

• Network devices can be plugged in and removed at any point.

• Firmware may need to be loaded after the device has been detected, but before it is usable by the system.

• Mounting a partition in /etc/fstab may require tools in /usr which is a network filesystem that cannot be mounted until after networking has been brought up.

We’ve been able to hack the existing system to make much of this possible, however the result is chock-full of race conditions and bugs. It was time to design a new system that can cope with all of these things without any problems. What we needed was an init system that could dynamically order the start up sequence based on the configuration and hardware found as it went along. Design of upstart upstart is an event-based init daemon; events generated by the system cause jobs to be started and running jobs to be stopped. Events can include things such as:

• the system has started,

• the root filesystem is now writable,

• a block device has been added to the system,

• a filesystem has been mounted,

• at a certain time or repeated time period,

• another job has begun running or has finished,

• a file on the disk has been modified,

• there are files in a queue directory,

• a network device has been detected,

• the default route has been added or removed.

In fact, any process on the system may send events to the init daemon over its control socket (subject to security restrictions, of course) so there is no limit. Each job has a life-cycle which is shown in the graph below: The two states shown in red (“waiting” and “running”) are rest states, normally we expect the job to remain in these states until an event comes in, at which point we need to take actual to get the job into the next state. The other states are temporary states; these allow a job to run shell script to prepare for the job itself to be run (“starting”) and clean up afterwards (“stopping”). For services that should be respawned if they terminate before an event that stops them is received, they may run shell script before the process is started again (“respawning”). Jobs leave a state because the process associated with them terminates (or gets killed) and move to the next appropriate state, following the green arrow if the job is to be started or the red arrow if it is to be stopped. When a script returns a non-zero exit status, or is killed, the job will always be stoped. When the main process terminates and the job should not be respawned, the job will also always be stopped. As already covered, events generated by the init daemon or received from other processes cause jobs to be started or stopped; also manual requests to start or stop a job may be received. The communication between the init daemon and other processes is bi-directional, so the status of jobs may be queries and even changes of state to all jobs be received. How does it differ from launchd? launchd is the replacement init system used in MacOS X developed as an “Open Source” project by Apple. For much of its life so far, the licence has actually been entirely non-free and thus it has only become recently interesting with the licence change. Much of the goal of both systems appears initially to be the same; they both start jobs based on system events, however the launchd system severly limits the events to only the following:

• system startup,

• file modified or placed in queue directory,

• particular time (cron replacement),

• connection on a particular port (inetd replacement).

Therefore it does not actually allow us to directly solve the problems we currently have; we couldn’t mount filesystems once the “filesystem checked” event has been recived, we couldn’t check filesystems when the block device is added and we certainly couldn’t start daemons once the complete filesystem (as described by /etc/fstab) is available and writable. The launchd model expects the job to “sit and wait” if it is unable to start, rather than provide a mechanism for the job to only be started when it doesn’t need to wait. Jobs that need /usr to be mounted would need to spin in a loop waiting for /usr to be available before continuing (or use a file in a tmpfs to indicate it’s available, and use that modification as the event). This is not especially surprising given that Apple have a high degree of control over both their hardware and the actual underlying operating system; they don’t need to deal with the wide array of different configurations that we have in the Linux world. Had the licence been sufficiently free at the point we began development of our own system, we would probably have extended launchd rather than implement our own. At the point Apple changed the licence, our own system was already more suitable for our purposes. How does it differ from initng? Initng by Jimmy Wennlund is another replacement init daemon intended to replace the sysvinit system used by Linux. It is a dependency-based system, where upstart is an event-based system. The notion of a dependency-based system is interesting to talk about at this point. Jobs declare dependencies on other jobs that need to happen before the job itself can be started. Starting the job causes its dependencies to be started first, and their dependencies, and so on. When jobs are stopped, if running jobs have no dependencies, they themselves can be stopped. It’s a neat solution to the problem of ordering a fixed boot sequence and the problem of keeping the number of running processes to a minimum needed. However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started. Initng uses run levels to ensure this happens, where a run level is a list of goal jobs that should be running in that run level. It’s also not clear how the dependencies interact with the different types of job, a dependency on Apache would need the daemon to be running where a dependency on “checkroot” would need the script to have finished running. Upstart handles this by using different events (“apache running” vs. “checkroot stopping”). Again while interesting, Initng does not solve the problems that we wanted to solve. It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot. A typical example would be that if the only dependency on the job that configures networking is the mount network filesystems job, then should that job fail or notbe a goal (e.g. because there are no network filesystems to be mounted) the result is that network devices themselves will not be configured. You could make everything a goal, and just use the dependencies to determine the order, however this is less efficient than just ordering the existing sysv-rc scripts (which can be done at install time). Another example is that often you simply don’t know whether something is a dependency or not without reading other configuration, for example the mount network filesystems may be a dependency of everything under /usr or may just be a dependency of anything allowing the user to login if it just mounts /home. The difference in model can be summed up as “initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to.” How does it differ from Solaris SMF? SMF is another approach to replacing init developed by Sun for the Solaris operating system. Like initng it’s a dependency-based system, so see above for the differences between those systems and upstart. SMF’s main focus is serive management; making sure that once services are running, they stay running, and allowing the system administrator to query and modify the states of jobs on the system. Upstart provides the same set of functionality in this regard, services are respawned when they fail and system administrators can at any time query the state of running services and adjust the state to their liking. Will it replace cron, inetd, etc? The goal of upstart is to replace those daemons, so that there is only one place (/etc/event.d) where system administrators need to configure when and how jobs should be run. In fact, the goal is that upstart should also replace the “run event scripts” functionality of any daemon on the system. Daemons such as acpid, apmd and Network Manager would send events to init instead of running scripts themselves with their own perculiar configuration and semantics. A system administrator who only wanted a particular daemon to be run while the computer was on AC power would simply need to edit /etc/event.d/daemon and change “on startup” to “on ac power”. What about compatibility? There’s a lot of systems administrators out there who have learned how Linux works already and will not want to learn again immediately, there’s also a large number of books that cover the existing software and won’t cover upstart for at least a couple of years. For this reason, compatibility is very important. upstart will continue to run the existing scripts for the forseeable future so that packages will not need to be updated until the author wants. Compatibility command-line tools that behave like their existing equivalents will also be implemented, a system administrator would never need to know that crontab -e is actually changing upstart jobs. Does it use D-BUS?

“To D-BUS people, every problem seems like a D-BUS problem.” —Erik Troan

The UNIX philosophy is that something should do just one job, and do it very well. upstart’s one job is starting, supervising and stopping other jobs; D-BUS’s one job is passing messages between other jobs. D-BUS does provide a mechanism for services to be activated when the first message is sent to them, thereby starting other jobs. Some people have taken this idea and extended it to suggest that all a replacement init system need do is register jobs with D-BUS and turn booting into a simple matter of message parsing. This seems wrong to me, D-BUS would need to be extended to supervise these services, provide means for them to be restarted and stopped; as well as deal with being process #1 which means cleaning up after children whose parent’s have died, etc. It seems far simpler to arrange for D-BUS to send an event to init when it needs a service to be started, and focus on being a very good message passing system. The IPC mechanism used by upstart is not currently D-BUS because of various problems, however it’s always been expected that even if init itself doesn’t communicate with D-BUS directly, there would be a D-BUS proxy that would ensure messages about all init jobs and events would be given to D-BUS and D-BUS clients could send messages to init to query and change the state of jobs. What is the implementation plan? Because this is process #1 we are changing, we want to make sure that we get it right. Therefore instead of releasing a fully-featured daemon and configuration to the world, we’re developing it in the following stages:

1. Principal development; at the end of this stage the daemon has been implemented and can manage jobs as described.
2. Replacement of /sbin/init while running the existing sysv-rc scripts. This is the shake-down test of the daemon, can it perform the same job as the existing sysvinit daemon without any regressions?
3. /etc/rcS.d scripts replaced by upstart jobs. These consitute the majority of tasks for booting the system into at least single-user mode, and contain many of the current ordering problems and race conditions. If the daemon solves the problems here, it will be a success.
4. Other daemon’s scripts replaced by upstart jobs on a package-by-package basis; this will be an ongoing effort during which upstart will continue running the existing sysv-rc scripts as well as its own jobs. During this time the event system may be tweaked to ensure it truly solves the problems we need.
5. Replcement of cron, atd, anacron and inetd. This will happen alongside the above and result in a single place to configure system jobs.
6. Modification of other daemons and processes to send events to init instead of trying to run things themselves.

The current plan is that we will be at least part of the way into stage #3 by the time edgy is released, with that release shipping with upstart as the init daemon and the most critical rcS scripts being run by it to correct the major problems For edgy+1 we hope to have completed stage #5 and be at least part of the way into the implementation of stage #6. From the start of development of edgy+2, no new packages will be accepted unless they provide upstart jobs instead of init scripts and init scripts will be considered deprecated. What state is it in now? The init daemon has been written and is able to manage jobs as described above, receiving events on the control socket to start and stop them. This has now been uploaded to the Ubuntu universe component in the upstart package for testing before it becomes the init daemon. We welcome any experienced users who want to help test this; install the package and follow the instructions in /usr/share/doc/upstart/README.Debian to add a boot option that will use upstart instead of init. If your system boots and shut downs normally (other than a slightly more verbose boot without usplash running) then it is working correctly. Other types of events will be added as required during development and testing. Currently only a basic client tool (initctl) has been written, compatibility tools such as shutdown will be written over the next week or two before it replaces our sysvinit package.

Apologies to anyone reading my blog on a Planet for the sudden dump of old posts from my blog. In fact, as it’s already been pointed out, I should apologise twice as much since I wrote Planet itself and am doubly-responsible for the flood. In order to satisfy the suggestions from the like of Jono Bacon and Matthew Revell and actually blog more about the things that I do, I decided it was time to try some new blog software. Until now I’ve been using PyBlosxom, and while it’s popular with the geek crowd, I’ve never really got on with it. The principal problem is that it’s a pain to actually add blog entries to it, or to manage those that you’ve added previously. I’m slightly ashamed to say that I wanted something with a web interface. Thom May pointed me at Typo which seems pretty neat so far, and it doesn’t require PHP which can only be a good thing! So now when I don’t post, I won’t be able to blame the tools. Damn. Maybe this wasn’t such a good idea, after all?

Or Dunc-tank in sorted order. I never wrote about this until now. But today, when I woke up, I had this strange feeling saying "jd, you gotta do something about DT today.". Actually, that was just my f**** alarm clock. And here I am, and I'm gonna talk to you about the dunc-tank experiment. First it's important to know what a real dunk tank is, because personally I've got no idea until I looked up on Google. So, you see? A dunk tank is a funny game finally. I must admit, if Anthony Towns was seated above the dunk, I would have vote in favor of DT!
Maybe I'm trying to give you some idea for the next DebConf, who knows More seriously, after the co-signed mail that Joerg Jaspert just sent, I wanted to say something more.
Some weeks ago, when our DPL (amen) asked us our opinion about that idea, I said "yeah, go ahead". Of course, that was my opinion at this time, and it changed a little. I still don't have objections about people getting paid, because I'm not jealous.
What does that mean? This means that if Debian was only composed of me, Anthony, and two release managers, I wouldn't have any problem. But that's not the case here, I discovered that there is a lots of people in Debian. Maybe should I tell DT people about that, don't you think? Reality is obviously different, and DT has been launched without a real consensus. We lost valuable people, and I dislike this project for that. We lost time. And we lost money, because time is money (don't beat me). That's an organised foutage de gueule ^[1] . But please, remember this is an experiment, so kids: don't do this at home.

Uses of Debian in Venezuela I'm just back from the IV Foro Mundial de Conocimiento Libre that happened in Matur n, Venezuela from the 17th to the 21st of October. Besides it being a great event and a great opportunity to meet great people (here are photos taken by Ana), it gave me the chance of seeing a bunch more uses of Debian that I didn't know about. Jos Parrella kindly provided me with a list that should open a short series of blog posts:

• SAPI - Servicio Autonomo de la Propiedad Intelectual (the Patents Office which will hold all of the Cuban and Venezuelan patents) uses Debian in all of their servers, including the HP StorageWorks 9.5 TB. They are listed in Who's using Debian.
• MCT - Ministerio de Ciencia y Tecnolog a (the Ministry of Science and Technology), is on charge of the migration process to Free Software. All the decision-makers uses at least dual boot, but most of them use exclusively Debian. At the conference I had a pleasant conversation with one of their representatives, discussing various aspects to keep in mind when choosing a piece of Free Software to use.
• PNAT, the National Plan for IT Literacy, uses Debian (or Ubuntu) in all laboratories around the country.
• CONATEL (the local FCC) uses Debian in servers and Internet Access Points.
• PDVSA (the national oil company) uses Debian in some areas, most notably they've setup videoconferences using OpenH.323 or MBONE using Debian in the PDVSA side.
• FONDAFA (the government fund for agricoltural development), uses Debian in their servers.
• SECODENA, the local Pentagon, uses Debian in their servers.
• DGIM, the local CIA, also uses Debian. Which is kind of creepy. The website runs on ASP, but they have a complete PDC/BDC and Mail Servers using Debian Sarge.
• EDELCA, an energy company in the southern states of the country, is implementing Debian right now. They have several hundreds of desktops already running Debian, and they're migrating network services (LDAP, Mail servers, etc.). There are two huge hydroelectric dams they operate.

More to come as I get back to other people I've met there.

or How Mozilla Corporation is FUDing on Debian. Again. This time, Christopher Beard, Mr Mozilla Marketing is talking about Firefox in Ubuntu, that it’s great, Ubuntu is a good boy, that it cooperates with Mozilla , and that by cooperating, it can use the great Firefox name and logo. To be honest, I don’t care that Ubuntu calls Firefox Firefox instead of Iceweasel or whatever. I don’t care that Ubuntu doesn’t care about freedom as much as Debian does. I could not care less. But, indeed, I do care about (yet other) false claims about Firefox in Debian. Let’s see what Christopher has to say about it:

I understand that Ubuntu is based upon Debian. Is that the same or different than the IceWeasel browser that Debian is shipping with their latest release?
It’s different. The patches that Debian applied to the Mozilla source code (which then resulted in their IceWeasel product) are more significant in scope than those in what Ubuntu is shipping (and branding as an official Mozilla Firefox release). Firefox in Ubuntu represents a somewhat more modest set of divergences from original Mozilla source code.

Let’s check out what it is about. And as I don’t want people to doubt my words, I’ll give you, readers, the ability to check for yourself what it is all about. First, please download the latest currently available patch for Firefox 2.0 beta 2 in Debian. It has been in experimental for some weeks now. It is a bit different from the patch I described earlier, so I’ll also tell you what has been changed since this 2.0b2 patch:

• security/coreconf/rules.mk is not patched for the perl dependencies thing any more because the patch (taken from bugzilla #325148) has been applied in Firefox 2.0rc1
• The patch from bugzilla #336056 has been applied because the directory in which we would build the 2.0rc1 version contained a ~ character, which would make the build fail because of this bug.
• The patch from bugzilla #294879 has been applied for obvious reasons. It has been applied later in the Firefox 2.0 release process.

(Also note that another change has been done since then, which is actually reverting a previous patch on gfx/src/gtk/mozilla-decoder.cpp, that was adding code that became dead code when we changed the way to fix the bug.) Next, please download the official and supported Firefox diff from Ubuntu. Uncompress both these diff files, and create two directories (let’s call them ubuntudiff and debdiff) which will contain the individual patches for each file. To fill these directories, please use the following commands:

$ filterdiff --strip=1 firefox_2.0+0dfsg-0ubuntu3.diff > ubuntu.diff
$ filterdiff --strip=1 firefox_1.99+2.0b2+dfsg-1.diff > deb.diff
$ cd ubuntudiff
$ splitdiff -d -a ../ubuntu.diff
$ cd ../debdiff
$ splitdiff -d -a ../deb.diff
If you don’t have the splitdiff and filterdiff utilities, you can get them from the patchutils tools (you will also need another of these tools later). Now, let’s have fun with these split patches… Files that are only patched in Debian: (excluding any file that would be in the debian subdirectory, that contains maintainer scripts for build and installation)

$ LANG=C diff -ru debdiff/ ubuntudiff/ grep "^Only in deb" grep -v debian wc -l
6
So, that’s 6 files that Debian patches that Ubuntu doesn’t. Let’s check what they are:

$ LANG=C diff -ru debdiff/ ubuntudiff/ grep "^Only in deb" grep -v debian

(reordering result for convenience)

Only in debdiff/: browser_base_content_aboutDialog.xul

Adds rows=5 to the user agent textbox so as to display the user agent string uncut by default

Only in debdiff/: content_html_content_src_nsGenericHTMLElement.cpp
Only in debdiff/: content_html_content_src_nsHTMLInputElement.cpp
Only in debdiff/: dom_src_base_nsGlobalWindow.cpp

It’s a patch I submitted in bugzilla #343953 and that got applied in Firefox 2.0. No surprise the patch is not applied on Ubuntu.

Only in debdiff/: extensions_reporter_Makefile.in

Patch I submitted in bugzilla #354413 and that got applied in Firefox 2.0.

Only in debdiff/: security_coreconf_rules.mk

Patch from bugzilla #325148 that got applied in Firefox 2.0 (see above), and a small patch to build the NSS library with debugging symbols (put -g in CFLAGS). That leaves only 2 small patches : CFLAGS = -g added to security/coreconf/rules.mk and rows=5 to browser/base/content/aboutDialog.xul Files that are only patched in Ubuntu: (same exceptions as for Debian)

$ LANG=C diff -ru debdiff/ ubuntudiff/ grep "^Only in ubuntu" grep -v debian wc -l
45
Yes, that’s right, that’s 45 files that Ubuntu patches that Debian doesn’t. Most are windows sizes and similar things that upstream can’t get right because they are values adapted to Windows, but that also includes some changes to the code and some other things. Let’s now check differences in files that are patched in both. It’s a bit of shell black magic, but you can check by hand that it does nothing wrong :

$ LANG=C diff -ru debdiff/ ubuntudiff/ filterdiff -p1 -x configure -x debian"*" lsdiff --strip=1 while read f; do diff -u debdiff/$f ubuntudiff/ awk "! /^--- ^\+\+\+ ^ ^[-+]*\@\@/ print \"$f\"; exit "; done

That gives a list of patch files that have more differences than line numbers changes, and that do not apply to the debian directory or the configure script (which is generated from configure.in), which is what we really want to compare here. You can take this list and run interdiff between these files from the ubuntudiff and debdiff directories. I’ll explain for you what these differences are:

• Makefile.in: Ubuntu adds a line in order to install the NSS include files,
• browser_app_Makefile.in: Patch from bugzilla #314927 applied by Debian
• browser_app_profile_firefox.js: (interdiff fails because the changes apply to different versions of the file) Ubuntu changes the homepage, and sets profile.allow_automigration to false,
• config_autoconf.mk.in: Ubuntu changes includedir and idldir,
• configure.in: Ubuntu changes MOZ_APP_DISPLAYNAME, and adds some echos,
• gfx_src_gtk_nsFontMetricsXft.cpp: Ubuntu sets FC_ANY_METRICS to some patterns,
• intl_lwbrk_src_nsJISx4501LineBreaker.cpp: Patch from bugzilla #161826 applied by Debian for sparc64, which is not an architecture supported by Ubuntu,
• modules_libpref_src_init_all.js: Ubuntu changes some default fonts from serif to sans-serif, while Debian changes fonts for serif, sans-serif and monospace (setting to generic names instead of Times, Helvetica and Courier) ; I think both should be applied in Debian, actually. Ubuntu also sets dom.event.contextmenu.enabled to false,
• security_nss_lib_freebl_unix_rand.c: (interdiff fails because the changes apply to different versions of the file): both patches have the exact same effect, but are different due to changes to original file between 2.0rc1 and 2.0 final.

Overall, Ubuntu applies the same set of patches as Debian, plus some more. A somewhat more modest set of divergences, huh ?!? For what it’s worth, Ubuntu, like Debian, builds its Firefox with flat chrome and pango enabled.

What’s different in the shipping Ubuntu version of Firefox than the proposed Debian version of Firefox (that didn’t ultimately ship)?
Technically, changes include fixes to the User Agent string and the feed preview, a well as addressing issues of coherent branding. More significant than any specific difference in code, however, is Ubuntu’s commitment to work together with Mozilla and our community on releases going forward to insure product quality and integrity.
Why are you working with Ubuntu when you wouldn’t work with Debian?
We did try to work with Debian and would prefer a situation in which we work together. Ultimately, Debian took a position that was consistent with their own policies, and not compatible with some of the exceptions to Mozilla trademark policies that we offered. While we understand and respect their decision not to work with us under our branding guidelines, Mozilla believes that brands like Firefox are important for consumer protection. In any event, Ubuntu developers are working closely with Mozilla developers to insure product quality and features that are what users expect when they use Mozilla Firefox, which means that they’ll ship (and will continue to ship) a fully branded version.

Reading between the lines, that means Debian is not working with Mozilla. It’s not like we’re submitting patches. No. Never. Ever. I’m also glad to hear from Asa Dotzler, in comments to Mr Beard’s article, that this great collaboration with Ubuntu will lead to patches applied to Firefox (I guess the paid Canonical employees having more time to deal with Mozilla than the volunteer Debian maintainers may have helped, especially considering they didn’t have to do all what we already prepared). Anyways, it’s not like some of the patches we sent got applied. So, while I’m at it, here is an exhaustive list of the bugs where we took or sent the patches that are applied to Iceweasel: #51429, #161826, #252033, #258429, #273524, #287150, #289394, #294879, #307168, #307418, #314927, #319012, #322806, #323114, #325148, #326245, #330628, #331781, #331785, #331818, #333289, #333308, #343953, #345077, #345079, #345080, #345413. These don’t cover the following patches (see the rationale for these in my previous article):

• security/manager/Makefile.in, security/nss/cmd/shlibsign/Makefile: Don’t build the shlib signatures, see the rationale in my previous article,
• security/nss/cmd/shlibsign/manifest.mn: Don’t build the mangle utility,
• browser/app/Makefile.in: Force linking against libxpcom.so despite -Wl,–as-needed,
• browser/app/profile/firefox.js, gfx/src/gtk/fontEncoding.properties, modules/libpref/src/init/all.js: Preferences changes,
• browser/base/content/aboutDialog.xul: That’s one of the patches that Debian applies that Ubuntu doesn’t. See above,
• config/autoconf.mk.in: Remainings of an old patch for pangocairo, which is now useless, and setting of mozappdir,
• config/rules.mk, configure.in: Patch from Thiemo Seufer to increase stability and performance on mips, useless upstream until bugzilla #258429 is fixed, but on my to_send list,
• extensions/inspector/Makefile.in: Something that should be added to #331785, but well, it’s WONTFIX, anyway,
• layout/build/Makefile.in, layout/build/nsLayoutModule.cpp: Change to the Gecko string,
• modules/libpref/src/nsPrefService.cpp: Add a preferences directory for /etc/firefox/pref,
• widget/src/gtk2/nsWindow.cpp: Extended mouse buttons support, on my to_send list,
• configure.in, xpcom/reflect/xptcall/src/md/unix/Makefile.in: Basic patch for bugzilla #343975,
• configure.in: Check for pangoxft.

It’s so great to spend a great amount of time on a package, send patches, try to understand how things work to get patches applied, and yet, see such denial and false claims about our work. So please, Christopher, Asa, and the others, just stop talking about Debian, it will be better for everyone. PS for Rob in comments out there: No, ColorZilla won’t work on Ubuntu, because of the ABI incompatibility I explained in my previous entry, that Mozilla doesn’t seem to care much about.

Tuxpaint could be a poster child for l10n. It has translations now for 68 different locales. The large number of locales poses somewhat of a problem for testing¹. Tuxpaint 0.9.16 just released. During the release preparation, the manual testing of all of these locales drove me nearly insane. So to help out next time, I created the following bash script using xautomation², imagemagick and xprop (from xbase-clients):

#!/bin/bash
testname="tuxpaint" 
root_geometry= xprop -root _NET_DESKTOP_GEOMETRY  cut -d= -f2 
root_x= echo $root_geometry  cut -d, -f1 
root_y= echo $root_geometry  cut -d, -f2 
xc=$((root_x/2-1))
yc=$((root_y/2-1))
capture_tuxpaint ()  
   tuxpaint --startblank --nolockfile --lang $1 --640x480 --windowed --nosound &
   sleep 1
   xte "mousemove $xc $yc" 
   xte "mouseclick 1" 
   sleep 1
   xte "key Escape" 
   sleep 2
   import -silent -window "Tux Paint" $testname-$1.png
   xte "mousermove -120 -40" 
   xte "mouseclick 1" 
 
for lang in  tuxpaint --lang help  awk '$1~/^[[:lower:]]/  print $1 ' ; do
   capture_tuxpaint $lang
done

I’m not entirely happy with it, but it’s at least a starting point. Ideally, I’d make .pat files to use with visgrep to find coordinates for the button images instead of hardwiring the approximate button positions. Also, the script needs a companion script to configure the system for testing (i.e. perform the locales configuration and install all fonts). Here are the resulting screenshots which show up a few problems with the translations. It looks like the quit dialog strings have been updated since they were first translated, so now we need retranslation for several languages. Also, there are some languages, like Swahili and Klingon, for which there are no locales in glibc in Debian. I’ll have to talk to the glibc maintainers about that. And for that matter, I don’t even know if a DFSG-free Klingon font exists. In any event, I’m delighted with the results, and look forward to using xautomation more for GUI testing.

¹ For one thing, several languages require special fonts. For another, there is no way to get tuxpaint to tell you what locales it supports. Granted, there is --lang help, but this is a crude approximation, as the mapping between language and locale is arbitrarily hardwired (which hmh on #debian-devel irc pointed out to me is just plain wrong). It doesn’t help me at all in the arduous task of configuring glibc locales so I can test them. For that, I need to look at i18n.c.

² I tried using xmacro at first, but xmacrorec just made my pointer jiggle back and forth. It seemed like it was updating the position for a short distance, and then X was resetting it back to the former position of the pointer. Some strange interaction with my window manager, perhaps? But ultimately I am happier with the xautomation approach anyway.

I&aposve been dawdling and delaying writing about our trip to wt:Halifax (Nova Scotia) two weeks ago for some reason, but I&aposm bound and determined to get some info down on, uh, electrons about it right now.

Hella facts about Halifax We flew out of Trudeau airport eeeeearly on a Friday morning; I&aposd shopped a bit on the Air Canada site and got $99 each-way tickets from Montreal on their wp:Air Canada Tango program, but they required that we be on the 7:50AM flight. Air Canada used to be one of my favorite airlines, but they have been getting continuously disproportionately high prices and conversely worse amenities (pillows and blankets now cost extra, thank you very much). Discount flights in Canada are pretty sparse these days, with the collapse of Jetsgo in 2005 and the recent repositioning of CanJet for charters only. So, you just have to take what you can get. The flight to Halifax was quick and painless; Amita June slept most of the way there. The airport there is pretty small (yay small airports), so we buzzed through baggage claim. Cabs into town were a pricey flat-fee of $52, which got our hackles up, so instead we waited half an hour and took the Airporter shuttle ($16/person) into Halifax proper. We got to see a lot of the Nova Scotia countryside from the shuttle -- lots of leaves turning and conifer forests. We had reservations at the pretty and historic Lord Nelson Hotel, well away from the waterfront, but still in the downtown area, near Dalhousie University. The Lord Nelson is in the "Splurge" section of Halifax hotels on Wikitravel, but since this trip was for my birthday and our 2nd wedding anniversary I thought it&aposd be nice to stay there for one of the 4 nights we were in town. But when I called to make a reservation, I was able to get a very competitive rate, mostly due to a healthy 40% CAA discount, and I booked us for all 4 nights. I&aposd asked for early check-in, but our room wasn&apost ready at noon, so we tried the Lord Nelson Pub for lunch and a beer. The food was tasty and the atmosphere was, uh, pubby. We pawed through the stack of tourist-info documentation that we&aposd picked up at the info desk at the airport, then went up to our big and well-lit room for a mid-afternoon nap. Yes, all three of us. When we woke up, Maj and Amita June went for a walk across the street to the pretty wp:Halifax Public Gardens. It&aposs a very nice Victorian-era public park with disturbingly large geese roaming about. When they got back, the three of us headed down to the waterfront for an early dinner. Halifax&aposs streets are very steep -- they virtually plummet into the water. But we walked straight down to Water Street then meandered along looking at the sights. Our intention was to eat at the Privateer&aposs Warehouse, a building with a bar and two restaurants stacked on top of each other. It&aposs part of the Historic Properties on Upper Water Street, a collection of restored buildings with shopping and food. But the Warehouse turned out to be too rambunctious for us on a Friday night, so we instead went around the corner to Salty&aposs for a slightly more sedate meal on the water. We had good food; Maj had some very nice crab cakes and I took on a large seafood salad. Amita June chomped on really tasty fish and chips. The walk back up the hill was a bit tough, so we stopped in the Rogue&aposs Roost brewpub on the way home for a little strengthener. Unfortunately, as it was after 8:30PM, the baby wasn&apost allowed into the pub. So, we had to leave her tied to a parking meter while we had a... just kidding. We retired to bed after a long day, all three. tags: halifax travel amitajune nova scotia

California Dreamin&apos All the leaves are brown, red, orange, chartreuse and saffron here in Montreal, yet we&aposre off to wt:San Francisco tomorrow morning (early) to visit friends and family. I love going to see my parents and brothers and sister-in-law and nieces, but I also like going back to see the wt:Bay Area (California). It hasn&apost always been like this. It used to be that when I went back, it felt like I&aposd never left, and that the life that we&aposve made here in Quebec was just a story I told myself. San Francisco seemed real; Montreal seemed distant. San Francisco was where I was supposed to be; Montreal was where I stubbornly and irrationally insisted on staying. (The fact that I stayed in a room in my parents&apos house didn&apost help with the stepping-backwards feeling.) But it&aposs been a few years, and Maj and I have been married here, had a baby, and made a business for ourselves. Even though we don&apost own a house (yet), we&aposre putting down roots. Now when I return to California, it feels like a visit -- like traveling to just another destination. And you know what? California is a fabulous place to visit. Northern California even more so. San Francisco is a spectacular place on the edge of the ocean with music, art, redwood trees, organic food, and a sense of humor about itself that no other town quite has. It will be good to get back to SF for a while. It will be nice to take the baby around to see people and things that she hasn&apost ever seen before. We&aposll be in wt:Palo Alto to stay with my folks, visit my brothers in wt:San Francisco, up to wt:Petaluma and wt:Oakland to visit friends, and maybe even down to wt:Big Sur for a campout next weekend. I&aposve got the Web 2.0 Conference that I&aposm covering for LinuxWorld. And I&aposm sure there&aposll be more in there; we need ot dip the baby&aposs toes in the Pacific sometime soon. tags: california montreal home

Some of you might have followed my volounteer work at the SOS Hermann Gmeiner International College in Tema, Ghana, where I was invited to introduce Linux at the school. Being a free software advocate, I thought this a great opportunity to spread Free Software where it can make the most difference and helps bridging the Digital Divide: In a developing country.Judging from the offer (which was very nice, it included housing, flights, freight, money), I expected the school to be very eager to learn about Linux. I was expecting that I only had to guide existing momentum. What I found instead was that there was zero interest from the people who matter in this case. I got all freedoms to implement what I want (as long as I didn t break anything), and I created some things I think are quite nice: Automatic installations that keep the installed windows intact, neat integration in the Active Directory setup, network monitoring with nagios, a transparent firewall that let s the teachers turn the internet for the class room off and on. But it felt like working for nothing, and I can t push people to use Free Software who are unwilling.So today I drew the concequences and quit with effect to the end of December. The principal, who s project it was in the first place, was sorry, but understanding of my issues, and accepted my resignation. I will be on a trip to the Volta lake and other parts of Ghana with friends from Accra for the next 10 days anywasy, so she will take the time to think and discuss how the school can make the best out of the remaining two months.I have not giving up on spreading Free Software here completely. If I find another project where I can be of more use afterwards, I will consider that. But this time I m going to be very picky where I go, and only take a shorter, better defined project. If I can t find something, then that s it and I ll go back to Germany to continue my studies.

As you may know, the future of the US world is again at stake in two weeks. Elections for the United States Senate and the United States House of Representatives will be held on November 7. Most general indicators are favorable to the Democrats (only 40% approve Bush administration, 46% vs. 34% think that Democrats would handle Iraq better than the GOP, 55% think US shouldn’t have invaded Iraq), and it seems they will likely get a majority in the House (latest polls show a 13-18 seats Democratic lead). Democratic Senate majority sounds far-fetched though. First of all, the members of the US Senate (which has 100 members, two per state, regardless of population) are elected by classes. The Senate is formed by three classes, so in this election only a third of the senators are challenged, which makes a turnover more difficult. Right now the Senate is composed by 55 Republicans, 44 Democrats and 1 Independent (who usually caucuses with the Democrats). Only 15 out of the 33 contested seats are held by Republicans, so Democrats would have to gain 7 of those 15 to get a majority (as a tie is broken by the Senate President, who happens to be the Vicepresident of the US, Dick Cheney). There are four close races that should be taken into account (Tennessee, Missouri, Virginia and New Jersey). Democrats would need to win three out of these to achieve majority. The amazing fact is that according to the latest polls they can make it. I’m not really fond of Democrats either (IMHO they are as right-winged as usual European right-winged parties), but lots of Dem Senate candidates are taking solid anti-war positions, which is quite good. In addition, I would say that anything that can take some power out of Bush hands it’s good news for the world. I just hope the anti-GOP wave is big enough. So, please, fellow American reader, on November 7th, vote the GOP out of the Senate and the House