What started with a little host naming theme riddle, and spread as a meme across various planets, has come full circle today, as I installed a new machine, khyber.madduck.net in the racks hosted at the LayerOne colocation site in Z rich. Init Seven, my favourite Swiss ISP, are sponsoring the rack space and connectivity: a routed /28 IPv4 subnet and a /64 IPv6 subnet. The machine is a Transtec Calleo 331 1U server with two quad-core AMD Opteron 2354 processors, 16Gb of RAM, and 4Tb of disk space, configured into a 2Tb RAID1 with LVM on top. It will be used to host a couple of KVM instances for a number of projects I currently operate off my main, personal server, pulse.madduck.net. Unfortunately, Transtec (as well as most other vendors I checked out) uses nVidia s MCP55Pro chipset, which had me sceptical long before I sent the order. nVidia is known to suck at thinking open-source, and the Linux kernel driver for their network chips, forcedeth, had to be reverse-engineered, because nVidia failed to cooperate. Apart from that, nVidia s chips are also often simply crap. Many of their motherboards come with broken APICs and the network chips often wreak havoc on subnets. I find it curious that a search for MCP55 or MCP55Pro on the nVidia website yields no results. Given these expectations, I insisted on a try-and-buy contract, which are not default in Switzerland (as opposed to the EU). My fears proved right, it wasn t long until I encountered the first problems with nVidia s crap: the network chip is unable to deal with segmentation offloading for larger packets, e.g. those with IPv6 payload in a bridging context, and generic segmentation offloading has to be turned off on the interface:

ethtool -K eth0 gso off

This fixes the issue, and so I stuffed it into /etc/network/interfaces and decided to keep the machine, especially after having invested the time into the server to figure out how to work the watchdog and the IPMI card. The folks behind Transtec s support hotline are helpful, but it took me a bit of time to get the information I wanted out of them. At least they deemed it perfectly natural that I wanted to run Debian, and never questioned that. Let s document some of the more internal specs here for posterity: The BIOS gives the impression of two separate watchdogs, one on the IPMI card (a SIMSO+) and one on the mainboard itself (a SuperMicro H8DMU+). Transtec told me that SuperMicro disabled the IPMI watchdog (called BMC watchdog in the BIOS) with firmware 1.29, leaving only the one on the motherboard. It took a bit of time to find out that this was a W83627THF and supported by the w83627hf_wdt driver, which I configured with module options timeout=120 nowayout=1. I left the BIOS setting at disabled since that would otherwise interfere with booting e.g. the awesome grml live system by rebooting the machine after the set amount of time. It also took a bit of time to get SOL working with the IPMI card. In the end, I found it to work best by configuring the console redirection in the BIOS to use COM2 with 38400/n/8, using hardware flow control, redirection-after-POST always, and vt100 emulation ANSI and vt-utf8 emulation were horrific. I instructed the bootloader to pass console=tty0 console=ttyS1,38400n8 to the kernel (order matters), and configured agetty in /etc/inittab like this:

T0:2345:respawn:/sbin/getty -8hL ttyS1 38400n8 vt100

To guard against the worst, I also left a grml CD in the drive, to be able to boot a rescue system via SOL in case everything else was broken. The following kernel boot parameters enable me to use the rescue system via IPMI:

vga=normal noeject noprompt console=tty0 console=ttyS1,38400n8 forensic

Michael Prokop of the grml project tried to create a customised ISO image with these options for me, but I kept getting medium errors trying to read it, independent of the burner, CD-writing-software, operating system used to burn, medium, and drive used to read, so it s possible that the ISO image is faulty. Michael used the following command line to create it (supporting files are here):

grml-live -t ~/templates-madduck -c GRMLBASE,GRML_MEDIUM,AMD64 -o \
  /grml-live/grml64-medium_2008.11-madduck -a amd64 -s sid -v \
  2008.11-madduck -g grml64-medium -r Schluchtenscheisser -V -b

Until we figure out the problem with this, the grml64 0.2 CD in the drive will do. The machine is now operational, but the IPv6 prefix I have still arrives at vera.madduck.net, a Xen-Instance hosted by Init Seven, which I will no longer need, once I find the time to migrate all the services it hosts. NP: This Will Destroy You: Young Mountain

Netatia has an interesting series of articles about running a computer for two people [1]. It is a bit of a kludge, they have a single X server that covers both displays and then use Xephyr to divide it into two virtual screens. The positive aspecct of this is that it shuld allow a single wide monitor to be used by two sessions as displays are getting wider regardless of the wishes of manufacturers and consumers [2] this should be useful. It’s a pity that no-one has solved the problem of having multiple video cards, sound cards, and input devices to allow a single desktop system to be used for 6 or more people. It seems that the problems that need to be solved are only the support for multiple video cards, mouse-wheel support, and sound support. Paul Ewald gave an interesting TED talk about changing the conditions for diseases so that they evolve to be benign [3]. The first example is Cholera which if spread by water will benefit from being as toxic as possible (to cause the greatest amount of diorrhea - killing the host not being a problem), but if spread by human contact benefits from leaving it’s host well enough to walk around and meet people. This and the other examples he cites seem like strong reasons for universal health-care provided by the government. If clean water is provided to all the poor people then cholera will evolve to be less harmful, and if a rich person (such as myself) is unlucky enough to catch it then the results won’t be so bad. He also notes that less harmful bacteria will often result in the victim not seeking anti-biotics and therefore less pressure for the disease to evolve resistance to anti-biotics. Therefore the people who really need them (the elderly, the very young, and people who are already sick) will find them to be more effective. Paul Stamets gave a great TED talk about fungus [4]. One of his discoveries was that fungi can be used for breaking down petro-chemicals (they can eat oil). It would be interesting to see this tested on a large scale with one of the oil spils or with the polluted land around an ooil refinery. Also he has patented a method for using fungus to kill wood-eating ants (such as the ones that briefly infested his home). Robert Full gave an interesting TED talk on robot feet [5]. I found the bit about leg spikes particularly interesting (I had always wondered why insects have spikey legs). Alan Kay gave a very interesting presentation on using computers to teach young children about science [6]. An OLPC is referenced. It makes me want to buy an OLPC for everyone I know who has young children. The start of the talk is a little slow. Dan Barber gave a very interesting TED talk about organic and humane production of foie gras in Extramuda [7]. Apparently it tastes a lot better too. Incidentally I don’t list all the TED talks I watch, only the better ones. Less than half the TED talks that I see announced seem interesting enough to download, and of those less than half are good enough that I will recommend them. The ones that I don’t recommend don’t suck in any way, it’s just that I can’t write a paragraph about every talk. Of recent times my video watching has been divided about equally between “The Bill” and TED talks. Here’s an interesting article about Sarah Palin and “anti-elitism”: The prospects of a Palin administration are far more frightening, in fact, than those of a Palin Institute for Pediatric Neurosurgery. Ask yourself: how has “elitism” become a bad word in American politics? There is simply no other walk of life in which extraordinary talent and rigorous training are denigrated. We want elite pilots to fly our planes, elite troops to undertake our most critical missions, elite athletes to represent us in competition and elite scientists to devote the most productive years of their lives to curing our diseases. And yet, when it comes time to vest people with even greater responsibilities, we consider it a virtue to shun any and all standards of excellence. When it comes to choosing the people whose thoughts and actions will decide the fates of millions, then we suddenly want someone just like us, someone fit to have a beer with, someone down-to-earth in fact, almost anyone, provided that he or she doesn’t seem too intelligent or well educated.[8] Sarah will be representing the Republican party in 2012, the desire for leaders of average intelligence (or less) will still be around then. It will be interesting to see how many votes she gets and amusing to see her interviewed. The proceedings of the “Old Bailey” - London’s Central Criminal Court have been published [9]. It’s interesting to read some of the historical information about the legal system at the time. It made me appreciate how civilised the UK (and other countries that I have visited) are now. Bruce Schneier writes about the feture of ephemeral communication [10]. He concludes with the point “until we have a Presidential election where both candidates have a complete history on social networking sites from before they were teenagers we aren’t fully an information age society“. Of course as he notes the rules are written by the older people, currently I don’t think that any candidate for high office (cabinet minister or above) anywhere in the world can have a good history on the Internet. During the course of a decade or more on the net it’s impossible not to write something that can be used against you and no reasonable person could avoid changing their views on some issues in such a time period. That’s enough to lose an election with the way things currently work.

• [1] http://netpatia.blogspot.com/2008/02/multiseat-computer-with-ubuntu-804.html


• [2] http://lenovoblogs.com/insidethebox/?p=155


• [3] http://www.ted.com/index.php/talks/paul_ewald_asks_can_we_domesticate_germs.html


• [4] http://www.ted.com/index.php/talks/paul_stamets_on_6_ways_mushrooms_can_save_the_world.html


• [5] http://www.ted.com/index.php/talks/robert_full_on_animal_movement.html


• [6] http://www.ted.com/index.php/talks/alan_kay_shares_a_powerful_idea_about_ideas.html


• [7] http://www.ted.com/index.php/talks/dan_barber_s_surprising_foie_gras_parable.html


• [8] http://www.newsweek.com/id/160080/page/1


• [9] http://www.oldbaileyonline.org/


• [10] http://www.schneier.com/blog/archives/2008/11/the_future_of_e.html

It's interesting to be here in Casar de Caceres for the joined i18n+QA+FTPmasters Extremadura meeting. Today, I've been sitting all day long in the biblioteca, which is used as hacklab, and just watched all great people around playing around with the concept of "tdebs". That weird beast can be described in two words as a concept of deb packages that would be able to distribute translation data in a more independent way then "regular" deb packages. Don't smash me as the concept is currently being twisted around in all directions by the nice folks here, to bring it to a great concept well suited for "side" material of our packages. To get an idea, you can read the original concept that popped up during the 2006 i18n Extremadura meeting, but got transformed drastically by ideas exchanged at Debconf 8 All this is made possible because the concept is now a very important issue for Emdebian and is strongly pushed by Neil Williams, whose nickname shall now be "MisterT", by general agreement of the folks here. What's really interesting right now is to see live discussions happening all the way round and see the concept slowly emerging from this. Something we could probably never do in regular mailing lists discussions or on IRC. We should really grant tons of rewards to the Extremadura region, and particularly out tireless C sar who again organised that meeting. It's still hard to say what will emerge from this and we willprobably need some more such live meetings to have real implementations of these tdeb beasts, but this is really how I like to see free software development going on. And now people begin to think about coding stuff and prepare patches, which is a good sign, in general.

Six months ago I investigated the options for Xen virtual servers [1]. I ended up receiving an offer of free hosting and not needing that, but the research was useful. There is a good range of options for Xen servers with different amounts of CPU power, RAM, bandwidth, and disk space. There are a couple of things that seem to be missing, options to upgrade from virtual servers to physical servers, and information on dedicated disk and database performance - but I’ll explain that later after some history. About a week ago a client needed a Xen virtual server in a hurry, their main server (a Xen system that I run on hardware that they rent) was getting a bit overloaded and needed to have one of the largest DomUs moved off. I ended up recommending Linode [2] based on my research and comments I received. The Linode server is working quite well and the client is happy, one nice feature of Linode is the choice of server rooms that they offer. I was able to choose a room in the same region as the other servers that the client owns and thus get ping times that are sometimes less than 2ms! Due to a missing feature in a program that I’m maintaining for the client a large number of MySQL queries are being made. Due to a problem I’m having with MySQL it won’t let me create a slave database server so all the queries go over the VPN and use a large amount of data. This combined with the other traffic that should be going over that link means that about 600G per month is being used, fortunately that is rather cheap. Linode staff handled this very well, after the server had exceeded it’s quota by 120G they asked my client to confirm that the traffic was legitimate and then suggested an upgrade to a plan that could handle the traffic (which went smoothly). Now I have another week to add the feature in question before I meet the quota again. Shortly after getting the new virtual server running at full capacity David Welton wrote a detailed review of Linode and Slicehost for the issues that matter to his use [3]. His conclusion seems strongly in favor of Linode. But now I am looking at getting a Slicehost [4] virtual server for the same client (for a different project) because Slicehost is owned by Rackspace [5], and the new project if successful will need a set of powerful servers and Rackspace seems like a reasonable company to host that. The problem with Rackspace is that they (and every other ISP I’ve researched so far) seems to offer little in regard to customers who need serious disk IO. I am planning some servers that will have a write bottleneck on a MySQL database (or maybe multiple shards), so serious disk capacity is needed. At least I would like to be able to get disk storage by the tray (12-14 disks) with the controllers having RAID-6 support. Rackspace only offers RAID-5 (according to the “livechat” person), and we didn’t get as far as discussing how to add more trays. What would be ideal is if there was an ISP that had both virtual servers and physical servers (so I could start with a virtual server and move to a physical server when things are working well), and also serious storage options. They would offer internal disks, external RAID arrays, and NetApp Filers [6] (or some equivalent device). It would be really nice if I could just instruct the ISP to add another NetApp Filer to my back-end network and have it done for me (I’m certainly not going to visit the US to install new hardware). It’s been over a day since I submitted a sales request to NetApp asking whether they partner with any ISPs and I haven’t received a response. OpenSolaris with ZFS also sounds good for disk IO performance (they have similar features to NetApp). Unfortunately the support for OpenSolaris among ISPs is not that great (while everyone offers Linux and Windows), and I haven’t used any recent version of Solaris. So using OpenSolaris would require finding someone with the skills to manage it who can work for my client - as opposed to a NetApp device that would be just like any other NFS server, SAN, or iSCSI server. But I’m not ruling OpenSolaris out, if someone knows of a good ISP that hosts OpenSolaris machines and supports adding dozens of disks and decent amounts of NVRAM for ZFS then I would be interested to investigate it. Joyent has some interesting OpenSolaris virtual server plans [7], they are a little pricey but offer large amounts of data transfer. They don’t provide any information on disk IO capacity (other than saying that they use ZFS for good performance). I’ve just downloaded Nexenta (Debian with the OpenSolaris kernel) [8] and will test it out over the next few days. One of the reasons I’m tending towards Rackspace at the moment (with Slicehost as the entry point) is that they seem cooperative to customer requests. My discussions with them (on a web based “livechat” and on the phone) have indicated that they may be able to do something special for me.

• [1] http://etbe.coker.com.au/2008/05/28/xen-hosting/


• [2] http://www.linode.com/


• [3] http://journal.dedasys.com/2008/11/24/slicehost-vs-linode


• [4] http://www.slicehost.com/


• [5] http://www.rackspace.com/


• [6] http://www.netapp.com/


• [7] http://www.joyent.com/


• [8] http://www.nexenta.org/

I love it when a lack-of-humor and lack-of-appropriateness-originated flamewar causes somebody to point me towards a very nice display of intelligent humor. Specially when it is so close to me, to my roots, to my family and my personal history. FWIW, for several years, while I was a BBS user, I used WereWolf as my nickname. Great thanks to Frank K ster - and, of course, to Christian Morgenstern.

The Werewolf - English translation by Alexander Gross A Werewolf, troubled by his name,
Left wife and brood one night and came
To a hidden graveyard to enlist
The aid of a long-dead philologist. "Oh sage, wake up, please don't berate me,"
He howled sadly, "Just conjugate me."
The seer arose a bit unsteady
Yawned twice, wheezed once, and then was ready. "Well, 'Werewolf' is your plural past,
While 'Waswolf' is singularly cast:
There's 'Amwolf' too, the present tense,
And 'Iswolf,' 'Arewolf' in this same sense." "I know that--I'm no mental cripple--
The future form and participle
Are what I crave," the beast replied.
The scholar paused--again he tried: "A 'Will-be-wolf?' It's just too long:
'Shall-be-wolf?' 'Has-been-wolf?' Utterly wrong!
Such words are wounds beyond all suture--
I'm sorry, but you have no future." The Werewolf knew better--his sons still slept
At home, and homewards now he crept,
Happy, humble, without apology
For such folly of philology.

Space and place has long been important to hackers. Whether it was/is the university lab, the workplace, the hacker con, or the particularly high-tech city, hackers congregate and meet face to face, often and everyday. One semi-new development has been the explosion and proliferation of hacker labs and spaces, such as Noisebridge located in the Mission district of San Francisco Foulab in Montreal. I recently got back from San Francisco and was able to spend a few nights at Noisebridge and was jaw droppingly impressed. The space is, well, spacious and nice (and located right by the Bart, a real +++++), but more important is that it is a thriving collective with all sorts of geeky participants and they have just souped up the space with all sorts of equipment, from the usual suspects (lots of computers) to lots of electronic gear such as oscilloscopes. What I also was impressed by was not only the blizzard of events but the open and accessible nature of the organization, which seemed to sit in some contrast to NYC’s hacker collective, NYC Resistor. Like so many organizations in this metropolis, they apparently are lacking in space to grow and the word on the streets (which I cannot confirm or deny as I have had very little contact with them but have heard this repeatedly) is that the organization has had a tough time letting in new members. Some folks are understanding of this given their space limitation, others have been less kind, and have referred to the group as a clique. There are already a few other initiatives under way to find a larger space so as to accommodate a more open, participatory atmosphere for a hacker space but I imagine that over time the culture and developments of NYC Resistor will also change as new spaces develops and do hope that this creates the conditions for more access rather than less (and again, I know next to nothing of the situation though I suspect space plays a real limiting role as it does with nearly everything else in NYC). And thankfully Rose White, a NYC-based sociology graduate student at CUNY, is paying close attention to the rise and development of these hacker spaces. She is well underway doing her dissertation dissertation on these hack spaces and I really look forward to her work.

So, I've just got a portable machine. I've configured it to be a pretty minimal installation of Debian Lenny, but one thing that makes me unhappy is mail handling. By default it came with several exim4- packages. Now in general Exim4 rocks. But it is a deamon running all the time, and overhead I could live without. I looked around to find a mail transport agent that would be more suited to the machine and was suprised to find nothing suitable. Basically I figure the machine will never generate "real" emails. Instead it will only receive mails from cron, etc. The machine will never have a real fixed IP, and so relaying mail externally is a waste of time. The mail should just go somewhere predictable and local. There are a couple of lightweight agents which will forward to another system, but nothing seems to exist which will queue mail locally only. So I've hacked a simple script which will do the job. Given the spool director /var/spool/skxmail the following command:

skxmail root < /etc/passwd

Produces this:

/var/spool/skxmail/
 -- root
     -- cur
     -- new
         -- 1227702470.P8218M243303Q22.vain.my.flat
     -- tmp
4 directories, 1 file

That seems to be sufficient for my needs. (I support the flag which says "read the receipient from the body). Of course to do this properly I'd be setgid(mailgroup). Instead I assume that local means everybody can see it and /var/spool/skxmail is mode 777. Ooops. Still happy to share if it sounds interesting. ObFilm: Dogma

So today, I happen to be looking at wikis for two small to mid-sized public proojects (MoinMoin and DokuWiki look like frontrunners right now — any suggestions?) Recently, I’ve also looked at blog and CMS software, and a host of other web apps. It’s as if these people have learned nothing about good software practices over the last 20 years. Warning: Rant Ahead So how many of you have been here before? You download WebApp X. It tells you to cd to your DocumentRoot and unzip/untar it there. At this point, most of them will tell you to chmod -R 777 the install directory. Some of the better ones, such as WordPress, will tell you to chmod it 777, or if that makes you nervous, to instead chown it to the user that your webserver runs as. It is at this point that you realize that the Java-based programs ship with their own webserver that takes 2 minutes to load and uses 2GB of RAM, while the PHP-based programs want you to give them 32MB RAM per process, and probably modify your global PHP settings in a way that breaks some other PHP web app you’re already using. As if that isn’t enough to scare you off, generally speaking, config files — including passwords to databases — are stored in the same directory, along with .htaccess files. Many of these programs are also downloading and updating plugins over the Internet, usually without any kind of cryptographic authentication, and overwriting their own program files in the process. Oh, and this is a class of app that is notorious for security problems to start with, and makes your server known to billions of people via search engines. Absolutely no opportunity for trouble here, of course! That sentence was dripping with sarcasm, in case you didn’t get it. It also makes it almost impossible for people such as Debian maintainers to package up some webapps (such as just about every single one that uses Ruby on Rails) because there is just no sane way to make it behave with respect to the Filesystem Hierarchy Standard. I’d love to see web app developers do a few simple things:

1. Separate code from data
2. Separate code from configuration
3. Separate all of the above from the DocumentRoot to the greatest extent possible

I realize that some of this is purportedly to make things easier to install when you have FTP access only. But to me it seems just really poor design. I’ve written webapps, and it’s not that hard to do this part right. Plus, doing the above right means that I no longer have to do something like use git on my WordPress installations because it’s too much of a hassle to apply security and plugin updates on all three separate ones otherwise.

Since everybody is posting pictures of the first snow of the year, I can't stay behind. Quickly popped out this morning to make this snap of my favorite habitat, the valley of the "Grote Nete":


(click on the image to enlarge)

nikon d300 tamron 17-50 f/2.8 2008 Joost Yervante Damad

A fellow lodger of my student dormatory just popped in and gave me the December issue of Linux Journal as a gift from his trip to the US (thanks for that!). When I skimmed the front page, I immediatelly noticed the line Reviewed: OpenMoko s Neo FreeRunner. The four page article by Cory Wright contains some general notes on the FreeRunner, a list of things he doesn t like and to my surprise installation instructions for Debian on the FreeRunner! It s nice to see one s work used in such public places. I especially liked the line I must say, the Debian installers sure have improved a lot since the Potato days. (Of course, the installer for the FreeRunner is mostly a dumb script that can not be compared to the great official debian-installer that you can use on a PC.)

I spent the second half of this morning delivering leaflets asking people to vote for the Weston Woods Path Project in today’s itv People’s Millions Big Lottery Fund competition. Just 50,000 gets a smooth, fairly flat, surfaced and edged path and takes some pedestrians off the bendy cliff-top road. Any surplus money from the call charges will be donated to charity. If you vote for it, you can comment on my blog to let me know if you want and you might like to watch WsM Forum for the reaction/news. It’s a beautiful day and North Somerset is a beautiful place but by hell is my jaw cold now! At least there’s no snow here.

• PGF (some examples): A TeX macro package for generating pretty graphics, from the same brilliant author who freed me from PowerPoint and Impress. The user-oriented PGF frontend called TikZ is at least as enjoyable as the more traditional PSTricks and it plays together with both PostScript and PDF!
• Impressive: An OpenGL-based PDF viewer formerly known as keyjnote. Some of its features even make sense, the others are at least nice to watch.
• rlwrap: Readline instead of ^[[D.
• Byte.fm: A professional web radio station all around music. The presenters make this a definite recommendation for German-speaking people.

The Timeless Way of Building: Diagnosis and Repair Architect Christopher Alexander's A Pattern Language is well known among programmers as the inspiration for the Design Patterns school of thought, but for some reason his other books are less widely read. My mother and father are both architects—"real" ones, not "software architects." :) My own amateur interest in the subject led me to read several of Alexander's books over the past couple of years. The Timeless Way of Building is the first volume of an open-ended series. (A Pattern Language is the second.) It is in this book that Alexander first lays out the idea of a pattern language. More importantly, he explains an entire theory and practice of design and construction, of which the pattern language is only one piece. Another piece of the Timeless Way is the practice of diagnosis and repair. Rather than a detailed plan of future projects, Alexander prefers to maintain a diagnosis of current problems. Each new project must be targeted at fixing these existing problem in the system: "At every moment we use the defects of the present state as the starting point for the definition of the new state" (p. 485). Alexander's goal is to create a type of spontaneous growth and order that is possible when change is incremental and decision-making authority is decentralized. Alexander was hired to produce a new type of master plan for the University of Oregon campus. The result was published as the third volume in the "Timeless Way" series: The Oregon Experiment. In it, he writes again about diagnosis and repair:

It is still not clear where global order will come from, without a master plan.… How is the problem solved in an organism? Essentially, the problem is solved by a process of diagnosis and local repair.… The master plan tells us what is right for the future. The diagnosis tells us what is wrong for the present. (p. 146–148)

In software, the same problems and conclusions have led to "agile" processes, which emphasize incremental change and flexible planning. But I haven't seen any software companies adopt a consistent practice of "diagnosis"—keeping track of what's wrong or unsatisfactory with the current system. This extends beyond defect tracking; it should include a constantly updated list of code that, although it works, is hard to understand, difficult to change, or otherwise likely to cause problems. (This should be easy to generate—every programmer knows certain parts of their code base that they dread working on for one reason or another.) This knowledge can then guide new development to include "repair" and address such problems before they become defects. I've applied this on a small scale to my own work, and now I wish I had used it at Amazon, where my team spent much more time maintaining and improving old code than writing new code.

My posting about receiving a Conceptronic CH3SNAS a month ago prompted Matt Palmer to work on the installer. Matt did a fantastic job and had patches for everything within a few days: he added support for the revision B of the DNS-323 to the kernel, added code to the kernel so the MAC address is parsed from flash, and added DNS-323 support to various components of the installer. Finally, he wrote a really good porting guide that will hopefully inspire other people to port the installer to more NAS devices. Porting to another Orion based device usually doesn't take much now that we have the infrastructure in place. While Matt's patches came too late for inclusion into the initial release of lenny, the release team seems agreeable to the idea of adding support for the D-Link DNS-323 and Conceptronic CH3SNAS to a stable update of lenny. To this end, the dns323-firmware-tools package which is needed to create proper firmware images for the DNS-323 was accepted for lenny a few days ago. Alan Fletcher also kindly agreed to make a serial cable for me, so hopefully I'll get a chance to test Matt's patches soon and integrate them into the installer. In the meantime, I created a few simple pages about Debian on the D-Link DNS-323 which I will extend once support has been integrated into the installer.

I've found myself writing this several times recently in the context of Telepathy, so I thought I'd make a blog post that I can refer people to. D-Bus is defined (by the wire protocol spec) to be an asynchronous message-passing system, and libdbus behaves accordingly; there are no blocking calls at the wire protocol level. However, libdbus provides a "blocking" API (dbus_do_something_and_block), via a mechanism I'll refer to as "pseudo-blocking" for the purposes of this post. Most D-Bus bindings (at least dbus-glib, dbus-python and QtDBus) expose this pseudo-blocking behaviour to library users. These pseudo-blocking calls work like this:

• send a method call message (call it M)
• while a reply to M has not been received:
  • select() or poll() on the D-Bus socket, ignoring all other I/O
  • whenever a whole message has been received:
  • check whether the message is a reply to M
  • if it is (call it R), stop
  • if it is not, put it on the incoming message queue

The messages received between M and R are delivered when the main loop is next entered. This can cause a number of problems:

• Messages are re-ordered: messages received between M and R aren't delivered until after the reply, violating the ordering guarantee that the D-Bus daemon usually provides. (This causes practical problems if a signal indicating object destruction is delayed - the client gets a method reply "UnknownMethod", has to guess that this is because the object has vanished, and can't know why it vanished until the signal indicating its destruction arrives with more details.)
• The client is completely unresponsive until the service replies - if the service has somehow got wedged (e.g. telepathy-gabble is meant to be purely non-blocking and asynchronous, but there are cases where it will do blocking I/O on SSL connections due to Loudmouth bugs), the client will be unresponsive for (by default) 25 seconds until the call times out. (Clients shouldn't crash or lock up, whatever happens to the services they depend on.)
• The client can't parallelize calls - if a signal (e.g. an incoming Text message in Telepathy) causes method calls to be made, a client that uses pseudo-blocking calls can't even start processing the next message until those method calls return
• If two processes make pseudo-blocking calls on each other, deadlock occurs. This is particularly tricky in the presence of a plugin architecture and shared D-Bus connections - a plugin that "knows" it's a client and not a service, and a plugin in the same process that "knows" it's a service and not a client, can end up sharing a connection, resulting in a process that is both a service and a client (and hence deadlock-prone). (We've seen this happen in the OLPC Sugar environment and on Nokia internet tablets; it's not just a theoretical concern.)

As a result, telepathy-glib's code generation mechanism does not generate any pseudo-blocking code. There are several alternative modes of operation that we do support:

• Fully asynchronous: call a method now, pass it a callback, get your callback called from the main loop later. This can be awkward to program with, but is the only way forward for most non-trivial projects.
• Re-entrant main loop: re-enter the main loop, processing all messages (D-Bus, GUI, network, anything), and run it until the reply has been received. This is useful for trivial projects (like telepathy-glib's regression tests), but dangerous for non-trivial projects (like Empathy), and I'm somewhat regretting implementing this.
• Ignoring the reply: when appropriate (it rarely is), you can make an asynchronous call with callback = NULL and the reply will be ignored completely

We make a couple of narrowly targeted exceptions to the "no pseudo-blocking" policy in internal code, by allowing telepathy-glib internals to make a small number of pseudo-blocking method calls to the dbus-daemon (which is the one component we can definitely trust to return results promptly). Here are some examples of the same strategies in other libraries:

• Pseudo-blocking: dbus-python method calls with no special keyword arguments; dbus-glib dbus_g_proxy_call; QDBus calls with mode QDBus::Block
• Fully asynchronous: dbus-python method calls with reply_callback and error_callback keyword arguments; dbus-glib dbus_g_proxy_begin_call; QDBusConnection::callWithCallback
• Re-entrant: QDBus calls with mode QDBus::BlockWithGui
• Ignoring reply: dbus-python method calls with ignore_reply keyword argument; QDBus calls with mode QDBus::NoWaitForReply

Based on feedback from my last couple of posts, I've written a guide to some good power management practices. It's aimed at people implementing desktop environments and power management interfaces for the most part, rather than end users or driver developers. Any suggestions for additions or corrections happily received.

I'm not aware of any OS that currently gets all of these correct.

I have just encountered a bug in Gnome that is much more visible when the hard drive is slow or overloaded and responds slowly. This gave me an idea - how about a simple transparent FUSE filesystem that does nothing else than delay, slow down and possibly reorder filesystem requests? Such a filesystem would be very useful for debugging. We developers tend to have high-performance systems and that hides many bugs, but if we could have a slower system on-demand, it will give us the ability to debug our applications better. Even better would be a virtual machine where one could arbitrarily slow down any specific aspects of the system - slower CPU, slower hard drive, slower RAM, slower network, … I know that engineers of network hardware have special devices that emulate long distance and noise - you take two wireless network cards and plug cables into their antennae sockets and then plug those cables into the device. The device then makes a connection between those cables, but with emulation of a variable distance, variable antennae and variable environmental noise, so the wireless cards can be tested for distance and noise resistance right in the lab. We could do the same in software for HDD, CPU, RAM and networking - imagine having two Xen instances and putting an emulated network between them with 10 hops, 1 second latency, 20% average packet drop traffic shaping and limited top bandwidth of 100 kbit/s. Now that is a real test platform for any kinds of applications that have anything to do with networking (or HDD, CPU or RAM). Anyone up to making that? Could someone of you use a system like that?

While a few developers have the chance to be paid for their job on Debian, most of us are doing it on our spare time, for fun. Yes, we are making the best operating system out there, for fun. Yet a large number of developers are taking this project way too seriously. To say it in other words, they have a carrot up the ass. I happen to prefer the French expression though, since it conveys better the rigidity of their attitude: they have a broomstick up the ass. Recently, when I launched the idea of the Bug Sprint, someone wrote to me to propose, instead of rewarding people with cookies, to send them money. This is the best example of broomstick behavior: taking the fun out of people by bringing back rampantly the very idea that almost split the project in two the last time it was introduced. The very same person, when faced with a sarcastic email sent to an announcement mailing list, immediately rushed in to send a ban request to list masters (a totally useless one, since I don t have the habit to make the same joke twice). That person happens to be the project leader. So it seems the only thing the DPL has to do, at a time when we d rather need people to fix RC bugs, is to protect the project from seditious members who don t share their love for broomsticks. When someone makes a bad-taste joke, you are not forced to laugh. But you are not forced to impose your rigid and moralist views either. And on top of that, throwing random accusations of sexism without any kind of justification does not improve the mood. This is even very close to defamation, but since I don t have a broomstick in the ass myself, I prefer to just laugh at them with the help of some fellow developers. I will go on shocking these people. And they won t like it more than in the past. Every time one of them climbs on her high horse to take a pathetic and patronizing stance while totally missing the point, it will bring a bit of fun. Bonus #1: as for the promised statistics, we are, at the time of writing, at 493 599 unique IPs who clicked on the trap. Quite a good investment for someone who d want to make money from smelly geeks. Bonus #2: here is a sexist picture.

I saw a posting about PowerDevil, the new KDE power management interface. It's somewhat disheartening - for the most part, it falls into the same traditional errors made in power management (ie, letting you change cpufreq settings, using "presentation" as a power management setting rather than getting applications to actually do the right thing). But my point isn't to bitch about PowerDevil. My concern is more about why we're still failing to get the message across about certain power management myths. Implementing power management incorrectly leads to wasted power, dead polar bears and wet carpets. It's important that we get this right.

• To a first approximation, the Powersave governor will only save you power if you're playing 3D games. The performance governor will basically never give you extra performance. Don't use them. Use ondemand instead. Do not make it easy for your users to choose them. They will get it wrong, because it is difficult to explain why this result is true.
  
• Thermal management is not the job of a power manager. Using power management to implement thermal management will result in your computer taking up more power overall, reducing battery life. Implement things in the right places.
  
• When people say "Presentation mode", what they mean is "Disable screensaver and automatic suspend to RAM". This is not a power management policy. It is an application behaviour policy. The sensible behaviour is for applications to request that these things be disabled when they switch to full-screen presentation mode. The foolish behaviour is to request that your users select "Presentation mode" in their desktop and then start their presentation in their application and then finish their application and then remember to disable "Presentation mode".
  
• It takes no more energy to scan out a framebuffer drawn with the 3D engine than with the 2D engine. If you're spending a significant proportion of the time on the GPU when in your desktop environment, you're already doing something wrong.

One of the problems is probably that most desktop programmers don't know how hardware behaves. How can we change that?

I live in one of a cluster of a dozen or so houses. As far as I can tell, none of them have any micro-generation capability (I think one has working solar panels and one has defunct panels), but we have an electricity substation and a small patch of land fenced off next to it. I’ve heard it said that the power distribution company would be open to selling the land. Surely we could do something here, with the stronger sun (compared to the most of the UK anyway) and the blustery winds? If only there was some model for grouping together for microgeneration… That’s what I understand as the idea behind the Carbon Co-op and I think it could make a difference. I’ve voted for it to be one of the ideas developed Friday 5th- Sunday 7th December 2008 at the Young Foundation, Bethnal Green, London (I may be in London one of those days). You can vote for it too, particularly if you’re interested in helping it happen too.