This weekend I finally finished the Linux Kernel Driver DataBase . Really I started this project in 2000, but in 2001 after some flames in the LKML (for CML2, the configuration engine used by my project) I put the project in a long hibernation. Now I had some time to finish it.

LKDDb is a database of hardware (and protocols), kernel configuration items and associated kernel driver files. Actually it has nearly 6000 entries (see the statistics). The database is generated automagically, quite fast: it take 2 minutes to scan the whole kernel source, and to interpret it. Unfortunately it is not complete and not accurate (I'm not sure if there is some black-list in kernel, but my scanner cannot distinguish the black-list with the supported hardware list).

Now I'm looking for some application of the database. In 2001 it seemed that there was some interest, but let see.

As a prototype, I dis-hibernate also the AutoKernConf, an automagical kernel configuration. The old version was a more powerful, so I need to forward-port some old features. BTW I changed an other time the project name: autoconfig was too similar to existing projects, kautoconfig, kernfig, kernautoconf were other names, but I like better the new name, and I can use the automagical word in the description ;-)

Do you see other uses of the database?

I've just uploaded a new version of the xorg package to unstable. This package builds (somewhat confusingly, but trust me, there's a good reason) the xserver-xorg package that many people use to create their xorg.conf's. For this upload, we've been ripping out the logic in the xserver-xorg scripts that create the xorg.conf. The reason for this is that the X server can actually handle most of this configuration automatically at runtime, and I'm working on making it do even more. So rather than rely on a custom hacked together script we're moving towards a shared codebase that all linux and bsd distros should end up using to dynamically configure X at startup. This will be a significant win and it's really my primary release goal for Lenny.

So while this work is still ongoing, I'd like to ask that you all, our wonderful users of unstable, test these changes in the 7.3+3 package out. You see, we don't force an update to your xorg.conf, so your previous config should continue to work just like it always did. However, what we need to know if these slimmer xorg.confs work also. So if you have an xorg.conf that isn't customized, please copy and rename the copy somewhere (don't call the copy xorg.conf or else the server might load it even if it's not in the canonical /etc/X11 location) and run dpkg-reconfigure xserver-xorg to generate a new file. You'll see a lot less questions than in the past, but your old responses should be used by default. When you're done, restart the server and make sure it loaded like it used to. If not, please file a bug using reportbug against the xserver-xorg package.

This is still a work in progress, and I've still got a number of things I want to carefully extract from the scripts before the release, most notably any reliance on external programs like discover and xresprobe. But as things are, the configs should be slimmer and there's less time spent answering questions that aren't useful to getting a working X server running. I'm sure I'll send out another call for testing as I remove more major parts of the config, but this is an important step. Thanks everyone for helping make Lenny awesome!

I hate spam! Which probably puts me in the same camp as 99.99999% of the world. The other 1 in 10 million are, of course, the spammers, who seem to take the space invaders approach to sending e-mail: we'll keep sending you more until you die. A few years ago I used to only receive perhaps 1 every 100 seconds, which was pretty annoying, but Spamassassin was quite able to filter out 99% of those and let through about 1-2 each day, which I could deal with. My spam levels increased to maybe 1 every 20 seconds, and late in 2005 I implemented a second layer of spam filtering on my laptop using DSpam. This worked quite effectively, but DSpam is really not the tool for the job - it's much more appropriate as a company-wide antispam solution, and potentially as a replacement for Spamassassin. It drove me nuts on my laptop because it's resource usage slowed down the interactive response. When I got my new laptop at the beginning of the year I decided against continuing with my rather baroque mail setup and to leave the spam filtering on the server. What I didn't realise is that my spam rate had increased again to around 1 every 8 seconds, and it has been slowly driving me to distraction ever since. It seems to have cranked up another notch recently, to perhaps 1 every 3 seconds now, so that 1% making its way through Spamassassin was getting to a very annoying several hundred each day. The longer I took to resolve it, the more time I would be wasting dealing with it every day. What I chose to apply on this occasion was CRM114, which I had some vague idea might be able to help. I was fairly impressed by the relatively simple install, but what completely blew me away was the speed with which it was able to learn to be useful. Starting from scratch, it seems to be correctly classifying over 90% of my incoming mail after about 12 hours of training, on a total of only 75 'Unsure' messages. Even after only an hour it was getting over 50% (I'll describe my actual CRM114 installation process in a comment below). So far there have been no false positives. Now that CRM114 is installed I will be able to look into some of it's other mail classifying features too, and I'm really looking forward to that too.

Two years after writing about what I found out when I was looking why my HDDs kept breaking down, I started to receive emails about this post. It seems that it was mentioned in an Ubuntu bug report and this morning I noticed that exactly this bug report was mentioned on Slashdot, so I can consider my post to be slashdotted (indirectly). For the past week, I’m counting 15,000 hits and it is still rising fast. It’s a good thing that I use a static HTML journal, so that my journal-hosting-pentium-233MHz can handle this. It’s also a good thing that the issue finally gets some attention.

(Image: From Amazon.com) What I am doing with Fedora? Nothing! But, when Shashank Sharma, my friend from New Delhi, sent me complementary copy of this book, I couldn’t resist myself to read it (yes, entire book). Excellent, well written book and must have for Feodra users/power users. It contains DVD of latest Fedora 7 too. And it also deals with general Linux stuffs (which I liked most in this book). I even recommended this book to my wife! Great job Shashank and Keir Thomas (co-author). Book’s homepage at Apress.com

There is a debate raging in anthropological circles over the role of anthropologists in the military and the Iraq War Purse Lip Square Jaw has a nice round-up of of articles on the topic. I have only read bits and pieces here and there but this morning I finally gave my full attention to this intriguing article by David Price, which not only covers the vexing debate but discloses the rampant plagiarism in the The U.S. Army/Marine Corps Counterinsurgency Field Manual.

* I started with my packages, ayttm, festival, sitecopy, tagtool and xosview. Some bugs were already fixed at upstream and forgotten and some was unreproducible. I am happy that submitters provided necessary information to keep my tempo high to really fix them. * I reported small typo (missing ‘ in description) for package ‘debian-maintainers‘ at 11:38 AM, Joey replied and done it (actually it was fixed in NEW version) at 11:56 AM!

During the cleaning of my home office I came across an old notebook from a few years back, full of notes about the configuration of my PCs. In it I find things like this:

Linux Kernel Config
2.0.36
Loadable Modules Support
 [*] Enable loadable module support
Network device support
 [*] Ether express pro 100 (mice)
 [*] NE2000 support (rats)
/dev/cdrom => /dev/scd0

Most interesting to me is the mention of mice.my.flat and rats.my.flat - fun hostnames! A quick google suggests that 2.0.36 was current sometime in early January 1999 I remember buying a network card, at that time, meant going to the local computer shop, and picking two cards at random from their "network card bin". Each card cost £5 and my approach was to go in and buy two cards at random, then get home and modprobe every module until I found one of them at least. If there was no joy I'd return the next weekend and exchange them for two more! Funny how things change. Nowadays on my primary machine, vain.my.flat, I have autodetection of my current (onboard) NIC. I don't have to worry about ports or iobase for my (onboard) sound. Everything just works... (Except for the GNOME-panel volume control applet which crashes on startup. Hmmm.) Same local domain name, different flat.

Perhaps my favorite article in Wikipedia, the List of homophonous phrases, was deleted from Wikipedia in mid-August. Those arguing for deletion claimed that it was original research and were, I suppose, correct in that designation. But that doesn't make me happy to see it go. I asked an admin to move the list into a temporary home in my userspace until I can find a better home for it. Please help me find one and I'll redirect. It's the latest in several unfortunate deletions I witnessed recently. I created a humorous "undeletionist" barnstar to give to an admin for undeleting some humorous project pages in Wikipedia (which have since been redeleted, but copied first). My barnstar was also deleted. In retaliation for this all, and in good fun, I proposed the Association of Deletionist Wikipedians for deletion. The delete tag stuck around for a week until it was deleted under suspicion that it might be a joke. True enough. On the other hand, I have similar suspicions about the subject of the article.

Even though Adrian von Bidder suggests that I'm starting a bit early with the DPL campaigning, I thought I could add a few more thoughts to the blagosphere.

Adrian suggested to make the discussion of our of "vision of Debian" a regular event on debconf - I think it's a nice idea, but that doesn't mean we should keep silent about that topic the rest of the year. I think one of the problems we see in Debian is that we are missing an aim to strive for. We are currently just moving from release to release, while more and more developers have actually lost interest in stable releases. The impression that a common ground like Debian 4.0 is irrelevant seems to prevail, because the people most vocal in our user base usually want to see the newest and shiniest software on their desktop. We have been spending more and more effort to cater to their wishes and have ignored the fact that a majority of Debian-based installations are not run by such people, but by system administrators who do mass roll-outs and neither want nor need the newest X.org with more *bling*.

Ubuntu, the biggest Debian-based distribution, has found its users by packing up new software behind a nice installer, adding some branding elements and then shipping it. Stability is an issue for Ubuntu, but they don't spend as much time as we do on eliminating the last seemingly important bug in some esoteric software package before actually releasing. That observation is the ground for one of my ideas: Our current metric to determine how far testing is away from a release is counting the number of rc bugs - it doesn't matter if the rc bug is X.org crashing for all people or if it is a policy violation in a package with 12 users. Perhaps we should start thinking about a new metric, keeping in mind that for a majority of users, some bugs matter a lot and some matter not at all. With the release of etch, the number of installations of popcon has risen a lot, so for the first time, we have actual usage data that could help us with such a metric. There are a few problems with that approach, though:

• Popcon data is inherently wrong, because single computer installations tend to have popcon, while large installations (like the one in Extremadura) don't.
• A majority of popcon users uses the last stable release, not the one currently in development.
• 3 bugs in packages seldomly used can be more annoying for our users than one central bug - you might be able to find a work around somewhere for something that happens to a lot of people, but a bug only hitting a few users will stay unfixed for a long time.

I don't have more time right now, so the rest of my ideas will follow in the next few days, so stay tuned or add me to your killfile.

It’s interesting to see how far, and yet how much more work we need to do on power management for Linux. I recently got a new laptop — a Lenovo Thinkpad X61s — and using the powertop tool, I was able to configure my system to the point where in what I can “airplane mail reading mode” (mailbox preloaded into memory, USB disabled, wireless and ethernet disabled, backlight down to 30% brightness, sloppily written power hogs like Firefox and Notes — every single application writer should be forced to run powertap and explain why their program feels it necessary to constantly wake up the CPU), I can get my usage down to about 9.8 watts. Using the 8 hours extended battery, that’s 8 hours of battery life, although granted doing very little. On the flip side, if I’m doing a major kernel compile, I can drive up utilization up to almost 30 watts, which means less than 3 hours of battery life. So that’s definitely the good news; Linux can sharply reduce its power consumption to the point where it is highly competitive with Windows. (And probably better than Vista, just because that OS is so heavy and bloated.) So thanks and and a tip of the hat to Intel and to Arjan van de Ven for making such a useful tool like Powertop available. So now for the bad news. Getting down to this level of power saving thriftness, where the laptop is carefully sipping only the minimal amounts of power from the battery, is definitely a work in progress. First of all, you can only get this level of power savings by unloading a specific USB driver, uhci_hcd. This will disable low speed devices (including unfortunately the fingerprint reader and the EVDO WWAN device if you were silly enough to buy one that was built into the laptop as opposed to a stand-alone card that you can swap between laptops and lend out to friends as necessary). But how many users are going to open up a terminal window, su to root and type the command “rmmod uhci_hcd”? And know how to reload the driver using “modprobe uhci_hcd” when they need to use the USB devices again? A similar problem exists for Network Manager; when the user disables the network by right-clicking on the applet, why doesn’t it automatically bring down the interface, instead of forcing the user to manually su to root and then type the command “ifconfig eth0 down; ifconfig wlan0 down”? A more serious problem is the Intel Wireless driver for the 4965. Even with the wlan0 interface configured down, and with the RF kill switch enabled, keeping the iwl4965 driver loaded will still cost you an extra full watt of power. When you’re down to 9.6 watts, that means that keeping the iwl4965 driver loaded when you don’t need it will cost you a 10% reduction in your battery life! That’s just sloppy, and hopefully it will be fixed in a future update to the iwl4965 driver, but as long as you don’t mind manually removing and reloading it, you can work around this power-saving oversight. A bigger issue, though, one for which no workaround exists, is that unlike the ipw3945 drivers, which at least had private, non-standard iwpriv commands to engage the 802.11’s power-saving features, the iwl4965 driver has neither the non-standard Intel iwpriv interfaces, nor the standard iwconfig interfaces for enabling any kind of powersaving features, including changing the transmit power of the card. So while powertop deserves plenty of kudos, iwl4965 deserves a wag of the finger from a power saving viewpoint. No doubt Intel just needs to allocate more money to its Open Source Technology Center so it get more of its crack developers to work improving Linux support for their processors and chipsets. Speaking of which, I’m still waiting for an Intel x.org 965GM driver that can support compiz/beryl and simultaneously show video clips at the same time… And being able turn off the 50 interrupts/second generated by the video card when they aren’t needed because 3-d graphics aren’t currently in use, without requiring a restart of the X server, would also be a nice touch. The bottom line is that Linux power savings and Linux support for laptops in general is much better than it was a year ago, and a lot of credit has to go to the efforts of Intel’s teams producing such good work as powertop, their wireless drivers, and their open X server drivers. We still have a lot of work left to be done, though!

Some time ago a friend of my sister in law contacted me and offered his services as a courier for gifts. I had told him that I knew someone in Israel and he was traveling to Sweden. So I contacted Lior with the strange proposal to give me a gift (and I promised to return the favour). A few days later I got a book about Israel from Lior, was very happy and started to ponder what kind of present could be fun and entertaining for him. I ended up going to the supermarket and bought all kinds of special swedish food (and obviously skipped some). I wrote some short notes with cooking instructions and explanations and sent the whole lot back to Israel.

I enjoyed both getting the present, reading the book and picking a present for someone I regard highly. I would like to recomment this as a fun game within Debian: If you know someone traveling abroad find out if there is a DD (or Debian contributor) around at the destination and send a present to that person. This kind of love bombing is low cost and lots of fun and creates bonds parallel to those of the Web of Trust. Try It!

Some people around Free Software tend to be too enthusiastic and very critical, both with themselves and with other people. Whenever Linux or Free Software appear in the media, people who wrote the articles are severely and merciless attacked, and the criticism is not always constructive. It might be hard to properly talk about these topics in a non-technical magazine or newspaper targeted towards the mainstream population, not only because they have to explain it in an understandable way for non-initiated people, but also because they’re usually very limited in space. Thus, many times the articles are inaccurate or incomplete, not always being a journalist’s fault, but also of the conditions in which the article has to be published, as well as the time they have to develop it. Someone, a journalist, told me this last weekend that she tries to avoid writing stuff about Linux herself due to the harsh attacks she’s subjected to from the Linux and Free Software activists as soon as she publishes anything on that topic. I kind of understand that. It might be the same than when you write a program and release it as Free Software, and then some people get too demanding and aggressive towards you for not having done a perfect program and submitting to their wishes, instead of thinking about the effort invested in what is done. I seriously think that we might be losing a lot of power and strength if we attack so much the media that support us than they fear us enough to avoid these topics. We’re on the same team, and it wouldn’t be so bad to be a little understanding and supporting of them, and rely on constructive and friendly criticism instead of treating them as if they were the enemy. They also have feelings, you know.

Last Saturday, we attended a party at stew and rcox’ place in Brooklyn. Biella, Micah, Clint and a bunch of other people were also there. It was very nice and we had a lot of fun. I had the chance to tell Clint that some of his blog postings really intrigue me; I now feel relieved. Besides, I’ve learned that boxing against Micah on the Wii is so tiring that even days after, my arms and back still hurt We left at around 1:45 am.
On our way back to Harlem, we stopped to have some tacos at El dolo on the 8th Ave and the 14th St, as Kevin suggested me before; for our surprise, they were amazing and have been so far, the best Mexican food we’ve had in NYC So, after getting off on the 145th station and walked nine blocks, we arrived to our warm place at 4:30 am.

I'll probably get laughed at by git/mercurial/darcs/whatever fanboys for even mentioning that I use subversion, but whatever. Just so I don't forget: If, in subversion, you want to merge changes on a branch to the mainline, then the following is wrong:

svn merge http://svnserver/branches/foo/ http://svnserver/trunk/ .

Instead, you should use this:

svn merge http://svnserver/branches/foo@REV http://svnserver/branches/foo@HEAD .

With the current directory being a checkout from /trunk/, and REV being the revision where /branches/foo and /trunk started to diverge; that is, either the revision where the branch was made, or the one where they were brought back in sync with a merge in either direction. Now hopefully I don't forget this for next time.

I just love my city.
Mexico City, chaotic, unknowable, imposible to understand, so huge, so varied, so broken, so feared, complex, unique... I think few people realize how easy it is to fall in love with this city, to drink its beauty, which leaps up always when you least expect it. Today, it was a good day for loving it. For one thing or another, I've missed the five previous ciclotones, which are held on the last Sunday of each month, 7:00 to 14:00. Today, finally, I made it.
What is it? It is a 32 Km route circling roughly around the mid-South and Center of Mexico City:

The weather is abnormally cold lately in Mexico City - We have reached temperatures around 5 Celsius in the last two weeks, which is agreeable for December/January, but certaiunly not for October. Anyway, today it was quite chilly from the early morning. I did this ciclot n on my own, as Gaby is out of town - I left home at around 11:30, and headed North towards the nearest point of the route, ~5Km north from here (Circuito Interior and Universidad). The chosen circuit is quite good and varied. And quite enjoyable, too.
And how did I fare? Well, not bad at all, I think... I did the ~37Km (32Km of the Ciclot n, plus the ~5 to get there - I came back by metro, as I was already too tired to be alert while driving in the traffic) in 2hr 13 minutes. According to my trusty Polar watch, I used 1831 Kcal (which is a bit less than what I eat per day), averaging at 138 heartbeats per minute.
Very enjoyable. This is one of the ideas that I most like that have been promoted by our local government: As Mexico City citizens, we need to claim back our streets. Also, we have to be responsible and think about different ways of moving around than by burning gasoline - Biking is fun, and it is not riskier than driving or walking. Try it!

I recently attended the Play! concert in Oslo, with a friend who was visiting Oslo from Bergen. The concert took place in an expo hall just outside Oslo, which was being used for the Electroworld electronics exposition. Wow, has that ever changed! I was there the first time, three-odd years ago, when it was arranged in Oslo Spektrum, and I was bitterly disappointed! It was nothing but a slightly larger Expert Electronics store. I was angry that I had actually been conned into paying for this. This year, it was completely different. They had a gigantic venue, with lots of interesting booths. Yamaha showed off some truly gorgeous digital pianos, and electronic drum sets, NRK was there, to show off their beautiful HD-1 HDTV production bus, and all kinds of interesting things. Even so, it was very clear that Expert was trying to get you to buy stuff. Something just doesn’t sit right with me about paying money to enter a store which is marketed as an expo. My previous experience with Electroworld meant I was not at all interested in repeating it. I wanted to catch the concert which was being held there. But they wouldn’t let me in unless I bought a ticket for the expo. Having already paid 70 EUR for a concert ticket, I was not enthusiastic about paying 15 more EUR to be allowed to enter the venue. But I relented, and was duly fleeced. I only got around 30 minutes of time before the concert started, so I didn’t get all that much time to look around. The first part of the concert started. The orchestra was the wonderful KORK (AKA Kringkastingsorkesteret, AKA “The Norwegian Radio Orchestra”). They started off with a beautiful fanfare by Nobuo Uematsu. However, their choice of music frustrates me deeply. They played some of the obvious classics, like Mario, Sonic The Hedgehog, some Zelda, but most of the time was wasted on some really lame music. It seems that their repertoire was not chosen based on the musical merit of the games, but on the current popularity. Why else would they be devoting time to some dull, insipid incidental music from WoW, Halo, and Silent Hill, without even an identifiable motif, as opposed to more of Koji Kondo’s wonderful Mario or Zelda tunes, or - damnit - Nobuo Uematsu’s Final Fantasy music? This is the thing that has left me the most disappointed. They practically ignored the elephant in the room of game music, Nobuo Uematsu. His compositions are absolutely beautiful, and they really shine when performed by a classical orchestra. You can practically tell that they’re meant to be performed by them, rather than some dinky Super Nintendo synthesizer chip. The Final Fantasy VI Opera, most of the Final Fantasy VII soundtrack… They then proceed to play One by Metallica. …WTF, you ask? WTF, indeed. Their justification was that it’s in the soundtrack from Guitar Hero. That’s absurd. And it wasn’t even a very good performance. It was still pretty funny that this very serious-looking classically trained vocalist slowly walked up on stage and gave a really good performance - he really did sound like the Metallica vocalist. Almost as an afterthought, at the very last, they play One-Winged Angel, one of the most dramatic and beautiful pieces from Final Fantasy VII. The audience breaks into absolutely wild applause. But - what - they bring in a METAL BAND on stage. A GODDAMNED METAL BAND! And they proceed to absolutely slaughter the whole piece! The choir and orchestra are drowned out by loud electric guitars with riffs that don’t make sense, and a drummer that can’t even keep the damned beat - the bass drum was all over the place! Seemingly not content with simply omitting his music, they have to absolutely demolish the performance of it. I actually have to say that this is by far the most disappointing concert I have ever gone to. Granted, my expectations were high, but just to witness a live performance by this wonderful orchestra of the Prelude, Anxious Heart, Tifa’s Theme, Eyes On Me, Fighting!, Still More Fighting! or Aerith’s Theme would’ve made the 85 EUR worth it. Judging from the audience’s applause when he first mentioned the Final Fantasy series, I think they agreed with me.