While developing web applications using Zope3, my Python web framework of choice, I always have the same issue: where should I store the UGC data? I usually used an NFS filesystem, using a Zope utility to manage the storage and retrieval of the files.

The most obvious disadvantage of this technique is that the NFS server is a single point of failure: if it disappears, there is no way to access the files from the application servers.

Another problem with this implementation is that all the UGC data files have to be served by the application servers, using resources which could be used to serve more clients instead of transmitting a static file over the network.

MogileFS is an application-level distributed filesystem which solves these problems, ensuring data integrity and redundancy, written by Danga. The only problem is that it is written in Perl, it works only with MySQL and requires WebDAV servers for the storage.

To solve my problem, I decided to borrow some of the ideas behind MogileFS and to develop my own distributed application-level filesystem. KoboldFS is written in Python, uses a PostgreSQL database to store the filesystem status and it is released under the GPL license.

I deployed it a few days ago on one of our production clusters, and it reduced the load of the application servers allowing us to serve the UGC files directly from nginx. You can find the source code along with a rough description here.

Many months ago I stumbled upon this German written meme about how users use their virtual desktops. I use virtual desktops since my very early Unix times (tvtwm on Sun Sparc SLC/ELC/IPX with greyscale screens running SunOS 4.x), so in the meanwhile I use them nearly everywhere the same way. Short Summary 3x5, no overlapping windows, either tiling or fullscreen, keyboard navigation, xterms, yeahconsole, FVWM, panel for systray. Window Manager of Choice My window manager of choice is FVWM since more than a decade. I tried others like Sawfish, Metacity and Compiz, but I couldn t get them behave like the FVWM I got used to, so I always came back. Since I hate overlapping windows, I use FVWM a lot like a tiling window manager. FVWM has this nice function to maximize windows so that they occupy as much space as available, but do not overlap other windows. This function was also often missing when I tried other window managers. I though do not want to use real tiling window managers since I have a few sticky windows around (e.g. the panner with the virtual desktops and xosview) and they shouldn t be overlapped either. Virtual Desktops Switching between virtual desktops is done with the keyboard only with Ctrl-Shift as modifier and the cursor keys. The cursor keys are usually pressed with thumb, ring and small finger of the right hand. Which hand presses Ctrl and Shift depend on the situation and keyboard layout, but it s usually either ring and small finger of the left hand, or pointer and middle finger of the right hand. So I m able to switch the virtual desktop with only one hand. I have always three rows of virtual desktops and usually four or five columns. The top row is usually occupied with xterms. It s my work space. The top left workspace usually contains at least one xterms with a shell and one with mutt, my favourite e-mail client since nearly a decade. At home the second left virtual desktop in the top row usually contains a full-screen Liferea (my preferred feed reader) while at work it contains the GNU Emacs main window besides two xterms. Emacs and the emacs server are automatically started at login. This also means that I switch the virtual desktops when I switch between mutt and Emacs for typing the content of an e-mail. Did this already during my studies. (At home mutt runs inside a screen, so there I just switch the virtual terminal with Ctrl-A Ctrl-A instead of the virtual desktop. Not that big difference ;-) The other virtual desktops of the the top row get filled with xterms as needed. Usually one virtual desktop per task. The middle row is for web browsers. One full screen browser (usually Conkeror or Opera) per virtual desktop, often opened with many tabs (tabs in Opera, buffers in Conkeror) related to the task I m accomplishing in the xterms in the virtual deskop directly above. The third row usually contains root shells for maintenance tasks, either permanently open ones on machines I need an administrate often (e.g. daily updates of Debian testing or Debian unstable machines), or for temporary mass administration (Linux workstations on the job, all Xen DomUs of one Xen server, etc.) using pconsole. yeahconsole Additionally I have a sticky yeahconsole running, an xterm which slides down from the top like the console in Quake. (It s the only overlapping thing I use. :-) My yeahconsole can be activated on every virtual desktop by pressing Ctrl-Alt-Z (with QWERTY layout, Ctrl-Alt-Y with QWERTZ layout). It s the terminal for those one-line jobs then and when, e.g. calling ccal, translate, wget or clive. Changes over time Of course the desktop usage changes from time to time: At work I have more than one monitor, so in the meanwhile the second row with the web browsers moved to the second screen with independent virtual desktops (multiple X servers, no Xinerama). The second row on the main screen at work is now used the same way as the third row with a slight preference for the permanently open shells while the third row is more used for mass administration with pconsole. At home I used XMMS respective Audacious for a long time (my FVWM panner and xosview are exactly as wide as WinAmp2/XMMS/Audacious, guess why:-) which usually was sticky the same way as the panner and xosview are. But when I started using last.fm recently, I moved to Rhythmbox (after testing some other music players like e.g. Amarok) which I use in fullscreen as I do with web browsers and the feed reader. So it occupies a complete virtual desktop, usually the second one in the middle row below the feed reader because I don t need a corresponding web browser for the feed reader. (Just found out that there is a last.fm player for text-mode, so maybe that will change again. :-) Another thing which changed my virtual desktop usage was the switch from a classical tabbed web browser (Galeon, Kazehakase, Opera) to the buffer oriented Conkeror. With a tabbed web browser I have either no overview over all open tabs (one row tab bar or truncated tab menu) or they occupy too much space of the browser window. That was another reason for more than one browser window and therefore more than one virtual desktop with fullscreen web browser windows. With Conkeror tabs are optional (and not even enabled by default), Conkeror uses buffer like Emacs and if you want to switch to another buffer, you press C-x b and then start typing parts of the buffer s name (e.g. parts of the URL or the web page title) to narrow down the list of buffers until only one is left or until you have spotted the wanted buffer in the list and choose it with the cursor keys. So the need for more than one browser window is gone. For a long time I didn t need any task/menu/start/whatever bar on my desktop. But since neither NetworkManager nor wicd have a comand-line interface (yet) and bluetooth seems also easier handled from the system tray my laptops also use either gnome-panel (big screen, long sessions with FVWM) or lxpanel (formerly used trayer; use it on small screen, short sessions with ratpoison or matchbox) on my laptops. It s sticky and always visible. (No overlapping, remember? ;-) The panel is usually at the bottom on the screen as by default with Windows or KDE, not at top as with GNOME and MacOS. Only on the OpenMoko, I have the panel at the top to be close to what I m used from Nokia mobile phones. Things I tried but didn t survive in my setup:

• Desktop icons nearly always covered if you use a tiling window manager. (I though use root window menus mostly for starting applications later occupying that space where I clicked. ;-)
• A button to minimize all windows. Only sissies without virtual deskops need that. ;-)
• Automatically scrolling logfile content on the desktop (root-tail, root-portal, etc) the space was too precious to not use it for xterms or web browsers. ;-)

Systems without Virtual Desktops Anyway, there are systems where I don t use virtual desktops at all. On systems with a screen resolution so small that there s not enough space for two non-overlapping, fixed font 80x25 xterms on the screen (e.g. on my MicroClient with 8 touch screen, the 7 EeePC or the OpenMoko) I do not use virtual desktops at all. On such systems I use all applications in fullscreen, so switching between applications is like switching virtual desktops anyway. My window managers of choice for such systems are ratpoison for systems with keyboard and matchbox for system without keyboard. With ratpoison you treat windows like terminals in GNU screen, so there are no new keybindings to learn if you re already used to screen (which I use nearly daily since more than a decade).

It s been quite the day. This evening, Jacob had a new first. He requested I read him an owner s manual for his bedtime story. Yes, I m sure years from now, he will still remember how to operate a Motorola W376g cell phone. He had found the manual in its box and had been carrying it around, reading it to himself for days already. I can feel him following in my footsteps - I remember pulling the car manual out of the glove box on long trips and reading it. For fun. So yes, Jacob chose an owner s manual over nice children s bedtime books involving caterpillars. This morning, I set out on what would be a 43-mile bike ride - home to Wichita. It was about 28F when I left. I had been wanting to ride from home to Wichita for some time now, and finally found the right day. The wind was at my back (mostly), the sun was shining (I even got burned a bit), and the ride was fun. It took me about 4.5 hours, including the 1.5 hours I spent for lunch and other breaks. I didn t take a completely direct route, but that was intentional. This is my second-farthest ride in a day, behind the time I rode 55 miles in a day for charity. But it is the farthest I ve gone in winter. I was taking the bike to the bike shop for its free 6-month tuneup. So I even had the perfect excuse to ride it. I hope to ride it back home if the weather is cooperative. As for the third milestone, while I was riding to Wichita, Barack Obama and Joe Biden were riding a train to Washington. I recorded it, and have been watching it this evening. Remembering all they said during the campaign, seeing how they act it really does give me some hope in this country s government, some hope that some important things will be accomplished in the future. One TV commentator pointed out that the Bush administration carefully avoided using the word recession as long as they possibly could, while Obama is trying as hard he can to be straight and direct with people about the situation. I appreciate that. What a milestone the next few days represent for the country.

The phrase on this title is often attributed to Pancho Villa (1878-1923), Mexican Revolution leader. He had a fame of cruelty, killing suspects before even questioning them.
Today, it started as a very nice day. I had even time in the morning to find, fix, upload and send upstream a trivial bug in libgruff-ruby... At 11:00, I left the Institute as my father came to the city to do some paperwork... We sat having a cup of coffee in a restaurant near the office we had went to at around 12:00, and my phone rang.
And it was from work. That's never a good sign. My boss told me he was facing a massive virus infection, and decided to disconnect the firewall. I corrected him - that will do no good once the virus is in our system, if you want to disconnect anything, disconnect all of our switches.
Came back, and found him and my coworker stunned and not knowing what to do. He says, the antivirus alarm went off almost simultaneously on the two computers he had on his desk, and in few minutes over 15 computers all over the Institute were ill. The symptoms? Programs not showing up in the taskbar, copy/paste functionality b0rken, many programs misbehaved or just didn't open... They were grimly facing a complete recovery operation they have grown used to: The whole OS has become corrupted or destroyed, we will have to open the computer, extract the HD, install it elsewhere, back it up, reinstall OS and applications, restore the backup. Yes, I know too many extra steps are included here, but I have come to accept their ways of dealing with Windows. Nobody says dealing with Windows is fun. I like my work to be fun, so I stay clear of theirs.
I insisted on turning back one one of the switches, the one for the servers and my machine (and some more in the same physical area). OK'd. But they didn't want to switch on any other switch, so a traffic capture (tcpdump / wireshark) led nowhere - but at least it gave my my Google back.
They have configured the antivirus software we deploy to all of the Windows machines in such a way that it deletes upon sight any malware - And when they manually scan, they blindly hit Delete whenever anything is found as well. Of course, no infected binary was left alive for me to inspect, and the machines were dead. But I was able to glimpse at the name of the deleted file: rpcss.dll.
After googling a bit - Bliss! Joy! I found the answer. So here is the set of interactions, and how they led to this killing spree. Please remember I am a Windows newbie and speak just out of guesswork.

1. This is a fast-spreading virus. My friend Rub n at DGSCA suggest it might be related to this report submitted today; at Barrapunto there is a thread about another virus that appeared four days ago, infected 1.1 million Windows machines on its first day, and so far is around the ninth million. Update: Equivalent thread at Slashdot, for the Spanish-impaired.
2. The virus infects at least two copies of a system binary: %system32%\rpcss.dll and \Windows\ServicePackFiles\i386\rpcss.dll. Windows uses the second one to restore the first one in case it is damaged, if I understood correctly.
3. The antivirus does not detect the infection when the library files are written, but when they are linked, so it only spots it the next time %system32%\rpcss.dll is brought into memory.
4. This is a very common library - It takes care of, well, RPC. So, quite probably, this file will be linked again on the next program launch - or accessed when a running program requires anything not currently in RAM? Dunno. The thing is, the library gets linked.
5. The antivirus will happily tell you it has killed a threat! Your nice RPC library is now defunct. M talo, luego virig as!
6. So, of course, notifying the taskbar of a new window appearing, or clipboard actions, or whatnot will refuse to work.
7. Machine restart, full system scan requested. The antivirus finds de second copy of this library in the master directory (\Windows\ServicePackFiles\i386). The virus used this location so that Windows won't restore a clean version over it. But yes, it will fall again under the claws of the antivirus... I guess. Anyway, the antivirus offers to delete this file as well, and does so.
8. User is desperate. My coworkers are desperate. I am... mildly annoyed?

Once I found this line of thought... I went to a working machine, inserted my flash memory, and copied %system32%\rpcss.dll to it. Went back to a sick machine, and ran cmd. Then, it was just matter of copy f:\rpcss.dll c:\windows\system32, a simple reboot (it never hurts to reboot in Windows!), and problem solved!
Oh, as a side rant: I find it extremely annoying and sad that many people I know, sometimes with more experience as a computer operator/supporter than what I have of experience as a living human being, are so scared of using a command-line interface. They were dismayed at seeing no drag-and-drop and no copy/paste functionality were available! copy is not an option.
Anyway... Today was an experience on how a simple, mostly-harmless and quite-fertile virus is able to be terribly magnified by the presence of a trigger-happy antivirus.
Why won't they give themselves a chance to try something else? Say, GNU/Linux? :-/

As I move to having more and more machines on my internal LAN, I felt the time had finally come that I sit down, and rebuild my network to take advantage of things such as gigabyte networking, LDAP, single-user sign on, and so forth. I'm doing partially for fun, and partially because its an interesting experiment to see how Linux from an IS environment compares to a Windows 200x IS environment (one of my former jobs was a 2000/XP/2003/Vista sysadmin position).

So, here's my current network setup
blacksteel <- *wireless* ---------------------------------------------- cerberus <-> Internet
/
dawn <------ *wired*-----------------------------------------------------
/
360 <---- *wired* -----------------------------------------------------

Online machines:
cerberus - WRT51GS
backsteel - My laptop
dawn - Development machine
360 - Xbox 360, used to play media from blacksteel

Offline machines (aka, machines I have, but haven't fired up since moving:
helios (PowerMac G4)
apollo (old Dell P3)
junker (RS/6000 rescued from the dumpster, might be dead)
alexandria (NSLU2; gave up its plug for dawn)
coldfusion (Coldfire Board, might be dead; ethernet controller is faulty, but might be able to use a USB based one to breath some life into it; can't autoreboot due to built in bootloader not supporting it; and no JTAG to sanely change the default bootloader).
siren (old MacBook Pro, has a dead internal HDD, but runs fine from an external hard drive. Was my Debian test box until its HDD went to dawn)
exodius - second WRT54GS used to be part of a WDS bridge.
unnamed dev box (not here yet, but likely soon).

Of all these machines, only apollo has a wireless card which ATM is non-functional. In addition, the wired bits of my network are 100Mbps, with a g based wireless hotspot (WPA secured). Futhermore, blacksteel, helios, and siren have gigabyte ethernet. apollo has 100MBps ethernet card. alexandria and dawn have 10MBps, which is painful, especially for NFS root.

I'll drop another 1Gbps NIC into apollo, replacing its wireless card, and give dawn, alexandria, and maybe coldfusion USB based NICs once I get around to resurrecting systems (alexandria and coldfusion don't have hard drives at the moment)

What I would like to do is use an Linux-based router and replace Cerberus. Helios has two gigabyte NICs, so it will take up this duty, as well as provide DHCPv4, and radvd (for IPv6) for the internal network. It's an old computer, and has an onboard model, and its position in my apartment will be close to a phone jack; maybe I'll set it up so I can dial in from outside the LAN in case something goes down (although my phones here are VoIP based so I dunno how useful that's going to be :-)).

Another box (I might task this to apollo, or helios) will run LDAP and NFS services, providing both a netboot based installation with preseed for fast re-installation, and NFS home folders for all machines except blacksteel (unless someone knows a great solution for having a laptop sync NFS and local home folders. helios will run mail, news, and any other untrusted net facing services, with everything else shielded behind it. All machines will run IPv4 and 6.

Anyway, this is the start of my plan in a nutshell, and I intend to continue discussion as I slowly build and implement this updated setup. Wish me luck :-).

Asheesh Laroia explained how to use ssh as a SOCKS proxy if you re behind a firewall that allows you to ssh to some machine not restricted by it: ssh -D. (I don t know how common such firewall setups would be, but the one at my Uni does indeed allow ssh to the outside over authenticated wireless.) With -D, ssh will listen in a local port, and behave as a SOCKS proxy. Set it up like this:

  % ssh -N -f -D localhost:4444 external-machine.example.com

And then you can point your SOCKS-capable application at localhost:4444. Chances are, however, that the application you want to use doesn t support SOCKS (like, in my case, Git). You can use tsocks then, which LD_PRELOADS a library that will divert an app s TCP traffic through the SOCKS proxy. To use it:

  % cat >~/.tsocks.conf
  server = 127.0.0.1
  server_type = 5
  server_port = 4444
  % tsocks git pull

AIUI, this is better than setting -L forwarding because you don t need a forward for each host whose port you want to access. If you re interested in starting ssh in SOCKS mode from .ssh/config, the configuration item is DynamicForward.

After upgrading an Ubuntu mail server from Hardy to Intrepid, two users could no longer connect via SSL to send email though sendmail. One was using msmtp and the other was using Outlook Express. The msmtp issue was tracked down as a supposed deficiency in msmtp. However, this left Outlook, which is neigh-impossible to debug. From the Debian msmtp bug linked from the Ubuntu bug, it seemed that the root cause was the server sending too much data during the initial connection. Packet captures of an Outlook connection seemed to back this up: Outlook negotiated STARTTLS fine, and then just never responded to the SSL handshake. It seems that something (openssl? sendmail?) changed between Hardy and Intrepid so that instead of using the /etc/ssl/certs/ca-certificates.crt file just for verification, its contents were now being sent during the SSL handshake. (I reduced the number of configured certs with sudo dpkg-reconfigure ca-certificates , and checked on the size of the handshake with openssl s_client -connect server:port wc -l .) It spewed 143 certs sent at every connection. Unsurprisingly, it seems some clients were choking on it (I would like to note that Thunderbird behaved correctly). In the end, I configured my CAfile to aim at just a single CA (the CA used to sign the server s SSL key), and that fixed both msmtp and Outlook. How fun.

Yesterday evening I received a quite nice mail - the people behind Linux New Media and the LNM Award asked me if I want to participate in the Jury for the 2009 edition of it. Sure I want. :) The process seems to be a very simple one (good), the Jury members make nominations for the various categories and then later on vote on them. Now, while I already thought about it and know candidates I will nominate, what about you? I list the categories for this years edition below. Do you think something/someone/whatever should be nominated in those categories? Why? Note that only products/projects/people/organizations/whatever that have been very significant in 2008 should be nominated. This editions categories are - Outstanding Contribution to Open Source / Linux / Free Software

• Most Linux / Open Source-Friendly Hardware Vendor
• Most Innovative Open Source Project
• Best Open Source Contribution for Mobile Devices
• Best Open Source Programming Language
• Most Significant Contribution for Security in Open Source

If you want, feel free to tell me what you think should go in those categories. Leave a comment here or send me email. If you also add a little reasoning why, even better.

I received a SanDisk Sansa e260 (v2) for my birthday. It was one of those presents that acquired for myself on behalf of someone else because it worked out easier that way. I bought it for cheap from Woot! I thought I'd use it to play podcasts while I'm at the gym. We've been listening to NPR for a while now, and there's lots of excellent programming, and it's all available in download for as well. Since I don't spend all that much time in the car, I'm not listening to much of it. The first mission was to upgrade the firmware. Getting the player into MSC (instead of MTP) mode with the original firmware was a little tricky, but I got there in the end. Then I had to locate the firmware in a Linux-installable format. I think I ended up resorted to using Windows, but I later found the raw firmware image without needing an installer. I'm using Rhythmbox at the moment to download the podcasts. I thought I'd try out its MTP support, just because it actually had it, but either Linux 2.6.26 has a problem with MTP, or the player is particularly flaky doing it with Linux, because I just seemed to get a pile of USB errors when connecting the device in this mode, so I'm sticking with MSC for now. Oh, and I had to put a .is_audio_player file in the root of the device to make it show up in Rhythmbox. I later discovered that it worked most optimally if I put

audio_folders=MUSIC/
folder_depth=2

in the file. I'm still trying to come up with an optimal way of managing the audio I want to listen to. It seems somewhat tedious to drag and drop all of the podcasts from Rhythmbox to the device. I kind of want some sort of plug and play auto-sync method. It seems I can't use a connection-triggered rsync, because some of the podcasts seem to get downloaded with non-FAT-filesystem-friendly filenames. Rhythmbox seems to take care of this by doing some sort of automagical renaming at transfer time. That said, not everything is showing up in the music list with sane names. There's a few things that have an unknown artist I think. That may be a deficiency in the ID3 tags of the MP3s themselves though... What I really want some way of getting home, hooking up the player and just getting what's new, without expending too much effort (at the time of hooking up the player, anyway. I'm happy to expend some effort getting to that point) The next weird thing is that the device wants to talk USB 1.1. I'm not sure what is at fault here, but I can, allegedly at least, make the device reconnect at USB 2.0 by removing and reloading the ehci-hcd module after connecting it. Here we have the device initially attached:

apollock@icarus:~$ lsusb -t
Bus#  5
 -Dev#   1 Vendor 0x1d6b Product 0x0002
Bus#  4
 -Dev#   1 Vendor 0x1d6b Product 0x0001
Bus#  3
 -Dev#   1 Vendor 0x1d6b Product 0x0001
   -Dev#   9 Vendor 0x0781 Product 0x7423
Bus#  2
 -Dev#   1 Vendor 0x1d6b Product 0x0001
   -Dev#   6 Vendor 0x413c Product 0x8103
Bus#  1
 -Dev#   1 Vendor 0x1d6b Product 0x0001

lsusb says that bus #3 is ID 1d6b:0001 Linux Foundation 1.1 root hub Then if I do a spot of presto-changeo...

apollock@icarus:~$ sudo modprobe -r ehci-hcd
apollock@icarus:~$ sudo modprobe ehci-hcd
apollock@icarus:~$ lsusb -t
Bus#  5
 -Dev#   1 Vendor 0x1d6b Product 0x0002
   -Dev#   3 Vendor 0x0781 Product 0x7423
Bus#  4
 -Dev#   1 Vendor 0x1d6b Product 0x0001
Bus#  3
 -Dev#   1 Vendor 0x1d6b Product 0x0001
Bus#  2
 -Dev#   1 Vendor 0x1d6b Product 0x0001
   -Dev#   7 Vendor 0x413c Product 0x8103
Bus#  1
 -Dev#   1 Vendor 0x1d6b Product 0x0001

Bam! It's moved over to bus #5, which lsusb says is 1d6b:0002 Linux Foundation 2.0 root hub Weird. I wonder if I should try running Rockbox on it? Seems not, it's only for v1 players. Oh well.

Making a simple PerlResponseHandler that does something like this:

sub handler  
    my($r) = shift;
    $r->content_type("text/plain");
    $r->print("You suck.\n");
    return Apache2::Const::HTTP_BAD_REQUEST;
 

doesn t make what you d be expecting. This is because mod_perl assumes, since you are specifying content type and something to print, that this is a legit and valid (200-ish) request. This is what you get:

POST http://myserver/resource --> 200 OK
Connection: close
Date: Mon, 05 Jan 2009 23:02:57 GMT
Content-Type: text/plain; charset=UTF-8
Client-Date: Mon, 05 Jan 2009 23:03:00 GMT
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
[ some headers snipped ]
You suck.
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>200 OK<</title>
</head><body>
<h1>OK</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

So, yes, I was expecting to return a 400 Bad Request return with just You suck on the response body. mod_perl disagrees, which, kind of sucks if you are trying to make a RESTful service with mod_perl HTTP handlers and filters. To avoid this behaviour, this is one way to go:

sub handler  
    my($r) = shift;
    $r->status(400); # or the one you are interested in
    $r->content_type("text/plain");
    $r->print("You suck.");
    return Apache2::Const::OK;
 

You can also take a look at status_line that overrides status. You can also avoid the content_type method call if you are sloth enough. Now, this is what you get:

GET https://myserver/resource --> 400 Bad Request
Connection: close
Date: Mon, 05 Jan 2009 23:27:08 GMT
Content-Type: text/plain; charset=UTF-8
Client-Date: Mon, 05 Jan 2009 23:27:10 GMT
Client-Peer: 209.234.249.38:443
Client-Response-Num: 1
[ some headers snipped  ]
You suck.

And so, beer awaits.

This morning, I got a mail that made me very happy. I followed it up a bit, and some hours later, echoed it over the pkg-ruby-extras list. And, yes, a blog posting won't hurt :)
I have done some rantings on why it is so painful to integrate cultures such as Debian and Rails. Those rants were part of quite a large rant-net and attracted a fair share of traffic/comments over here. Flamefesting over your blog is fun! :-) but anyway, I am delighted to say that at least some people worth their weight in code were watching, interested.
The mail I got this morning (yes, follow the links above!) was from Hongli Lai, one of the very nice people at Phusion - The people behind Phusion Passenger (a.k.a. mod_rails) and Ruby Enterprise Edition. Yes, people with a very different mindset to mine (specially when it comes to being a 100% Free Software person).
Hongli invited me to try their new DebGem service (still in Beta, although quite usable as it is). They are offering an auto-built full repository of Rubyforge, translated to Debian packages. They are currently supporting Debian 4.0 (Etch) and Ubuntu 8.04 and 8.10 (Intrepid and Hardy). And, yes, installing any arbitrary Ruby module is now just as easy as aptitude install libsomething-ruby. For over 20,000 Gems.
There is a catch, yes. The service is currently free, while they finish the public beta period. Their pricing is available.
Best luck to you guys. And... Shall you enjoy fierce competition from Debian proper! ;-)
[Update]: They just posted the official Beta announcement for DebGem

(or echo "blosxom hold" sudo dpkg --set-selections) A few people had pointed out to me that my blog was appearing weirdly in Google Reader. I myself read my blog via Planet Debian and Planet Linux Australia, where it appears fine, so I hadn't noticed the problem first hand. I did some digging tonight with the Feed Validator, and I think I've figured it out. Funnily enough, it all started with a previous session with the Feed Validator... I wanted my blog to be UTF-8, with it being the new hotness and all. Blosxom was spitting it out as ISO-8859-1. I've just learned from this evening's debugging, that this is the behaviour of CGI.pm's header() function if not provided with the -charset parameter. I'd previously worked around this by defining my own content_type for the RSS and HTML flavours. In this file, I just put text/xml; charset=utf-8and text/html; charset=utf-8, respectively. Unfortunately, this then breaks Blosxom's internal HTML-escaping-for-RSS-feed code, which is conditional on the content type being XML:

if ($content_type =~ m \Wxml$ )  
        # Escape <, >, and &amp;, and to produce valid RSS
        my %escape = ('<'=>'&lt;', '>'=>'&gt;', '&'=&gt;'&', '"'=>'&quot;');  
        my $escape_re  = join ' ' => keys %escape;
        $title =~ s/($escape_re)/$escape $1 /g;
        $body =~ s/($escape_re)/$escape $1 /g;
       

This block of code was being ignored because the regex was anchored. "text/xml; charset=utf-8" didn't match "\Wxml$". Whenever I last had an attempt to make my blog's feed pass the Feed Validator, I must have edited the Blosxom source code to make this regex not be anchored. Then on November 11, 2008, I updated my Blosxom package to 2.0-14+etch1, to address CVE-2008-2236 (incidentally, there doesn't appear to be a Debian Security Advisory for this?), and my local modification got clobbered. That'll teach me to make local modifications and not put the package on hold. Blah. It seems I'm screwed, there's no way to have a well-formed UTF-8 RSS feed without requiring a source-code modification to Blosxom. I've either got to change the way the header() function is being called, or I've got to change the regex to allow me to specify a charset in the flavour file, and not disable HTML escaping. Speaking of screwed, Blosxom seems incapable of making a well-formed RSS 0.91 feed anyway. It wants to put the <pubDate> tag inside an <item>, which is a violation of the DTD. There's supposed to be one for the whole <channel>, not one per item (or "story" in Blosxom-flavour-speak). You can't just move that from the story flavour file to the head flavour file, because a pile of variables that you'd otherwise use seem to only get set on a per-story basis. Double blah. I'm starting to feel like Blosxom is outliving its usefulness, but I really do not feel like embarking on a wholesale move to a new blogging platform, I'm fairly entrenched.

Some people consider the gnome usability guidelines a nuisance and some consider certain applications way too simplistic. While it is really hard to get the usability right, it's well worth it. We need to keep in mind that as computer oriented people we tend to see things differently. Things that are simple to us aren't really that simple to the "normal people". But one of the simple things we can do the insure that the software we write serve the people it's designed for is to remove all not needed pop-ups and questions. A good way to detect these would be to ask yourself why would a user choose anything else but the most logical option. It's kind of hard to explain, so lets pick an example: Firefox is updated, the first question it usually asks after upgrade is about incompatible extensions. The user is presented with choices, check for updates or cancel. Now, we are dealing with internet browser so the user should be connected to the internet so there is no problem with checking for the updates, we can rule out that scenario. The other scenario that I can come up with is that a developer doesn't want to update some certain extension. So at least I can't come up with any reasonable scenario why someone would want to select anything else but to upgrade the main option of upgrading the extensions. Why not just leave out the option and instead do the upgrade automatically. If you wish to be transparent, you can show the user that you are doing the upgrade. Or you can do like some applications do and just do it without bothering the user with the options. I know I sound like a Google fanboy, but Google generally gets this right. Their applications skip out all upgrade related notices and just do the upgrade. Regular user doesn't want to upgrade because the user has been scared with incompatibility notices and upgrade checklists for so long. Just going ahead with the upgrade in complete silence keeps their software up to date as well. Another example would be from few years back: The Ubuntu installation. Back in the day Debian was working on Debian Installer, which is also used as the main installer in Ubuntu alternative installation media. Debian Installer is capable of doing most things silently, but with Debian it by default asks a lot of questions. It doesn't matter, since most people who install Debian can be categorized as developers. But in my opinion, the thing that made Ubuntu a success was that it doesn't ask the questions that can be answered without asking the user. So, back to usability. There are basically 2 camps, the "normal" users and the developers. Developers want and need to see a lot of the backend behaviour, just to debug problems. Currently a lot of the open source software is focused towards developers while they are gaining grounds on the "normal" population as well. We should start focusing on the users for a change.

While it's obvious that being a kernel developer is absolutely the best thing anyone in life could ever aspire to, I'm finding it increasingly difficult to justify not having just given up and gone off to be an analyst instead. Two stories stuck with me this week. The first claims that we can expect Android netbooks to be on sale within a year or so. The reasoning behind this? Android runs on x86 and has a MID profile. Oh, yeah, and it sounds like a cool idea. Sure, we need to ignore straightforward facts like, oh, I don't know, it being UNSPEAKABLY PAINFUL TO DO OS DEVELOPMENT ON ARM, and hence it being NO SURPRISE WHATSOEVER TO FIND THAT BUILDS AND RUNS ON COMPUTERS THAT ACTUALLY GO FAST AND SIT ON PEOPLE'S DESKS. Or that MID refers to the class of devices like Nokia's internet tablets or, well, any of these and not a netbook. And, leaving those, the evidence we have for this is that "It sounds like a cool idea".

It's easy to write an article on why something sounds like a good idea, and even easier to make it sound like a good business plan if you're not the one who's going to lose money on it. But yeah, maybe Google's going to push for Android on netbooks. It's going to be a kind of crowded place to be, what with Microsoft and Intel and Canonical and Xandros and half a dozen Chinese Linux distributions that you've never heard of and really hope never to again once you've seen their distributions, but it's possible. And so these guys might even be right. But the sum total of their insight here is that (a) Google have a product, and (b) Google might want to enlarge the market for that product. The rest is a collection of extraneous facts designed to make you think that they've actually done something more impressive than typing make, and frankly if that's enough to get you noticed then gentoo users the world over are missing out badly.

It's ok, though. There's worse. Google are making a secret OS, which by my count means that there's been eleventy billion people inside Google working on a fucking OS for the past TWENTY YEARS and this time we can tell not because people have, y'know, gone to Google and seen it, but because they're busy leaking information about their secret OS by removing the user agent string from a bunch of their web traffic.

Of course, as Clint points out in his article, Google's a web company and so would want to produce a web OS. The obvious way to develop this web OS would be to, uh, run it on top of Android, an OS currently entirely unsuited to desktop use due to little things like the lack of decently accelerated framebuffer drivers on any hardware you can currently obtain (rather than, say, any of the operating systems already deployed in Google, all of which share one striking feature - the ability to RUN A WEB BROWSER AT A DECENT SPEED). And even though basically everyone in Google seems to have an Android device, it'd be vital to prevent anyone from knowing that they used Android to browse the web, so scratch the user agent.

But that makes no sense. So what Clint's clearly getting at is that people using the Google web OS are browsing the web using the Google web OS, and so remove the user agent string to avoid leaking that information. That makes sense, up until you actually think about it. A web OS would run in a web browser. Why would you run a web browser in a web browser?



Google's not an OS company. They don't employ enough people to develop a full OS and releasing a Linux derivative themselves would just be a way of tarnishing their brand in a hideous manner. Any move into the home would have to be in the form of appliances with known hardware configurations, and seriously, what's the point? It's not a big enough market. If Google's going to push applications, it's going to do it through the browser. And if it's doing it through the browser, then right now it doesn't matter what the underlying OS is. There could be any number of reasons for the mystery of the missing user agent strings, but if it's because Google are trying to prevent people from working out that they're developing an entire software stack that they're going to push out onto arbitrary hardware then I have an Android netbook to sell you.

That'll be $50,000, please.

I ve owned two different GPSs in the past: a Garmin GPS III, and the Garmin eMap. Both are based on similar underlying firmware. This week, I did some research on more modern GPS units and decided to buy a Garmin nuvi 500. Here s my review. Overview The Garmin nuvi 500 is one of only two models in the nuvi line that are waterproof. The nuvi 500 and 550 are both designed as hybrid units: useful both in the car and outdoors. The 500 includes street-level maps (they appear to be the same quality as Google) for the entire United States, detailed topographical maps for the entire United States, and a global basemap. It also includes a microSD slot (or is it miniSD - I forget) for additional maps that you can buy anything from marine maps to other countries. It also includes a default POI (points of interest) database containing over 5 million points: restaurants, gas stations, parks, hospitals, you name it. Most contain an address and a phone number in additional to the coordinates. Unlike GPS units that you find built in to some cell phones, this is all stored on flash memory on the unit: no Internet connection required. The nuvi 500 is a portable yellow pages, topographical map, and incredibly detailed street atlas all in one. Car Use The nuvi 500 comes with a car charger and suction cup windshield mount in the box. (It doesn t come with an AC charger, but it can charge over USB.) It also comes with yay a user-replaceable battery. The windshield mount is very sturdy and I am happy to have one that isn t permanent. In the car, the device performs admirably. I have read some other reviewers that have compared its routing to other GPSs and found that the nuvi generally picks the same route as Google, and almost always a better route than other GPSs. If you deviate from the selected route, it automatically re-calculates a new route for you. It will show you the next turn coming up, and has either 3D or flat map displays. There s a one-touch where am I feature. It displays your current coordinates, some sort of street address suitable to read to someone ( 1234 S. Main or I-95 exit 43 type of thing), along with buttons to show you the nearest police stations and hospitals. The unit also features multi-stop routing. You can either tell it where all you re going in a defined order, or tell it all your stops and let it create an optimal route. This feature works, but it your stops are close by (and involve some of the same roads), it may wind up skipping some stops thinking you ve made your first one. The speaker announces upcoming maneuvers, though it doesn t have a synthesizer for street names. It s also supposed to work with Bluetooth headsets, though I haven t tested that feature. I found the map quality to be excellent. It seems to be on par with Google (in fact, I think both use data from Navteq). I was surprised with how many country dirt roads and side roads it knows about and of course, regular city streets are all on there, with one-way indications and the whole lot. In fact, the quality of coverage was so good that I was surprised when it missed roads or got things wrong. Out in rural areas, or small towns, this happens from time to time: it thought that an abandoned railbed was an unnamed road and wanted me to drive on it (it was clearly not drivable), and in some other cases also thought some abandoned roads were still drivable. I don t think this is a device issue though; it s an underlying data issue, and everyone else probably has the same problem. Its arrival time estimates are quite accurate, and its interface is smooth and easy. It has some optional accessories I haven t tried, such as real-time traffic reports from wireless sources, boating mode, etc. Outdoor and Geocaching Mode Our other main use for the device is outdoor hiking and geocaching. It really shines here. It has special support for the GPX files from geocaching.com. The device supports paperless caching . Not only can it put caches on the map, but if you download the GPX file for your caches, you ll also get the full description, hint (behind a separate button, of course), most recent 5 logs, and the like right there on your screen. You can also log your finds on the device, and upload a file from it to geocaching.com later to log them on the site. This is an incredible time saver over my old method: printing out a bunch of maps, downloading waypoints to the eMap, taking notes, then logging things later. I found outdoor mode not quite as refined as the auto mode, however. It kept forgetting that I wanted to use an arrow as my current position indicator (the default hiking boots for walking mode didn t provide an accurate enough direction indication for my tastes). Finally realized that saying don t ask me about the configuration was the key to getting it to remember the configuration. It sometimes took a surprisingly long time to realize we weren t standing still any longer. On the other hand, the quality of the GPS receiver was amazing. It even got a strong signal in my house. And I wasn t even sitting at a window. The topographical maps are a nice addition, and the breadcrumb mode is always helpful when geocaching and hiking, too. My natural way of holding it meant that I accidentally turned it off a few times, because I had a finger holding it right over the power button. But it powers back up and re-obtains the signal quite rapidly. The different modes (automobile, outdoor, bicycle, and scooter) are mainly different collections of settings: 3D map or flat, what indicator to use, to navigate off-road or along roadways, to ignore one-way indications or heed them, etc. PC link The nuvi 500 has a USB port. Plug it into the PC, and you see its internal flash drive, vfat formatted. You can upload and download GPX files there, store photos if you like. Be careful what you mess with though, because its built-in maps and system data are also on that drive. If you have the SD card inserted, that will also be presented to your computer when you plug the device in. Garmin has some Windows software to make it easier to upload/download stuff, but I haven t tried it. Annoyances As you can tell, I really like this GPS, but there are a few things about it that annoy me. The #1 thing that annoys me isn t actually the GPS itself, but Garmin s website. I went to register the device online, but it wouldn t let me because I don t have the Windows-only browser plugin on my Linux machine, and it wanted to talk to the GPS for some reason. I went to send them a support request about that, only to discover after I had typed in the request that their email us form is broken in Firefox on all platforms. Bad show, Garmin. The on-screen keyboard (it s a touch-screen device with only one hard button for power) isn t QWERTY layout; it s alphabetical layout, and it makes me inefficient when entering data. I found myself logging my finds on the unit, but taking notes about them on paper because that was faster. Garmin has a feature listed on their website for a toggle between QWERTY and alphabetical layout, which they apparently offer on only their more expensive GPSs. What? There is no reason to not but that in all your firmware. The device lacks a few features I was used to on my GPS III and eMap. It doesn t support any kind of real-time position indication to the PC; all communications is just accessing stored data on the internal flash drive. I used to think that was a nice feature, but in reality, I haven t used it in years. It also lacks the display of the exact location of each GPS satellite, though the incredible quality of its receiver means that I don t really care any more. (I used to use that information to help figure out which window to put it by if in a car/train or something.) It also lacks the level of configuration that was present in the settings screens on the older units. There s no battery saver mode (sample every 5 seconds when going at a constant velocity instead of every 1) like the older units had. The sun and moon screen likewise is gone, but added is awareness of timezones; the Nuvi 500 can show you the local time at your present position synced up with the GPS satellites. The compass is not the most helpful screen, though after some practice, it is functional. The documentation about it is confusing, but really the thing that was more confusing was this: in walking mode, the arrow that indicates what direction you re walking in updates faster than the arrow that indicates what direction you should be walking in. Once I realized what was going on there, it was easier to use. The compass does tell you what your bearing is, in degrees, but only when you are not seeking a destination. It will not tell you the bearing to the destination, though you can estimate it from the simulated compass face on the display. When seeking a specific point, especially through terrain with obstacles such as trees, it is useful to be able to use a compass for the final approach because a GPS unit can t tell you which direction it s pointed only which direction it s moving, so when you re not moving or moving slowly, it s not helpful. I did some experimentation today with the compass screen, as well as my real compass, and was able to navigate to my destination rather precisely using the combination of them. That said, a more functional compass screen would still be better. Conclusion Overall, I m very happy with the nuvi 500. It s not the same as a top-of-the-line device in either the outdoor or automotive category, but on the other hand, it s cheaper and more convenient than buying two devices with similar features. The geocaching features are excellent, the build quality is excellent as well. The system is stable and performs well. (Some other reviews worried about whether the case is solid enough; it seems quite solid to me.) I wish there were a faster way to toggle between 3D and flat map views, and forgetting about my walking mode icon is annoying, but other than that I have very little to complain about. Garmin s geocaching features (found on this unit and several other in their lineup) is great.

For those following d-devel, you may notice that I've recently been working on improving one of the cornerstones of Debian infrastructure; the Debian Archive Kit, or dak for short. Most DDs and DMs don't notice dak exists expect when trying to determine why their latest upload was rejected, and then yelling at the powers that be. I'm here to shead some light on this mythicial beast.

First off, a quick history lesson:

dak (also known as projectb) is a replacement for Debian's original archive software, known simply as dinstall. dinstall itself was a fairly large perl script that does what dak process-unchecked/process-accepted does today. James Troup did a fairly decent summary of dinstall, and its issues

James Troup's README.new-incoming (from dak's git repo):

The old system:
---------------
o incoming was a world writable directory

o incoming was available to everyone through http://incoming.debian.org/

o incoming was processed once a day by dinstall

o uploads in incoming had to have been there > 24 hours before they
were REJECTed. If they were processed before that and had
problems they were SKIPped (with no notification to the maintainer
and/or uploader).

dak's first commits were in 2000, and rolled out onto ftp-master.d.o sometime in 2001 or 2002 (I can't find an exact date for this). Since then, dak is also used on security.d.o, and on backports.org (fun fact for bpo people; the dak installation there is now up to date, and tracking git's tip).

So now that you know the history lesson, what specificially does dak do is the next question. Simply put, dak is the glue that binds the rest of the Debian's backends together; both britney and wanna-build/buildd depend on it. It handles management of uploads to the archive, handles stable release updates, as so forth. It is also the only Debian archive software that uses an actual database backend, and scales fairly well handling over 10,000 packages, and 12 architectures. Unfortunately, there are also a lot of issues with dak as it stands.

Sections of the code base have bitrotted over the years; legacy and legacy-mixed support have died, the import-archive function is shot (more so now than ever, see below), the test suite is non-functional (never a good sign), the docs are out of date, and in many places non-existant, doing a release (both point and full) requires editing the database and so forth.

In addition, dak, while written in python, is written in a fairly procedural style, and and some very ugly code in some places. For instance, the original Debian Maintainer code was handled by having the uid's in the database prefixed by dm: vs having a flag somewhere, and had some hardcoded variables like checking for "unstable", as well as quite a few bugs which caused interesting behavior when uploading to a non-unstable suite such as experimental or one of the proposed queues. (for those of curious, I recommend checking the dak git tree to see what the old DM code looked like, and then aside from the design, find the two major bugs which caused a lot of the weirdness with DMs). It should be stated that the last merge from redid the DM code and design sanely using the new update framework.

These issues have lead to the genesis of the dak v2 project, which is an attempt to replace dak with a module, rewritten from the ground up to be more secure and modular, although its not gotten very far as of writing. I personally don't believe that the current iteration of dak is so bad as scrapping and rewritting is necessary. Instead, I've been working to implement v2 features in dak by aggressive refactoring and cleanup, with the hope of negating the need for a rewrite.

So now thats out of the way, I bet you probably are interested in my .plan for dak. Well, lets go over I've implemented so far.

* An update database framework for dak, which will allow for easy database upgrade and migration, vs the "does it work yet?" approach to applying schema updates. Simply type dak update-db, and your done!

* 822 formatted output for queues (http://ftp-master.debian.org/new.822); this information is now used on DDPO pages

* Rewriting DM management code to have more of a brain than the previous implementation.

What's next on the TODO list

* Content file generation from the database (part of removal of apt-ftparchive, but thats another blog post ;-)).

Oh, as a side note to my current readers, my blog has changed names to "Notes from Underground", after one of my favorite novels, and futher in reference to exploring the mysterious underground that is Debian's backend code. We're also now on Planet Debian :-).

A few days ago, I forwarded a mail from Sven Luther to the debian-vote and debian-project mailinglists. The reason I did so was that, apart from the last two paragraphs, the mail's tone was reasonable, polite, and did not try to rehash the old arguments. Also, and more importantly, I thought it was a useful contribution to the discussion at hand. I then received a private email in response to that forward from someone who shall remain nameless that expressed disapproval of this forwarding. Not because of the content (which i could understand), but because of the person whose mail I forwarded. I am shocked beyond words that such a thing is possible. Mind you, I'm not supporting at this point that we just allow Sven back in the project; the decision to throw him out was based on solid arguments, and though I am sad that it was necessary, I do support the decision that has been made. However, I would think we would have expelled him because of the problems that resolved around his person, rather than because we didn't like his name, background, or, say, the color of his skin. Problems can be solved. Personalities can change. Throwing a person out of the project because of interpersonal problems is one thing. Forbidding him to ever join it again is quite another. I'm not saying we should just forget the whole thing, pretend it never happened, and move on. However, I do not think that just because the man has a history, we should ignore his useful contributions. Although I disagree with some of the suggestions he made, I do happen to think that this particular contribution was useful, and so I forwarded it. I do disapprove of some of the things he's done, but that doesn't mean I disapprove of the person; I do believe any person deserves a minimum of respect. Apparently some people in this project disagree with me on that, however. I'm not sure what I'll do about that, but suffice to say that I will not be part of a project that considers a person's history to be of more importance than a person's current behaviour. Let's hope this is just the opinion of one person...