Mozilla has been distributing Firefox builds for GNU/Linux systems for a while, and 4.0 should even bring official builds for x86-64 (finally, some would say). The buildbots configuration for these builds uses gcc 4.3.3 to compile the Firefox source code. With the C++ part of gcc, it can sometimes mean side effects when using the C++ STL. Historically, the Mozilla code base hasn t made a great use of the STL, most probably because 10+ years back, portability and/or compiler support wasn t very good. More recently, with the borrowing of code from the Chromium project, this changed. While the borrowed code for out-of-process plugins support didn t have an impact on libstdc++ usage, the recent addition of ANGLE had. This manifests itself in symbols version usage These are the symbol versions required from libstdc++.so.6 on 3.6 (as given by objdump -p):

• CXXABI_1.3
• GLIBCXX_3.4

And on 4.0:

• CXXABI_1.3
• GLIBCXX_3.4
• GLIBCXX_3.4.9

This means Firefox 4.0 builds from Mozilla need the GLIBCXX_3.4.9 symbol version, which was introduced with gcc 4.2. This means Firefox 4.0 builds don t work on systems with a libstdc++ older than that, while 3.6 builds would. It so happens that the system libstdc++ on the buildbots themselves is that old, which is why we set LD_LIBRARY_PATH to the appropriate location during tests. This shouldn t however be a big problem for users. Newer gcc, new problems As part of making Firefox faster, we re planning to switch to gcc 4.5, to benefit from better (as in working) profile guided optimization, and other compiler improvements. We actually attempted to switch to gcc 4.5 twice during the 4.0 development cycle. But various problems made us go back to gcc 4.3.3, the main contender being the use of even newer libstdc++ symbols:

• CXXABI_1.3
• GLIBCXX_3.4
• GLIBCXX_3.4.5
• GLIBCXX_3.4.9
• GLIBCXX_3.4.14

GLIBCXX_3.4.14 was added in gcc 4.5, making the build require a very recent libstdc++ installed on users systems. As this wouldn t work for Mozilla builds, we attempted to build with -static-libstdc++. This options makes the resulting binary effectively contain libstdc++ itself, which means not requiring a system one. This is the usual solution used for builds such as Mozilla s, that require to work properly on very different systems. The downside of -static-libstdc++ is that it makes the libxul.so binary larger (about 1MB larger). It looks like the linker doesn t try to eliminate the code from libstdc++ that isn t actually used. Taras has been fighting to try to get libstdc++ in a shape that would allow the linker to remove that code that is effectively dead weight for Firefox. Why do we need these symbols? The actual number of symbols required with the GLIBCXX_3.4.14 version is actually very low:

• std::_List_node_base::_M_hook(std::_List_node_base*)
• std::_List_node_base::_M_unhook()

With the addition of the following on debug builds only:

• std::string::_S_construct_aux_2(unsigned int, char, std::allocator<char> const&)
• std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_S_construct_aux_2(unsigned int, wchar_t, std::allocator<wchar_t> const&)

The number of symbols required with the GLIBCXX_3.4.9 version is even lower:

• std::istream& std::istream::_M_extract<double>(double&)

It however varies depending on the compiler version. I have seen other builds also require std::ostream& std::ostream::_M_insert(double). All these are actually internal implementation details of the libstdc++. We re never calling these functions directly. I m going to show two small examples triggering some of these requirements (that actually generalize to all of them). The case of templates

#include <iostream>
int main()  
    unsigned int i;
    std::cin >> i;
    return i;
 

This example, when built, requires std::istream& std::istream::_M_extract<double>(double&), but we are effectively calling std::istream& operator>>(unsigned int&). It is defined in /usr/include/c++/4.5/istream as:

template<typename _CharT, typename _Traits>
class basic_istream : virtual public basic_ios<_CharT, _Traits>  
    basic_istream<_CharT, _Traits>& operator>>(unsigned int& __n)  
        return _M_extract(__n);
     
 

And _M_extract is defined in /usr/include/c++/4.5/bits/istream.tcc as:

template<typename _CharT, typename _Traits> template<typename _ValueT>
        basic_istream<_CharT, _Traits>&
        basic_istream<_CharT, _Traits>::_M_extract(_ValueT& __v)  
            (...)
         

And later on in the same file:

extern template istream& istream::_M_extract(unsigned int&);

What this all means is that libstdc++ actually provides an implementation of an instance of the template for the istream (a.k.a. basic_istream<char>) class, with an unsigned int & parameter (and some more implementations). So, when building the example program, gcc decides, instead of instantiating the template, to use the libstdc++ function. This extern definition, however, is guarded by a #if _GLIBCXX_EXTERN_TEMPLATE, so if we build with -D_GLIBCXX_EXTERN_TEMPLATE=0, we actually get gcc to instantiate the template, thus getting rid of the GLIBCXX_3.4.9 dependency. The downside is that this doesn t work so well with bigger code, because other things are hidden behind #if _GLIBCXX_EXTERN_TEMPLATE. There is however another (obvious) way to for the template instantiation: instantiating it. So adding template std::istream& std::istream::_M_extract(unsigned int&); to our code is just enough to get rid of the GLIBCXX_3.4.9 dependency. Other template cases obviously can be worked around the same way. The case of renamed implementations

#include <list>
int main()  
    std::list<int> l;
    l.push_back(42);
    return 0;
 

Here, we get a dependency on std::_List_node_base::_M_hook(std::_List_node_base*) but we are effectively calling std::list<int>::push_back(int &). It is defined in /usr/include/c++/bits/stl_list.h as:

template<typename _Tp, typename _Alloc = std::allocator<_Tp> >
class list : protected _List_base<_Tp, _Alloc>  
    void push_back(const value_type& __x)  
        this->_M_insert(end(), __x);
     
 

_M_insert is defined in the same file:

template<typename ... _Args>
void _M_insert(iterator __position, _Args&&... __args)  
    _List_node<_Tp>* __tmp = _M_create_node(std::forward<_args>(__args)...);
    __tmp->_M_hook(__position._M_node);
 

Finally, _M_hook is defined as follows:

struct _List_node_base  
    void _M_hook(_List_node_base * const __position) throw ();
 

In gcc 4.4, however, push_back has the same definition, and while _M_insert is defined similarly, it calls __tmp->hook instead of __tmp->_M_hook. Interestingly, gcc 4.5 s libstdc++ exports symbols for both std::_List_node_base::_M_hook and std::_List_node_base::hook, and the code for both methods is the same. Considering the above, a work-around for this kind of dependency is to define the newer function in our code, and make it call the old function. In our case here, this would look like:

namespace std  
    struct _List_node_base  
        void hook(_List_node_base * const __position) throw ();
        void _M_hook(_List_node_base * const __position) throw ();
     ;
    void _List_node_base::_M_hook(_List_node_base * const __position) throw ()  
        hook(__position);
     
 

which you need to put in a separate source file, not including <list>. All in all, with a small hack, we are able to build Firefox with gcc 4.5 without requiring libstdc++ 4.5. Now, another reason to switch to gcc 4.5 was to use better optimization flags, but it turns out it makes the binaries 6MB bigger. But that s another story.

I spent most of last week working on Ubuntu bug 693671 ("wubi install will not boot - phase 2 stops with: Try (hd0,0): NTFS5"), which was quite a challenge to debug since it involved digging into parts of the Wubi boot process I'd never really touched before. Since I don't think much of this is very well-documented, I'd like to spend a bit of time explaining what was involved, in the hope that it will help other developers in the future. Wubi is a system for installing Ubuntu into a file in a Windows filesystem, so that it doesn't require separate partitions and can be uninstalled like any other Windows application. The purpose of this is to make it easy for Windows users to try out Ubuntu without the need to worry about repartitioning, before they commit to a full installation. Wubi started out as an external project, and initially patched the installer on the fly to do all the rather unconventional things it needed to do; we integrated it into Ubuntu 8.04 LTS, which involved turning these patches into proper installer facilities that could be accessed using preseeding, so that Wubi only needs to handle the Windows user interface and other Windows-specific tasks. Anyone familiar with a GNU/Linux system's boot process will immediately see that this isn't as simple as it sounds. Of course, ntfs-3g is a pretty solid piece of software so we can handle the Windows filesystem without too much trouble, and loopback mounts are well-understood so we can just have the initramfs loop-mount the root filesystem. Where are you going to get the kernel and initramfs from, though? Well, we used to copy them out to the NTFS filesystem so that GRUB could read them, but this was overly complicated and error-prone. When we switched to GRUB 2, we could instead use its built-in loopback facilities, and we were able to simplify this. So all was more or less well, except for the elephant in the room. How are you going to load GRUB? In a Wubi installation, NTLDR (or BOOTMGR in Windows Vista and newer) still owns the boot process. Ubuntu is added as a boot menu option using BCDEdit. You might then think that you can just have the Windows boot loader chain-load GRUB. Unfortunately, NTLDR only loads 16 sectors - 8192 bytes - from disk. GRUB won't fit in that: the smallest core.img you can generate at the moment is over 18 kilobytes. Thus, you need something that is small enough to be loaded by NTLDR, but that is intelligent enough to understand NTFS to the point where it can find a particular file in the root directory of a filesystem, load boot loader code from it, and jump to that. The answer for this was GRUB4DOS. Most of GRUB4DOS is based on GRUB Legacy, which is not of much interest to us any more, but it includes an assembly-language program called GRLDR that supports doing this very thing for FAT, NTFS, and ext2. In Wubi, we build GRLDR as wubildr.mbr, and build a specially-configured GRUB core image as wubildr. Now, the messages shown in the bug report suggested a failure either within GRLDR or very early in GRUB. The first thing I did was to remember that GRLDR has been integrated into the grub-extras ntldr-img module suitable for use with GRUB 2, so I tried building wubildr.mbr from that; no change, but this gave me a modern baseline to work on. OK; now to try QEMU (you can use tricks like qemu -hda /dev/sda if you're very careful not to do anything that might involve writing to the host filesystem from within the guest, such as recursively booting your host OS ...). No go; it hung somewhere in the middle of NTLDR. Still, I could at least insert debug statements, copy the built wubildr.mbr over to my test machine, and reboot for each test, although it would be slow and tedious. Couldn't I? Well, yes, I mostly could, but that 8192-byte limit came back to bite me, along with an internal 2048-byte limit that GRLDR allocates for its NTFS bootstrap code. There were only a few spare bytes. Something like this would more or less fit, to print a single mark character at various points so that I could see how far it was getting:

	pushal
	xorw	%bx, %bx	/* video page 0 */
	movw	$0x0e4d, %ax	/* print 'M' */
	int	$0x10
	popal

In a few places, if I removed some code I didn't need on my test machine (say, CHS compatibility), I could even fit in cheap and nasty code to print a single register in hex (as long as you didn't mind 'A' to 'F' actually being ':' to '?' in ASCII; and note that this is real-mode code, so the loop counter is %cx not %ecx):

	/* print %edx in dumbed-down hex */
	pushal
	xorw	%bx, %bx
	movb	$0xe, %ah
	movw	$8, %cx
1:
	roll	$4, %edx
	movb	%dl, %al
	andb	$0xf, %al
	int	$0x10
	loop	1b
	popal

After a considerable amount of work tracking down problems by bisection like this, I also observed that GRLDR's NTFS code bears quite a bit of resemblance in its logical flow to GRUB 2's NTFS module, and indeed the same person wrote much of both. Since I knew that the latter worked, I could use it to relieve my brain of trying to understand assembly code logic directly, and could compare the two to look for discrepancies. I did find a few of these, and corrected a simple one. Testing at this point suggested that the boot process was getting as far as GRUB but still wasn't printing anything. I removed some Ubuntu patches which quieten down GRUB's startup: still nothing - so I switched my attentions to grub-core/kern/i386/pc/startup.S, which contains the first code executed from GRUB's core image. Code before the first call to real_to_prot (which switches the processor into protected mode) succeeded, while code after that point failed. Even more mysteriously, code added to real_to_prot before the actual switch to protected mode failed too. Now I was clearly getting somewhere interesting, but what was going on? What I really wanted was to be able to single-step, or at least see what was at the memory location it was supposed to be jumping to. Around this point I was venting on IRC, and somebody asked if it was reproducible in QEMU. Although I'd tried that already, I went back and tried again. Ubuntu's qemu is actually built from qemu-kvm, and if I used qemu -no-kvm then it worked much better. Excellent! Now I could use GDB:

(gdb) target remote   qemu -gdb stdio -no-kvm -hda /dev/sda

This let me run until the point when NTLDR was about to hand over control, then interrupt and set a breakpoint at 0x8200 (the entry point of startup.S). This revealed that the address that should have been real_to_prot was in fact garbage. I set a breakpoint at 0x7c00 (GRLDR's entry point) and stepped all the way through to ensure it was doing the right thing. In the process it was helpful to know that GDB and QEMU don't handle real mode very well between them. Useful tricks here were:

• Use set architecture i8086 before disassembling real-mode code (and set architecture i386 to switch back).
• GDB prints addresses relative to the current segment base, but if you want to enter an address then you need to calculate a linear address yourself. For example, breakpoints must be set at (CS << 4) + IP, rather than just at IP.

Single-stepping showed that GRLDR was loading the entirety of wubildr correctly and jumping to it. The first instruction it jumped to wasn't in startup.S, though, and then I remembered that we prefix the core image with grub-core/boot/i386/pc/lnxboot.S. Stepping through this required a clear head since it copies itself around and changes segment registers a few times. The interesting part was at real_code_2, where it copies a sector of the kernel to the target load address, and then checks a known offset to find out whether the "kernel" is in fact GRUB rather than a Linux kernel. I checked that offset by hand, and there was the smoking gun. GRUB recently acquired Reed-Solomon error correction on its core image, to allow it to recover from other software writing over sectors in the boot track. This moved the magic number lnxboot.S was checking somewhat further into the core image, after the first sector. lnxboot.S couldn't find it because it hadn't copied it yet! A bit of adjustment and all was well again. The lesson for me from all of this has been to try hard to get an interactive debugger working. Really hard. It's worth quite a bit of up-front effort if it saves you from killing neurons stepping through pages of code by hand. I think the real-mode debugging tricks I picked up should be useful for working on GRUB in the future.

Although I promised in my previous entry that I'd made my last mention of the redisfs - replication friendly redis-based filesystem I have to disappoint. Ben Sykes informed me that he'd made a fork of redisfs, called shredisfs. I like forks. Forks are good, and this particular one was very welcome - from the README:

Steve's original managed around 250k per second for writes on my test machine, this version does about 15MB a second writes. ... Read speed on the test machine here now is around 180MB/sec..

Needless to say I "stole" the improvements and rolled them into my original release. Thanks all round to those of you that submitted bug reports, suggestions, and codez. I also had a release out briefly which used zLib to compress the values of keys stored in memory. Unfortunately that lead to a net-slowdown for files which were bigger than a single block - due to the overhead of file-system appends which translated to: "fetch", "compress", "append", and "decompress". Finally on the subject of qpsmtpd, my favourite SMTP-server - the exceptionally talented Matt Sergeant released a proof of concept rewrite in Javascript. Using nodejs this new SMTP server is heavily asynchronous and you can find it online under the name Haraka. Although my javascript-fu is weak I'm very impressed by the codebase out there so far, and I'd expect good things of it in the future. Even if it does use Javascript and not my beloved Perl. Finally I'm a year older. But birthdays aren't important. ObQuote: "Do you wanna know what makes all my candy taste so special? " - Epic Movie.

It seems kind of unfair, given the name, but duplicity really doesn't like to be run in parallel. This means that some naive admin (not me of course, but uh, this guy I know ) who writes a crontab

 @daily  duplicity incr $ARGS $SRC $DEST
 @weekly duplicity full $ARGS $SRC $DEST 

is in for a nasty surprise when both fire at the same time. In particular one of them will terminate with the not very helpful.

 AttributeError: BackupChain instance has no attribute 'archive_dir'

After some preliminary reading of mailing list archives, I decided to delete ~/.cache/duplicity on the client and try again. This was not a good move.

1. It didn't fix the problem
2. Resyncing from the server required decrypting some information, which required access to the gpg private key.

Now for me, one of the main motivations for using duplicity was that I could encrypt to a key without having the private key accessible. Luckily the following crazy hack works.

1. A host where the gpg private key is accessible, delete the ~/.cache/duplicity, and perform some arbitrary duplicity operation. I did duplicity clean $DEST
2. Now rsync the ./duplicity/cache directory to the backup client.

Now at first you will be depressed, because the problem isn't fixed yet. What you need to do is go onto the backup server (in my case Amazon s3) and delete one of the backups (in my case, the incremental one). Of course, if you are the kind of reader who skips to the end, probably just doing this will fix the problem and you can avoid the hijinks. And, uh, some kind of locking would probably be a good plan... For now I just stagger the cron jobs.

I run RAID1 on all of the machines I support. While such hard disk mirroring is not a replacement for having good working backups, it means that a single drive failure is not going to force me to have to spend lots of time rebuilding a machine.

The best possible time to set this up is of course when you first install the operating system. The Debian installer will set everything up for you if you choose that option and Ubuntu has alternate installation CDs which allow you to do the same.

This post documents the steps I followed to retrofit RAID1 into an existing Debian squeeze installation. Getting a mirrored setup after the fact.

OverviewBefore you start, make sure the following packages are installed:

apt-get install mdadm rsync initramfs-tools

Then go through these steps:

1. Partition the new drive.
2. Create new degraded RAID arrays.
3. Install GRUB2 on both drives.
4. Copy existing data onto the new drive.
5. Reboot using the RAIDed drive and test system.
6. Wipe the original drive by adding it to the RAID array.
7. Test booting off of the original drive.
8. Resync drives.
9. Test booting off of the new drive.
10. Reboot with the two drives and resync the array.

(My instructions are mostly based on this old tutorial but also on this more recent one.)

1- Partition the new driveOnce you have connected the new drive (/dev/sdb), boot into your system and use one of cfdisk or fdisk to display the partition information for the existing drive (/dev/sda on my system).

The idea is to create partitions of the same size on the new drive. (If the new drive is bigger, leave the rest of the drive unpartitioned.)

Partition types should all be: fd (or "linux raid autodetect").

2- Create new degraded RAID arraysThe newly partioned drive, consisting of a root and a swap partition, can be added to new RAID1 arrays using mdadm:

mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb2

and formatted like this:

mkswap /dev/md1
mkfs.ext4 /dev/md0

Specify these devices explicitly in /etc/mdadm/mdadm.conf:

DEVICE /dev/sda* /dev/sdb*

and append the RAID arrays to the end of that file:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf
dpkg-reconfigure mdadm

You can check the status of your RAID arrays at any time by running this command:

cat /proc/mdstat

3- Install GRUB2 on both drivesThe best way to ensure that GRUB2, the default bootloader in Debian and Ubuntu, is installed on both drives is to reconfigure its package:

dpkg-reconfigure grub-pc

and select both /dev/sda and /dev/sdb (but not /dev/md0) as installation targets.

This should cause the init ramdisk (/boot/initrd.img-2.6.32-5-amd64) and the grub menu (/boot/grub/grub.cfg) to be rebuilt with RAID support.

4- Copy existing data onto the new driveCopy everything that's on the existing drive onto the new one using rsync:

mkdir /tmp/mntroot
mount /dev/md0 /tmp/mntroot
rsync -auHxv --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* /* /tmp/mntroot/

5- Reboot using the RAIDed drive and test systemBefore rebooting, open /tmp/mntroot/etc/fstab, and change /dev/sda1 and /dev/sda2 to /dev/md0 and /dev/md1 respectively.

It's my birthday, so a week ago I bought myself a present: The office is 4.5 miles from the BART station, so up until now I've been taking a bus. Unfortunately the timings aren't that frequent (every 45-60 minutes). Plus I haven't been doing any exercise since I moved out here. Now it seems to be getting drier I figured it was time to sort out a bike. I wanted something flexible, and have heard nothing but good things from the Brompton owners I know, so that's what I ended up with. I'm very happy with it; despite my lack of recent exercise it's pretty easy to ride. I've actually found it doesn't take any longer than taking the bus, and is sometimes quicker, because I avoid having all of the waiting around for it. I bought it from Warm Planet Bikes, who were incredibly helpful. They had a selection in store that they let me try (and quite a few other models of folding bikes too) and when I made my decision it turned out they could do the customizations I wanted in shop. So rather than me having to order it and wait a month I was able to cycle away that same evening! Full marks to Matt for staying late to sort me out (plus he's a Brompton owner too).

In my previous post, I was pointing out some of my frustrations with Chromium/Chrome. A very kind reader, Timo Juhani Lindfors, gave me answers to many of the problems that I pointed out and got me straight on some issues:

• I can disable full zoom in Iceweasel (i.e., I only want to zoom the text, not the images). According to Chromium's project BTS, it seems that this will be a WONTFIX bug, despite many people asking for it (and it having a lot of duplicates).

Timo told me that there is an extension called "Font Size Increase" (ID: ombpcpigmndepfckcifdblemkabaoihk) that adds a button on the side of the URL bar that you can click to increase the size of the text, without affecting the size of the images. Highly recommended. Not that I need this one as much as "Font Size Increase", but there is also a "Font Size Decrease" (ID: mpajngnpcmjjeoflljdjpnehcfaldcia) that, as the name implies, decreases the size of the text. It is nice to have this one when you clicked on the button to increase the size of the font one or two times too many.

• In Iceweasel, I have a way (without resorting to many extensions or hackish solutions) to use the equivalent of Firefox's "View > Page Style > No Style" (i.e., to ignore any CSS sheet that the document author may have specified on the page).

Timo also told me that there is a very cool extension called "Chrome Page Style" (ID: ddpkgkegfklikkmfmneldonhldahhacb) that allows you to alternate between stylesheets provided by the page (see Joey Hess's ikiwiki's CSS Market for an example of that). It also has the possibility of disabling all the CSS'es that a page may have, which, in many cases around the web actually improves the readability of the pages, for many "fancy" things detract from readability, especially if you have poor eyesight. Although I had said that I didn't want to resort to extensions, this was a simple solution enough that made me consider revisit my requirements. I do, though, wonder why some frequently used extensions are not built into the programs from stock.

• In Iceweasel, I can press Ctrl+Q and make the damn browser quit, instead of me having to use the mouse/trackpad/pointing device. This is probably a lack of my understanding on how to associate key bindings to Chromium, but the problem still stands.

Of course, Timo also set me straight and told me about Ctrl+Shift+Q for that purpose, which I had missed completely.

• Similarly, when there is only one tab open, pressing Ctrl+W in Chromium does not close the tab (say, presenting me a blank tab or the list of most visited sites), nor it closes the application.

As a poor writing of my part, Ctrl+W does close the tabs. But when there is only one tab present, it doesn't close the window. I'm OK with that, since I now know the way to close the window. Thanks again Timo. Oh, and one point that may not be known by many is that using about:flags in Chrome does something analogue to Firefox's about:config (even with the warning). And tweaking with it you can try to see if you can enable hardware acceleration of some parts of your pages, as present in newer releases of Chromium. So, there I am: way more satisfied than with earlier with Chromium, but still having Firefox/Iceweasel as my primary browser. Good to have two usable browsers now.

Continuing my series about my desktop in a shell, in this chapter I will present you my irssi setup.Irssi is the most customized part of my setup, using many scripts from the irssi-scripts Debian package and some from the internet , handling 6 irc networks with a total of 34 channels plus JabberXMPP, ICQ, Twitter and identi.ca. My custom irssi settingsI like my conversations to look like Buddy, you rock instead of Buddy: you rock , so:
/set completion_char ,I also like *long highlighted text*, so:
/set emphasis_multiword onHide mIRC and ANSI colors when turned on. This can be used to eliminate angry fruit salad syndrome in some channels. (Quote from irssi.org/documentation/settings, do I have to say more?):
/set hide_colors onWhen talking in more than one channel, it sometimes comes handy to repeat something from the past said in THAT channel, not searching though all:
/set window_history onTell irssi to shutup when away, noone will hear/see the beep anyways:
/set beep_when_away offI want my screen to inform me about new messages, so I need this:
/set bell_beeps onBeep for query text, notices, dcc, dcc text and hilight:
/set beep_msg_level MSGS NOTICES DCC DCCMSGS HILIGHMy used irssi scriptsactive_notice.pl (from irssi-scripts)Show notices in the active window or the corresponding query instead of the status-window.
No further configuration needed.adv_windowlist.pl (from anti.teamidiot.de)Some really smart windowlist, see my previous post about adv_windowlist.pl, I won t explain its features and settings here again.autorealname.pl (from irssi-scripts)Show peoples real-name when they join (my $nick $name memory is poor :()
No further configuration needed.chansort.pl (from irssi-scripts)Automatically sort my channels alphabetically, grouped by network.
Enable via /set chansort_autosort on<Update>kaimi asked me yesterday, whether chansort could ignore the network and just sort ALL channels in alphabetical order. Well, it could not, until yesterday ;) Fetch either the patch against chansort.pl or my chansort-ng.pl and set chansort_ignore_network to ON. You can also set chansort_ignore_windowtype to mix channels and queries if you like to.</Update>hack-whois-in-current-window.pl (from dgl.cx)Display /whois in the current window or the corresponding query instead of the status-window.
No further configuration needed.keepnick.pl (from irssi-scripts)Restore my nick after it is free again (like after a ping timepout).
Set the nick via /keepnick [-net chatnet] [nick]nickserv.pl (from irssi-scripts)Automatically authorize me on networks with NickServ.
Enable by settings services address, nick and password via:

/nickserv addnet freenode NickServ@services.
/nickserv addnick freenode Zhenech mypass

screen_away.pl (from irssi-scripts)Set away when screen is detached.
Will be enabled automatically, but you maybe want to tweak screen_away_message :)seen.pl (from irssi-scripts)Add a /seen nick command to lookup when and where nick was online the last time.
No further configuration needed.trackbar.pl (from irssi-scripts)Draw a line when you switch irssi windows so you can see which part of the conversation you have already seen.
No further configuration needed.BitlBeeI m using BitlBee as my irssi-to-every-im gateway of choice ;) But while it sounds easy, my setup isn t. I am using BitlBee only to connect to jabber.die-welt.net, my Jabber/XMPP server. There I have a pyICQt instance running, to connect to ICQ via Jabber (yes, BitlBee would support that directly, but that would fail on me when I would like to connect from a different Jabber-client). Twitter and identi.ca are handled via Jabber too: identi.ca does support this by themself, for Twitter I use the tweet.im service.
But besides of my strange setup, BitlBee is easy: apt-get install bitlbee, connect irssi to localhost:6667, you will be force-joined to &bitlbee, where you do:

register somepassword
account add Jabber you@jabberserber.example.com jabberpassword
account on 0

and done, you@jabberserver.example.com is online :)The all awaited screenshot (from my last irssi-post):

And the whole ~/.irssi/config: http://files.die-welt.net/irssi-config.txt, read it carefully while waiting for the next post about newsbeuter :)

In the software community, people hold strong opinions on the subject of listening to users. Some feel that users are an essential source of information for making successful products, as evidenced in the customer development methodology, and seek to involve users deeply in product development. Others believe that users don t know what they want, invoking the quote attributed to Henry Ford, If I d asked customers what they wanted, they would have said a faster horse . Some say that user needs are unknowable except through the lens of a marketplace, where people choose in aggregate which products suit them best, and customers vote with their wallets (anything else is anecdata ). Regular readers will not be surprised that I believe they are all right, but only in certain contexts. The right strategy for involving users in product decisions will depend on factors related to the product itself, the market, and the product development method being used. One of the most important is the life cycle stage of the product: is it a new and rapidly evolving concept, or a mature commodity, or somewhere in between? Simon Wardley explains this well over on his blog, so I won t rehash his points here, but will add a few of my own. If what we re looking for is inspiration for a new product, it s here that Henry Ford was right: users generally won t hand you a complete product vision on a silver platter. They ll frame their input in terms of what they know, and the choices already available to them. However, this doesn t mean that users don t have a role to play in this instance: watching users can be a great source of inspiration. It s the combination of domain knowledge and passionate imagination which triggers the creative spark. Henry Ford applied his engineer s interests to a problem which was evident all around him. If our goal is to test whether a new product is a good fit for its users, there is no substitute for user feedback. We can guess at whether there is a fit, and our intuition may be good, but users are the ultimate judges, and we don t know if we re right or wrong until users evaluate it. So ask them! By engaging in dialogue with individual users, we can learn unexpected things which will help to refine the idea. If we don t find what they think until our new product is released, we risk making something that no one wants. Why wait until it s too late? It can be challenging to extract useful feedback for a product which doesn t yet exist, but this effort is well worth it to avoid wasting much more effort in software engineering. When our objective is to incrementally improve an existing product, individual anecdotes can mislead us. A given change may be an improvement for one user, but a disaster for another. What we want to know is whether the new version is better for the population as a whole, and in this case, we do well to rely on data. There are pitfalls here as well, of course. We need to choose our questions carefully, and realize that users will often resist any change: not because they re stodgy by nature, but because they have to invest effort in adapting to the change. I think of incremental improvement as a joint investment made between product developers and their users, to improve the whole system of people and technology for the better. By choosing the right tool for the job, we can make better decisions, improve faster, and ultimately solve the right problem for our users.

I've been promoted - my job title will now be 'Software Architect'. This seems to mean I'll be writing documents rather than code - any higher-level, and I'd need Mark Shuttleworth's spacesuit. Is this a good thing? Anyway, I celebrated by opening my last bottle of Debian wine. <spam>Unfortunately, I can't take up my new role until we have hired a replacement. UK-based developers who know Perl and ideally PostgreSQL might want to look at the job description, although I'd recommend sending your CV and covering letter to me directly. I don't get a referral bonus, it's just that I don't like what that agency does with your carefully-crafted application.</spam> I'd like to thank everyone who maintains LaTeX for their assistance in this matter. In other news, I'm studying a couple of Open University courses part-time - one on software development (i.e. UML), and one on interaction design. They're going well so far - I've had a couple of tutorials, and the first part of an assignment came back with good marks. I've been reading the Lean Software Development books by Mary and Tom Poppendieck. They seem to put Agile development into a 'respectable' historical context of similar improvements in manufacturing, which might be useful for persuading project managers.

I have been maintaining Debian Bug Tracking System graphs for a few years, now, though not very actively. They initially were available on people.debian.org/~glandium/bts/, but there have been some recent changes. A while ago, I started experimenting with brand new graphs on merkel.debian.org/~glandium/bts/, and when merkel was announced to be dying a few months ago, I got in touch with the QA team to know what to do with them, and it was decided we d put them on qa.debian.org. I unfortunately didn t follow up much on this and only recently actually worked on the migration, which took place 2 weeks ago. The result is that the graphs have officially moved to qa.debian.org/data/bts/graphs/, and links on the Package Tracking System have been updated accordingly. There is now also an additional graph tracking all open bugs in the BTS, across all packages: Today, I added a new feature, allowing to consolidate data for multiple arbitrary packages in a single graph. Such graphs can be generated with the following URL scheme (please don t over-abuse of it):

http://qa.debian.org/data/bts/graphs/multi/name0,name1,name2,etc.png

As an example, here is a graph for all the bugs on the packages I (co-)maintain:

http://qa.debian.org/data/bts/graphs/multi/dehydra,diggler,iceape,iceweasel,libxml2,libxslt,livehttpheaders,mozilla-dom-inspector,nspr,nss,pyxpcom,venkman,vmfs-tools,webkit,xulrunner,zfs-fuse.png

And here are the bugs affecting Mozilla-related packages:

http://qa.debian.org/data/bts/graphs/multi/iceape,icedove,iceowl,iceweasel,nspr,nss,xulrunner.png

I guess the next step is to allow per-maintainer consolidation through URLs such as

http://qa.debian.org/data/bts/graphs/by-maint/address.png

Update: per-maintainer consolidation has been added. (Hidden message here: please help triaging these bugs)

The nominations for Debian Project Leader elections for 2011 are open. As always, I'm tempted to nominate myself, but after thinking about it, I again conclude that I shouldn't. If I did nominate myself, there's a couple of things I'd like to push for.

• re-think the new maintainer process: if you are known among existing members of the project to do good work for the project, and to be trustworthy, then you should be offered membership, and that means voting rights (upload and login rights are less important)
• in general, attract more new contributors to the project
• more rotation for key positions: it is good for the project if positions like DAM, DSA, secretary, etc, get new people into these positions, so that the knowledge and skills get spread out more
• improve how we deal with conflict, especially internally: we've reduced general flame levels a lot over the past fix years or so, but we still have problems dealing with conflict or disagreements
  • this should probably include a more explicit consensus on what is acceptable and what is unacceptable behavior within the project (lists, irc, bts, etc), and how this should be enforced
  • this could also include education and training of how to deal with conflict; perhaps even about dealing with cross-cultural issues
  • could also include neutral "ambassadors", "diplomats", or "hostage negotiators" that step into a heating situation and attempt to defuse it before it blows up
• regular online training sessions, e.g., for packaging in general, packaging of particular kinds of applications, etc
  • Debian Women has run a number of such sessions over IRC, and Ubuntu runs such sessions regularly
  • both popular and useful
  • realtime access to a teacher is qualitatively different from reading books or web pages

The DPL position is inherently a political one, which is why nothing in the list above is technical. On the other hand, I want to concentrate my Debian time on doing stuff that is fun, and right now that means writing code. I want to write code for Debian.

Time for a seventh Debian XSF News issue!

• On the documentation side, I tried to come up with a squeeze backports policy. There are some new upstream releases and xorg-server 1.10 coming up (read below), as well as X stack s migration to testing, to consider before thinking of backporting though.
• Where backports will likely help is for old intel cards (see our FAQ entry). According to Chris Wilson, current kernels/intel drivers are better for 855+, but 830/845 still need a miracle.
• Migrating the X stack to testing needs a bit of coordination, because of the always painful dependency between the nouveau driver and the Linux kernel version. As mentioned in my mail to debian-release@ that would be quite self-contained otherwise. Bottom line: we re waiting for the kernel.
• As announced on debian-x@, the plan is to wait for this migration to happen before uploading new upstream releases of libdrm, the intel driver, and mesa to unstable. Both libdrm and mesa were prepared for Ubuntu, and will be likely uploaded to experimental soon. The intel driver was prepared, but only built against xorg-server 1.10 and uploaded to experimental for now.
• On the xorg-server 1.10 side, the (very late) RandR patches were proven broken, so reverting RandR 1.4 was decided. With some minor tweaks (to the radeon driver), that went quite well, and xorg-server 1.10 got released. Unfortunately, there s still some missing fix for the build system, which explains why I didn t upload a final 1.10 to experimental yet; several patches were proposed already (I checked a bunch of them), but we re still waiting for a final patch.
• As a result of the RandR 1.4 reverts, the video ABI was broken again. Fortunately I gave it a quick try right before 1.10rc3 got released, and noticed the breakage, so the video ABI got bumped. I was therefore very happy with my decision to only upload a bunch of drivers to experimental, built against 1.10rc* (see DXN#6), since I only needed to upload the following video drivers again: ati, dummy, fbdev, intel, mach64, nouveau, r128, vesa. It might make sense to add cirrus to that list next time, for tests under kvm and the like.
• After I poked Peter Hutterer, he kindly and promptly released snapshots for the following input drivers: kbd, mouse, and synaptics, which I uploaded both to unstable (built against 1.9), and to experimental (built against 1.10).
• If you re having troubles with your synaptics device, make sure you have no configuration issues in xorg.conf; even if that used to work somehow, autoconfiguration might fail if there s a misconfigured device (for example, wrong Device option there). You might want to have a look at our how to configure input documentation.
• On the driver side, for 1.9/unstable, Tormod Volden prepared updates for the mga and savage video drivers, which I sponsored.
• Julien Viard de Galbert prepared a new xserver-xorg-video-openchrome snapshot, which I uploaded to unstable.
• He also prepared the addition of a debug package, which I also sponsored after setting the Dm-Upload-Allowed field to yes since he s doing a pretty good job already. It s currently sitting in NEW accordingly.
• A while ago, Thibaut Girka requested the addition of an udeb for the libxtst library, it s now sitting in NEW. Thibaut is working on a virtual on-screen keyboard, initially for use in the Graphical Installer for devices lacking a keyboard (but likely useful in other circumstances). Work is going on, stay tuned.
• Steve Langasek was kind enough to MultiArch-ify both libx11 and xorg; both were uploaded.
• Wayland is getting way more funnier than planned: it is now needed to get mesa built against wayland libraries (to get the wayland EGL platform), and the needed mesa bits are (last I asked) only in master for now.
• On the bugs side, I spent many hours pinging submitters and triaging bugs, mostly on the ati and intel video drivers, as well as on xorg-server again, along with the xorg meta package. (Clicking on a bug graph below leads to a cached full-year view.)

  x-x-video-ati	x-x-video-intel	xorg-server	xorg

• According to the UDD view of XSF bugs, and to Julien s TXBW1, we went down from 880 bugs in november to 540 bugs now.

Christian is a figure of Debian, not only because of the tremendous coordination work that he does within the translation project, but also because he s very involved at the social level. He s probably in the top 5 of the persons who attended most often the Debian conference. Christian is a friend (thanks for hosting me so many times when I come to Paris for Debian related events) and I m glad that he accepted to be interviewed. He likes to speak and that shows in the length of his answers but you ll be traveling the world while reading him. My questions are in bold, the rest is by Christian. Who are you? I am a French citizen (which is easy to guess unless you correct my usual mistakes in what follows). I m immensely proud of being married for nearly 26 years with Elizabeth (who deserves a statue from Debian for being so patient with my passion and my dedication to the project). I m also the proud father of 3 wonderful kids , aged 19 to 23. I work as team manager in the Networks and Computers Division of Onera the French Aerospace lab , a public research institute about Aeronautics, Space and Defense. My team provides computer management services for research divisions of Onera, with a specific focus put on individual computing. I entered the world of free software as one of the very first users of Linux in France. Back in the early 1990 s, I happened (though the BBS users communities) to be a friend of several early adopters of Linux and/or BSD386/FreeBSD/NetBSD in France. More specifically, I discovered Linux thanks with my friend Ren Cougnenc (all my free software talks are dedicated to Ren , who passed away in 1996). You re not a programmer, not even a packager. How did you come to Debian? I m definitely not a programmer and I never studied computing (I graduated in Materials Science and worked in that area for a few years after my PhD). However, my daily work always involved computing (I redesigned the creep testing laboratory and its acquisition system all by myself during my thesis research work). An my hobbies often involved playing with home computers, always trying to learn about something new. So, first learning about a new operating system then trying to figure out how to become involved in its development was quite a logical choice. Debian is my distro of choice since it exists. I used Slackware on work machines for a while, but my home server, kheops, first ran Debian 1.1 when I stopped running a BBS on an MS-DOS machine to host a news server. That was back in October 1996. I then happened to be a user, and more specifically a user of genealogy software, also participating very actively in Usenet from this home computer and server, that was running this Debian thing. So, progressively, I joined mailing lists and, being a passionate person, I tried to figure out how I could bring my own little contribution to all this. This is why I became a packager (yes, I am one!) by taking over the geneweb package, which I was using to publish my genealogy research. I applied as DD in January 2001, then got my account in July 2001. My first upload to the Debian archive occurred on August 22nd 2001: that was of course geneweb, which I still maintain. Quite quickly, I became involved in the work on French localization. I have always been a strong supporter of localized software (I even translated a few BBS software back in the early 90 s) as one of the way to bring the power and richness of free software to more users. Localization work lead me to work on the early version of Debian Installer, during those 2003-2005 years where the development of D-I was an incredibly motivating and challenging task, lead by Joey Hess and his inspiring ideas. From user to contributor to leader, I suddenly discovered, around 2004, that I became the coordinator of D-I i18n (internationalization) without even noticing You re the main translation coordinator in Debian. What plans and goals have you set for Debian Wheezy? As always: paint the world in red. Indeed, this is my goal for years. I would like our favorite distro to be able to be used by anyone in the world, whether she speaks English, Northern Sami, Wolof, Uyghur or Secwepemcts n. As a matter of symbol, I use the installer for this. My stance is that one should be able to even install Debian in one s own language. So, for about 7 years, I use D-I as a way to attract new localization contributors. This progress is represented on this page where the world is gradually painted in red as long as the installer supports more languages release after release. The map above tries to illustrate this by painting in red countries when the most spoken language in the country is supported in Debian Installer. However, that map does not give enough reward to many great efforts made to support very different kind of languages. Not only various national languages, but also very different ones: all regional languages of Spain, many of the most spoken languages in India, minority languages such as Uyghur for which an effort is starting, Northern Sami because it is taught in a few schools in Norway, etc., etc. Still, the map gives a good idea of what I would like to see better supported: languages from Africa, several languages in Central Asia. And, as a very very personal goal, I m eagerly waiting for support of Tibetan in Debian Installer, the same way we support its sister language, Dzongkha from Bhutan. For this to happen, we have to make contribution to localization as easy as possible. The very distributed nature of Debian development makes this a challenge, as material to translate (D-I components, debconf screens, native packages, packages descriptions, website, documentation) is very widely spread. A goal, for years, is to set a centralized place where translators could work easily without even knowing about SVN/GIT/BZR or having to report bugs to send their work. The point, however, would be to have this without making compromises on translation quality. So, with peer review, use of thesaurus and translation memory and all such techniques. Tools for this exist: we, for instance, worked with the developers of Pootle to help making it able to cope with the huge amount of material in Debian (think about packages descriptions translations). However, as of now, the glue between such tools and the raw material (that often lies in packages) didn t come. So, currently, translation work in Debian requires a great knowledge of how things are organized, where is the material, how it can be possible to make contribution reach packages, etc. And, as I m technically unable to fulfill the goal of building the infrastructure, I m fulfilling that role of spreading out the knowledge. This is how I can define my coordinator role. Ubuntu uses a web-based tool to make it easy to contribute translations directly in Launchpad. At some point you asked Canonical to make it free software. Launchpad has been freed in the mean time. Have you (re)considered using it? Why not? After all, it more or less fills in the needs I just described. I still don t really figure out how we could have all Debian material gathered in Rosetta/Launchpad .and also how Debian packagers could easily get localized material back from the framework without changing their development processes. I have always tried to stay neutral wrt Ubuntu. As many people now in Debian, I feel like we have reached a good way to achieve our mutual development. When it comes at localization work, the early days where the everything in Rosetta and translates who wants stanza did a lot of harm to several upstream localization projects is, I think, way over. Many people who currently contribute to D-I localization were indeed sent to me by Ubuntu contributors .and by localizing D-I, apt, debconf, package descriptions, etc., they re doing translation work for Ubuntu as well as for Debian. Let s say I m a Debian user and I want to help translate Debian in my language. I can spend 1 hour per week on this activity. What should I do to start? Several language teams use Debian mailing lists to coordinate their work. If you re lucky enough to be a speaker of one of these languages, try joining debian-l10n-<yourlanguage> and follow what s happening there. Don t try to immediately jump in some translation work. First, participate to peer reviews: comment on others translations. Learn about the team s processes, jargon and habits. Then, progressively, start working on a few translations: you may want to start with translations of debconf templates: they are short, often easy to do. That s perfect if you have few time. If no language team exists for your language, try joining debian-i18n and ask about existing effort for your language. I may be able to point you to individuals working on Debian translations (very often along with other free software translation efforts). If I am not, then you have just been named coordinator for your language I may even ask you if you want to work on translating the Debian Installer. What s the biggest problem of Debian? We have no problems, we only have solutions We are maybe facing a growth problem for a few years. Despite the increased welcoming aspects of our processes (Debian Maintainers), Debian is having hard times in growing. The overall number of active contributors is probably stagnating for quite a while. I m still amazed, however, to see how we can cope with that and still be able to release over the years. So, after all, this is maybe not a problem Many people would point communication problems here. I don t. I think that communication inside the Debian project is working fairly well now. Our famous flame wars do of course still happen from time to time, but what large free software project doesn t have flame wars? In many areas, we indeed improved communication very significantly. I want to take as an example the way the release of squeeze has been managed. I think that the release team did, even more this time, a very significant and visible effort to communicate with the entire project. And the release of squeeze has been a great success in that matter. So, there s nearly nothing that frustrates me in Debian. Even when a random developer breaks my beloved 100% completeness of French translations, I m not frustrated for more than 2 minutes. You re known in the Debian community as the organizer of the Cheese & Wine Party during DebConf. Can you tell us what this is about? This is an interesting story about how things build themselves in Debian. It all started in July 2005, before DebConf 5 in Helsinki. Denis Barbier, Nicolas Fran ois and myself agreed to bring at Debconf a few pieces of French cheese as well as 1 or 2 bottles of French wine and share them with some friends. Thus, we settled an informal meeting in the French room where we invited some fellows: from memory, Benjamin Mako Hill, Hannah Wallach, Matt Zimmermann and Moray Allan. All of us fond of smelly cheese, great wine plus some extra p t home-made by Denis in Toulouse. It finally happened that, by word of mouth, a few dozens of other people slowly joined in that French room and turned the whole thing into an improvized party that more or less lasted for the entire night. The tradition was later firmly settled in 2006, first in Debconf 6 in Mexico where I challenged the French DDs to bring as many great cheese as possible, then during the Debian i18n meeting in Extremadura (Sept 2006) where we reached the highest amount of cheese per participant ever. I think that the Creofonte building in Casar de C ceres hasn t fully recovered from it and is still smelling cheese 5 years after. This party later became a real tradition for DebConf, growing over and over each year. I see it as a wonderful way to illustrate the diversity we have in Debian, as well as the mutual enrichment we always felt during DebConfs. My only regret about it is that it became so big over the years that organizing it is always a challenge and I more and more feel pressure to make it successful. However, over the years, I always found incredible help by DebConf participants (including my own son, last year a moment of sharing which we will both remember for years, i think). And, really, in 2010, standing up on a chair, shouting (because the microphone wasn t working) to thank everybody, was the most emotional moment I had at Debconf 10. Is there someone in Debian that you admire for their contributions? So many people. So, just like it happens in many awards ceremonies, I will be very verbose to thank people, sorry in advance for this. The name that comes first is Joey Hess. Joey is someone who has a unique way to perceive what improvements are good for Debian and a very precise and meticulous way to design these improvements. Think about debconf. It is designed for so long now and still reaching its very specific goal. So well designed that it is the entire basis for Joey s other achievement: designing D-I. Moreover, I not only admire Joey for his technical work, but also for his interaction with others. He is not he loudest person around, he doesn t have to .just giving his point in discussion and, guess what? Most of the time, he s right. Someone I would like to name here, also, is Colin Watson. Colin is also someone I worked with for years (the D-I effect, again ) and, here again, the very clever way he works on technical improvements as well as his very friendly way to interact with others just make it. And, how about you, Rapha l? I m really admirative of the way you work on promoting technical work on Debian. Your natural ability to explain things (as good in English as it is in French) and your motivation to share your knowledge are a great benefit for the project. Not to mention the technical achievements you made with Guillem on dpkg of course! Another person I d like to name here is Steve Langasek. We both maintain samba packages for years and collaboration with him has always been a pleasure. Just like Colin, Steve is IMHO a model to follow when it comes at people who work for Canonical while continuing their involvment in Debian. And, indeed, Steve is so patient with my mistakes and stupid questions in samba packaging that he deserves a statue. We re now reaching the end of the year where Stefano Zacchiroli was the Debian Project Leader. And, no offense intended to people who were DPL before him (all of them being people I consider to be friends of mine), I think he did the best term ever. Zack is wonderful in sharing his enthusiasm about Debian and has a unique way to do it. Up to the very end of his term, he has always been working on various aspects of the project and my only hope is that he ll run again (however, I would very well understand that he wants to go back to his hacking activities!). Hat off, Zack!I again have several other people to name in this Bubulle hall of Fame : Don Armstrong, for his constant work on improving Debian BTS, Margarita Manterola as one of the best successes of Debian Women (and the most geeky honeymoon ever), Denis Barbier and Nicolas Fran ois because i18n need really skilled people, Cyril Brulebois and Julien Cristau who kept X.org packaging alive in lenny and squeeze, Otavio Salvador who never gave up on D-I even when we were so few to care about it. I would like to make a special mention for Frans Pop. His loss in 2010 has been a shock for many of us, and particularly me. Frans and I had a similar history in Debian, both mostly working on so-called non technical duties. Frans has been the best release manager for D-I (no offense intended, at all, to Joey or Otavio .I know that both of them share this feeling with me). His very high involvment in his work and the very meticulous way he was doing it lead to great achievements in the installer. The Installation Guide work was also a model and indeed a great example of non technical work that requires as many skills as more classical technical work. So, and even though he was sometimes so picky and, I have to admit, annoying, that explains why I m still feeling sad and, in some way, guilty about Frans loss. One of my goals for wheezy is indeed to complete some things Frans left unachieved. I just found one in bug #564441: I will make this work reach the archive, benefit our users and I know that Frans would have liked that.

---

Thank you to Christian for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

7 comments Liked this article? Click here. My blog is Flattr-enabled.

I was interested to hear about a company here in the UK called ALLOW Ltd., offering marketing database management under a we ll get you off lists, then pay you to go back on at your pleasure basis. That sounds a fair deal to me, so I decided to sign up for it.

Our technology is built using some of the best and most secure tools in the industry. We have partnered with infrastructure providers who handle some of the most sensitive data in the UK (such as medical and financial records). Both the digital and physical security measures we have implemented are amongst the strongest available anywhere. This includes full encryption of all data at all times, full implementation of secure socket layers, security certificates and physical restriction of access to the data, our servers and our offices. Our systems have been fully penetration tested (that means we ve asked people to try and break in).

(There are other suitable assertions in various places they even have a set of principles about safeguarding data.) Unfortunately, this promise is rather undermined in several ways after noticing the first couple, I did a little digging to see what else was exploitable. Here s the final part of the joining process, where you choose a username and password combination: The text I ve cropped too eagerly says Choosing a secure password is an essential part of protecting your personal information , or thereabouts. I duly chose a complex password that fitted the requirements, and to my surprise it was rejected. I tried another, and it was rejected; then a third and a fourth. By trial and error I worked out what was going on: 1. The password must contain only the listed special characters, not just include one of them. That s a bit of a problem, because even assuming a basic ASCII set, 15 characters are unavailable to users; 80 are left, so that s about a 15% fall in the available combinations*. Not a good start. More concerning is the presence of a security question field. It s used for resetting the password in the event losing it, but this technique for recovery has long been ridiculed the shared secret is often common knowledge amongst friends, and sometimes (as in this case) the available questions are fairly easy for an attacker to find the answer to in public records or solicit from the victim without arousing much suspicion: 2. The security questions available include First pet s name and worse, First school name . It s pretty pointless enforcing stringent password requirements, and then bypassing them with something so susceptible to a dictionary attack. I was pleased to find that failing to log in to an account more than a few times results in a temporary lockout, which should deter casual brute force attacks. But I wanted to know how that security question would be used, so I forgot my password and followed the links to reset it. Here s the form: Actually the first form, not shown here, initially just asks for a username, giving an error message if it isn t registered, and here s another problem: 3. The password reset process confirms the existence, or non-existence, of a given username half the credentials required to log in to any visitor. I d be prepared to take a bet that most users will choose What was the name of your first school? as a security question. The first pet you have is often at such a young age you can t remember it clearly; the name of the street you grew up on might change a couple of times if you moved house. But first school I attended? I ll never forget that, so it makes most sense to use as a backup password . It s also the best one for an attacker to try and find out from public sources. But that aside, as you can see the password is not generated at random and communicated to the real account holder out-of-band, in the manner of many other sites. Instead: 4. A new password is immediately set to a value already known by the attacker. Once inside, an attacker can also change the security question or answer, or both, so you can t even regain your account by telephoning the company unless you can convince them you re genuine, in which case the security question was a total waste of time anyway. I awarded some marks for notifying the user by email that the password has been changed, but immediately docked them again because bingo! You re now a victim of identity theft! Let s assume you ve been locked out, the security question has been changed and you want your account back. ALLOW don t let you telephone them; you either have to dig around and find an address to send an email, which we all know can be intercepted, or (and you re encouraged to) contact them through a form on the site. You ll probably include some personal details, because you want to convince them of your real identity; indeed, two of the options on the form are I ve got a question and Something doesn t work . I sent my findings through this very form, under the latter heading, and to my surprise: 5. Despite promises of full encryption of all data at all times, full implementation of secure socket layers , the contact form is transmitted to ALLOW in the clear, with no protection whatsoever. So now anyone listening in your connection knows all about you too: your ISP, any of the peers along the route, the deep packet inspection advertisers if your ISP is less than reputable, and the neighbour who connects to your wireless and slips you a fiver every month for the privilege. Nice work, privacy specialists. (For the record:

• I have sent these findings to ALLOW and await a response have a response.
• Yes, this scenario is unlikely, but it doesn t fill me with confidence about the rest of their business activities.)

* please feel free to correct my maths. It was never my strongest subject.

---

Privacy specialists should hire security specialists is a post from: jwiltshire.org.uk Flattr

I spend too much time writing Emacs Lisp code these days. Unfortunately, the more I do the more I find new useful tools to improve my work-flow and save time for doing more Lisp. D'oh. I did not work on any big thing these last weeks, so I'm thinking it's a good time to talk about the various code and patches I sent to multiple Emacs packages. el-get el-get, a fabulous tool that installs and handles all the external Emacs packages I use. A friendly war started on the development list about autoloads handling. The discussion was overall pointless, since we had a very hard time to communicate our ideas, and we did not understand each others several times. In the end, el-get now supports autoload correctly and do not load automatically all your packages, improving the startup time, and using the Emacs way to do things. Which is always better, obviously. git-commit-mode

I've started to use git-commit-mode some times ago. I usually use git-commit with the -v option to see what I'm committing. I though it would be useful to color the diff with diff-mode, so I wrote a patch just to do that, which was merged today by Florian. magit

Some weeks ago, I decided to give a try to magit, and loved it. I am not always using it, but for basic operations it is very useful. But I really soon found some things I did not like and therefore send patches to enhance it. First, I've added a patch to honor status.showUntrackedFiles which I use in my home directory. In the mean time, I've also added a patch to allow adding an arbitrary file. Yesterday, I sent another pull request, not closed for now, which adds the possibility to visit files in another window from a diff file, and the support for add-change-log-entry directly from the displayed diff. Useful for these old projects still using ChangeLog files but accessible through git (hi Emacs & Gnus!). Gnus

Nothing remarkable, but I write a couple of fixes and enhancements to the Sieve manage mode, to the Gravatar code and cleaned-up some very very old code. Also added the possibility to set list-identifier as a group parameter. Org-mode

I spent most of my time working on my jd/agenda-format branch, which is soon to be merged. I've also just got developer access to the Org-mode patch work and repository, so I'll be able to break things even more! ;-) ERC

I fixed the bug that annoyed me for a long time. Now erc-track does not reset the last channel status on window visibility changes not made by the user.

Wow, it's the last day of February and this is the first journal post I've made for the whole month. Needless to say, this month isn't working out quite the way that I'd intended at the start of year, although this is due to external forces that I hadn't been able to predict. The work project that I've been working on since last September or so (and in some sense since last June) hit the crunch, and I was pulled off of nearly everything else I was doing to work on that full time. I've been working 45-50 hours a week for the whole month, and probably will for a bit of March as well. The good part is that we're hitting our deadlines. We're going to push back by two weeks due to a design problem we missed until integration testing, but it looks like only two weeks. And given that we're picking up the entire complex account management system, teasing it apart from another major system that it's part of, and bringing it up in my group, along with reimplementing some bits around the edges and working on very old and fragile code, I'm very impressed. Unfortunately, that's also eaten my life. I'm trying to keep up with occasional things here and there, but projects like Debian got put almost entirely on hold. When I'm working this much during the week, I basically collapse on the weekends, rather than work on other projects like I normally do. I've been playing video games or just watching TV, and doing a bit of reading, but not working on free software or writing book reviews. I'm going to get a bit of time off at the end of this project, and hopefully will be able to rebalance my life again. This sort of huge push is quite unusual for work, which I like. I don't like working this way, and I've been occasionally rather unhappy with how little life outside of work I feel like I have (I can't imagine what working 60 hours a week regularly would be like). But there's only about a month or a month and a half left and then we'll be done and can go back to a more relaxed pace. I'm going to see if I can get a review finished tonight just because I hate going through a month without posting a book review. I've read five books (and three magazines) that I've not reviewed yet; sadly, the reviewing brain requires quite a bit more energy than the reading brain. Or perhaps not sadly, since it means I'm still reading. Expect posting to be minimal for the next month or so, and then it will hopefully pick up again.

This past weekend I've had the pleasure to join our friends from Valls, in the Camp de Tarragona, for our annual Cal otada in Picamoixon's countryside. This was the fourth time in a row I attend, and as always, it's been a blast, even if Enric and Clara weren't there, and the Valencian group was reduced to just 5 of us. Unfortunately I had my share of alcohol on Friday evening/night while partying with my workmates so when we got to Tarragona I was basically wasted. This made me not want to take a single sip of any kind of beverage not consisting of a 100% of water during the two days, but that didn't, of course, spoil a single moment of fun. Again, we've had the full traditional pack: prepare, cook and enjoy the delicious cal ots; our share of mayhem just after eating them, during the cal ot war, which this time resulted in a really filthy face and hair; our little walk around the area, including a visit to the chapel of the altar boy ; play in the metres tall mountains of gravel in the quarry and a brief visit to the ruins of an abandoned house, to discover none of its floors have colapsed yet.

A great finish for a great weekend is getting to visit Jordi and Anna, after 3 years of no luck, and finally meeting their lovely 2.5 years old daughter Martina. This weekend just rocks, and I'm already looking forward to next year's!

February has been again a busy month for me. Here s a quick summary of what I did: Multi-Arch work I have spent many days implementing and refining dpkg s Multi-Arch support with Guillem Jover (dpkg co-maintainer) and Steve Langasek (beta-tester of my code ). Early testers can try what s in my latest pu/multiarch/snapshot/* branch in my personal git repository. A Debian DVD shop I m always exploring new options to fund my Debian work (besides direct donations) and this month with the Debian Squeeze release I saw an opportunity in selling Debian DVD. Nobody provides DVD with included firmwares and quite a few people would like to avoid the SpaceFun theme. So I built unofficial Debian DVDs that integrate firmware and that install a system with the old theme (MoreBlue Orbit). Click here to learn more about my unofficial DVDs. On my blog In my People behind Debian series, I interviewed Mike Hommey (Iceweasel maintainer) and Maximiliam Attems (member of the kernel team). I started a Debian Cleanup Tip series and already published 4 installments:

• Debian Cleanup Tip #1: Get rid of useless configuration files
• Debian Cleanup Tip #2: Get rid of obsolete packages
• Debian Cleanup Tip #3: Get rid of third-party packages
• Debian Cleanup Tip #4: Find broken packages and reinstall them

For contributors, I wrote two articles: the first gives a set of (suggested) best practices for sponsoring Debian packages and adapted my article as a patch for the Developers Reference. In the second article, I shared some personal advice for people who are considering participating on Debian mailing list: 7 mistakes to avoid when participating to Debian mailing lists.

Click here to subscribe to my free newsletter and get my monthly analysis on what s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.