Five years ago, I started work on OfflineIMAP. I couldn't find any mail reader that offered good IMAP support and a good feature set. Rather than write my own mail reader, I simply wrote OfflineIMAP and used it with mutt. OfflineIMAP does a bi-directional sync between an IMAP server and local mailboxes. This lets you work offline, and also speeds up reading since each new message doesn't have to be downloaded from the network on the spot.

I kept hoping that OfflineIMAP would become obsolete soon, as mail readers got better. Back in 2004, two years after writing OfflineIMAP, I looked at mail readers. In 2005, after some more frustrations with mail readers, I wrote a comparison. I wound up sticking with mutt and OfflineIMAP each time.

I've gone out looking at mail readers again. Here's what I've found.

KMail
Overall a nice reader, KMail has almost every feature and setting I want. It has "disconnected IMAP" folders, which download every new message in folders to the local disk as part of the routine mail checking. It then caches local changes and syncs them to the remote on the next mail check. This boosts interactive performance and permits offline operation -- very similar to OfflineIMAP. KMail has keyboard shortcuts for most things, and keyboard shortcuts can be added or changed for most other things as well.

KMail also integrates with the KDE calendar and addressbook, which I already use. That's nice, too.

I have two big gripes about it though.

Back in 2004, I noticed that KMail crashes a lot. By 2005, it was worse. Sadly, KMail still has a tendency to crash. I've seen an average of 1-2 crashes per day, due to SIGFPE, SIGSEGV, and I think also SIGILL. This doesn't make me happy at all. Especially to see that it's no better on that in three years. Just don't try emptying trash while your mailbox is being synced, for one thing....

Gripe #2 is that there is absolutely no way to select a different alternative in a multipart message without using the mouse. Simply no way using the keyboard. It's also cumbersome, though possible, to view attachments using the keyboard -- you have to press Enter to open the message in its own window, then tab to the attachment.

KMail also sometimes works a bit sluggishly -- for instance, when you delete a message, you first see strikethrough through it, then it disappears. It doesn't feel very "snappy".

Evolution
Evolution has a decent core. It is easy to get set up and has an extensive set of keyboard shortcuts. It does IMAP downloading and syncing like Kmail, and it does it by default. It doesn't offer all that much flexibility in configuration, but probably enough.

Here's my gripe. There is no way for it to show a total message count next to each folder in the folder list. It will show an unread message count, but not a total message count. You have to click on each folder individually to see a total message count. I can't figure out why this is missing from Evolution. It's one of the main benefits to switching from mutt, and so I didn't bother looking at Evolution any more past there.

Thunderbird / Icedove
By default, it isn't all that capable of a mail reader. There aren't that many configuration options, and the keyboard shortcuts -- while existing for most things -- are cumbersome.

The Nostalgy extension helps with the keyboard shortcuts significantly. You still can't change some of them (Ctrl-L for forward, anyone) -- at least not without an extremely cumbersome process involving editing text files.

Thunderbird can do automatic IMAP downloading and syncing like KMail and Evolution, but for some inexplicible reason, only for your INBOX. In fact, Thunderbird won't even check for mail in folders other than INBOX unless you set an undocumented configuration option. It seems to assume that nobody does server-side mail filtering.

If you want IMAP downloading for offline use or performance, you have to manually invoke a download operation. There is a Sync on Arrival extension, but it isn't compatible with Thunderbird 2.0. From reading comments online, there are a lot of people frustrated about that.

So Thunderbird strikes out as well.

mutt + OfflineIMAP
The good thing about this combination is performace. mutt is extremely fast, and OfflineIMAP works faster than anything else for IMAP downloading. mutt is also far more configurable than anything else.

There are some annoyances about mutt.

#1 on that list is the lack of a folder list. There is just no way to see a list of folders along with new or total message counts. You can press c, Enter to go to the next folder with unread mail, which is something, but not enough. There have been numerous abortive projects over the years to address this, but for whatever reason, mutt itself doesn't have this yet. Probably the most promising current project is this one.

#2 is HTML mail. I don't mind the lack of default support for HTML mail. That's to be expected. There are some things that do bug me involving viewing HTML mail. First off, sometimes people attach graphics to messages that also have an HTML component. Viewing these graphics doesn't represent a security risk, but mutt doesn't make them available to a browser for viewing -- you have to manually save them if you want to view them. Also, you normally don't want to load graphics from the Internet for HTML mail. The only way to accomplish that with mutt is to set your browser to lynx or something; just using Firefox to view a HTML component will load all of that.

#3 is handling of embedded URLs. xterms can pass mouse clicks, and it would be nice if mutt made URLs clickable like other mail readers do.

#4 is the IMAP support. No support for caching, fragile, etc. That's why I use OfflineIMAP. That works, but it's a hassle.

#5 is printing. Printouts from mutt just spew the text of the message at the printer. No page numbers, formatting, nothing. muttprint makes that situation a bit better, but the integration is flaky and weird.

Conclusions
I'm not sure what I'll do. None of these are really where I want them to be, though mutt and KMail are probably the closest.

As a result of commenting on some problems in CUBA's Jam Busting June site I was made aware of the Working Draft of WCAG 2.0 or the Not-Working Draft, as I think it could be called. If you want to comment on it yourself, you had until the end of 27 June 2007 (US EDT, I assume). Put simply, it's too confusing for both web developers and site managers. Even the Quick Reference is rather long for practical use. It's very indirect - it's not even clear whether xhtml over HTTP is preferred. I submitted a comment about that - also rather disappointing to find that W3C appear to fail to use stripslashes() when needed. I submitted another comment about the backwards step of allowing Javascript-required sites to be called accessible. Javascript is dangerous for accessibility and bad for power consumption. What the blue blazes is an AJAX-only webmail doing as one of the accessibility examples? I see that notable sites have already criticised WCAG 2.0. While I don't necessarily agree with everything they say, this is a very disappointing situation. Will WCAG 2 be rescued? Are accessible-friendly webmasters screwed if WCAG 1.0 is replaced by 2.0 in our national standards? It lets all the Flash-only cowboys claim WCAG-conformance as long as they note their site relies upon Flash. (Question: is a site still WCAG-2.0-accessible if it requires Flash to view the note that says it requires Flash?) Should we start looking towards MACCAWS, WHATWG, expanding IETF to cover the web, or starting another democratic/meritocratic standards body?

Parents and educators should be aware about what kind of games their children play. In the current state, this means that the parents must be gamers themselves to know about that, because there’s no way of knowing whether a game includes sex, violence, sexism, etc. until you play it. The idea is to provide a way of classifying games so that parents can know in advance whether the game involves certain culture-dependent controversial stuff, and being able to use that when considering what games they want their children to play, or what games they want to play themselves. Not that I’m the proper one to make the classification myself, but it should be done by teachers and educators, who I think are the best suited for this kind of task. In any case I’m willing to lead and coordinate the project, with the support of just some Spanish teachers for the moment. I hope that in the future teachers of other cultures will like to get involved too and the system can evolve to a multicultural one. The result should be as culturally independent as possible, but that would only be possible if members of different cultures help. There’s no way to provide a totally objective classification of moral stuff, so instead of trying to make the classification as aseptic as possible, it might make more sense to make it the other way round: trying to have all the different possible points of view, even when some of them might not even fit my own beliefs. Links:

• TIGRS , The Independent Game Rating System (Wikipedia)
• PEGI, Pan European Game Information (Wikipedia)
• ESRB, Entertainment Software Rating Board (Wikipedia)
• DebTags, Debian Package Tags (Categories for Debian packages)
• Previous discussion about this in the Debian Games Team Mailing List: 1, 2. 3

Enrique Dans writes about the "The Coming Battle: Apple's iPhone vs. Corporate IT Departments". He makes an interesting point on how this new gadget can be a point of friction between managers of IT departments and their employees using this kind of gadgets, usually corporate executives. Syncing the database with RIM devices, or redirecting email accounts (iPhone is not capable of using corporate Blackberry or MS Exchange servers), can be a problem in corporate world. If you read those articles, they blame iPhone for not supporting all these enterprise-class software. Basically, IT departments are blaming iPhone for not supporting their competition systems, and they are not requiring all them (and themselves) to be using standards, which were just designed to avoid this kind of problems. They are afraid. No more comments.

Enrique Dans writes about the "The Coming Battle: Apple's iPhone vs. Corporate IT Departments". He makes an interesting point on how this new gadget can be a point of friction between managers of IT departments and their employees using this kind of gadgets, usually corporate executives. Syncing the database with RIM devices, or redirecting email accounts (iPhone is not capable of using corporate Blackberry or MS Exchange servers), can be a problem in corporate world. If you read those articles, they blame iPhone for not supporting all these enterprise-class software. Basically, IT departments are blaming iPhone for not supporting their competition systems, and they are not requiring all them (and themselves) to be using standards, which were just designed to avoid this kind of problems. They are afraid. No more comments.

Last December, I went to the Delta de l'Ebre with a good number of friends to spend one of December's three-day weekends. Les Terres de l'Ebre are the southest region of Catalunya, right next to the border with the Valencian Country. The Ebre river forms a big delta as it meets the Mediterranean, and the great amount of wildlife, specially avian, in the Natural Park makes it a fantastic place to visit. We would sleep, more or less, in Cinta's appartment in L'Ampolla. The first night we managed to fit 17 persons for dinner, and about 14 stayed to sleep in any possible corner of the flat. As soon as the Valencian group arrived, we tried to meet the people coming from the North, but apparently they got lost and we stayed on a sightseeing spot admiring how easily the weather changes here. In an hour or so, we went from a sunny day to a quick but heavy shower and back to sunny. When we finally found each other, we decided it was a bit too late to do anything too special, given the short days in December, so we walked South along the beach, on our way to the Salines, the salt processing facility, in the Punta de la Banya.

Agua para todos... During the walk, we witnessed what the people from the Delta have been warning about for decades: the Delta is disappearing. The progressive overuse of the river's flow, excessive irrigation, damm construction and degradation is killing this unique place. The waters are slowly claiming what once was theirs. An electricity line, at the beginning of our trip being at the end of the beach starts approaching the sea, or viceversa, as you move southward, to end up many metres inside the water. The old people in the Delta still remember the 200m tall lighthouse in the Illa de Buda, which the sea managed to devour.

Sunset at the salines del Delta When we reached the Punta we could watch a beautiful sunset over the mountains of Els Ports, and when it got dark and cold we headed to Tortosa, the area's capital, which I had recently visited for the first time for one of Softcatal 's meetings. There, we had dinner at a bar and met Jordi and Anna for a while, before going back to Ampolla, going to bed relatively early due to everyone being very tired.

On Saturday, we decided to have a look at the Delta... from above. So we drove to the top of the Cim del Caro, where we could admire a great view over the flat farmlands of the Delta, and then found a nice trail which took us to a nice forest, where we had lunch. After a visit to Alfara de Carles, we went back to the house, had a great dinner and played fun games at the table. Unfortunately, Sunday morning was dedicated to cleaning up the mess in the appartment and packing, as part of the group, the Jordis from Alcoi and the twins, had to leave early. The remaining four Valencians decided to go to Deltebre to have a local paella. It was quite good, although the bar tender laughed at our request to eat directly from the paella, rejecting dishes. Small cultural clash. ;)

I remember downloading and viewing the pictures of the trip a few days after getting back in Val ncia, and then losing track of them. Last week, while testing an EPIA board with some random hard drive, I suddenly realised there was a big chance the pictures would be on that drive, and after some search, I found them; I had been close to mkfs the disk a few minutes before. I hadn't stopped blaming myself for misplacing these pictures, as they remind me of this wonderful weekend. Luckily, they are back! Going back to the state of the Delta, experts say it will be mostly gone in 200 years. All of the Terres de l'Ebre have fought against the previous government's "PHN" plan to divert part of the river's flow to the North and South, to supply Barcelona, and specially the South of Val ncia, where the developing of the coastal areas is out of control. I fear that whenever PP regains power in Madrid, they will retake this idea. There is a lot of money involved in building a 700 km long scar in our land. The Delta de l'Ebre is a fantastic place which doesn't deserve a terrible fate like this, only for the benefit of the usual suspects. As the good people of this land would say...

Lo riu s vida. No al transvasament!

Last December, I went to the Delta de l'Ebre with a good number of friends to spend one of December's three-day weekends. Les Terres de l'Ebre are the southest region of Catalunya, right next to the border with the Valencian Country. The Ebre river forms a big delta as it meets the Mediterranean, and the great amount of wildlife, specially avian, in the Natural Park makes it a fantastic place to visit. We would sleep, more or less, in Cinta's appartment in L'Ampolla. The first night we managed to fit 17 persons for dinner, and about 14 stayed to sleep in any possible corner of the flat. As soon as the Valencian group arrived, we tried to meet the people coming from the North, but apparently they got lost and we stayed on a sightseeing spot admiring how easily the weather changes here. In an hour or so, we went from a sunny day to a quick but heavy shower and back to sunny. When we finally found each other, we decided it was a bit too late to do anything too special, given the short days in December, so we walked South along the beach, on our way to the Salines, the salt processing facility, in the Punta de la Banya.

Agua para todos... During the walk, we witnessed what the people from the Delta have been warning about for decades: the Delta is disappearing. The progressive overuse of the river's flow, excessive irrigation, damm construction and degradation is killing this unique place. The waters are slowly claiming what once was theirs. An electricity line, at the beginning of our trip being at the end of the beach starts approaching the sea, or viceversa, as you move southward, to end up many metres inside the water. The old people in the Delta still remember the 200m tall lighthouse in the Illa de Buda, which the sea managed to devour.

Sunset at the salines del Delta When we reached the Punta we could watch a beautiful sunset over the mountains of Els Ports, and when it got dark and cold we headed to Tortosa, the area's capital, which I had recently visited for the first time for one of Softcatal 's meetings. There, we had dinner at a bar and met Jordi and Anna for a while, before going back to Ampolla, going to bed relatively early due to everyone being very tired.

On Saturday, we decided to have a look at the Delta... from above. So we drove to the top of the Cim del Caro, where we could admire a great view over the flat farmlands of the Delta, and then found a nice trail which took us to a nice forest, where we had lunch. After a visit to Alfara de Carles, we went back to the house, had a great dinner and played fun games at the table. Unfortunately, Sunday morning was dedicated to cleaning up the mess in the appartment and packing, as part of the group, the Jordis from Alcoi and the twins, had to leave early. The remaining four Valencians decided to go to Deltebre to have a local paella. It was quite good, although the bar tender laughed at our request to eat directly from the paella, rejecting dishes. Small cultural clash. ;)

I remember downloading and viewing the pictures of the trip a few days after getting back in Val ncia, and then losing track of them. Last week, while testing an EPIA board with some random hard drive, I suddenly realised there was a big chance the pictures would be on that drive, and after some search, I found them; I had been close to mkfs the disk a few minutes before. I hadn't stopped blaming myself for misplacing these pictures, as they remind me of this wonderful weekend. Luckily, they are back! Going back to the state of the Delta, experts say it will be mostly gone in 200 years. All of the Terres de l'Ebre have fought against the previous government's "PHN" plan to divert part of the river's flow to the North and South, to supply Barcelona, and specially the South of Val ncia, where the developing of the coastal areas is out of control. I fear that whenever PP regains power in Madrid, they will retake this idea. There is a lot of money involved in building a 700 km long scar in our land. The Delta de l'Ebre is a fantastic place which doesn't deserve a terrible fate like this, only for the benefit of the usual suspects. As the good people of this land would say...

Lo riu s vida. No al transvasament!

I finally decided myself to switch my laptop to using a LVM over LUKS setup instead of LVM alone. The problem it involves is that there is currently no way that I know to do this in-place. I do have an external disk on which I could pvmove everything and back, but that would not be very challenging ;). So I started wondering how doable that would be. LUKS has a 1032 sectors (a sector being 512 bytes) overhead in which it store the LUKS header. Following the header, encrypted data just takes as much space as its decrypted counterpart, linearly. So all it would take to do in-place LUKS conversion would be to move data by 1032 sectors, and encrypt it. Which means the main problem is to have 1032 sectors free after the current volume. Fortunately, LVM doesn’t take up all the space on the partition a physical volume is setup, except if you’re unlucky or created partitions specifically so that it uses all the space. Because LVM physical volumes use the space by big chunks, 4MB by default), if the size of the partition is not 4MB aligned (modulo the LVM header), you have some free space between the physical volume end and the partition end. We thus need this free space to be greater than 1032 sectors. On my laptop, there is more than 1.8MB free there, so there would be no problem.
Note the debian-installer seems to do more than a simple cryptsetup luksFormat, as LUKS volumes it creates have an overhead of 2056 sectors instead of the 1032 sectors I got creating a volume by hand. Next problem is how to move the data and encrypt it in-place. Bonus points if that can be done without unmounting the filesystems. My first guess was to play around with the device mapper and a specially crafted FUSE filesystem to create a fake LUKS volume that could be half encrypted and half not, and would evolve, with time, into a fully encrypted LUKS volume. Since I didn’t know much about the device mapper, I looked at it a bit, and discovered a FUSE intermediate would be unnecessary: not only can you remap on-the-fly (which I already guessed was somehow possible, considering how LVM can do it, though I didn’t know much about the interaction between LVM and the d-m), but you can also mix dm-crypt and linear chunks in the same d-m table. It also appears LVM over LUKS has an overhead that is actually not necessary. How sad both use the device mapper but yet can’t do an efficient setup with it. When you setup LVM over LUKS, you open a LUKS volume, which will create a dm-crypt device mapper, and use this device mapper as the physical volume for LVM. Logical volumes created with LVM are actually device mappers using the physical volume as destination device. When you access a block on an LVM logical volume, your access is first mapped through the LV device mapper and then mapped again through the dm-crypt mapper. So it goes through 2 device mappers, but it could do with only one: the linear mapping used in most of the cases with LVM could be replaced by crypt mappings, that are just as linear, but also handle the encryption/decryption part. Playing around under qemu with a freshly installed etch system, I was able to implement this and came up with the following script. It should work with most setups, but use at your own risk, it comes with no warranty.


MAJOR=$(awk '/device-mapper/ print $1;exit ' /proc/devices)
dmsetup ls --target linear while read map extra; do
  TABLE=$(dmsetup table $map while IFS=" :" read start length linear major minor offset; do
    map2=
    if [ "$major" -eq "$MAJOR" ]; then
      IFS=":" read map2 extra <<EOF
$(dmsetup info -c --noheadings -j $major -m $minor)
EOF
    fi
    if [ -z "$map2" ] [ $(dmsetup table "$map2" wc -l) -gt 1 ] [ $(dmsetup table --target crypt "$map2" wc -l) -eq 0 ]; then
      echo $start $length $linear $major:$minor $offset
    else
      IFS=" " read cstart clength crypt format key IVoff dev coffset <<EOF
$(dmsetup table "$map2")
EOF
      echo $start $length crypt $format $key $(expr $IVoff + $offset) $dev $(expr $coffset + $offset)
    fi
  done)
  if (dmsetup table $map; echo "$TABLE") sort uniq -c grep -v "^ *2 " > /dev/null 2>&1;
  then
    echo Diverting $map
    dmsetup reload $map <<EOF
$TABLE
EOF
    dmsetup resume $map
  fi
done


[ Update: There was a dmsetup resume missing in the script ] After running this script, you should be able to run cryptsetup luksClose on your LUKS volume. Once you’ve done it, pvdisplay and other LVM tools will show you nothing, because the physical volume will just have disappeared… but everything will still be in place. Note that if you leave the LUKs volume open, you’d better be careful NOT to do operations with LVM (such as pvmove) because it could break everything. Note that on sid, you would need to add a –showkey option into the $(dmsetup table "$map2") line. This option is necessary for dmsetup to display the encryption keys. I wonder if LVM upstream would consider supporting LUKS volumes directly… Anyways, back to the in-place conversion to LVM over LUKs. The idea would be to:

• Create a file the size of the LUKS headers + one block.
• Create a loopback device for this file.
• luksFormat the loopback device.
• luksOpen the loopback device, so that dmsetup can give us the key to use. I wonder if there are ways to get the key not involving the creation of the device mapper.
• luksClose it, because we actually don’t need it
• Create a device mapper that linearly maps the whole LVM physical volume (i.e the real partition, but without the empty tail).
• Remap the LVM device mappers so that they use our own device mapper instead of the partition as physical volume.
• Here is the trickiest part, and I still have to take a look at how pvmove does to know exactly how this can be done (I guess it will require an additional device mapper): move data chunk by chunk (offsetting with the size of the LUKS headers), starting from the end of the LVM physical volume, and encrypt at the same time. Once a chunk is moved, adjust our device mapper so that it maps to the linear unmoved data followed by the moved encrypted data. As I said earlier, this is a setup that works.
• Once everything is moved, we are in a situation where our device mapper is a single crypt mapping, in the same shape as a luksOpen would have made it. We can now copy the LUKS headers at the beginning of the partition, and add the appropriate configuration to /etc/crypttab.

If I got it all right, both LUKS and LVM should be believing they have done this setup themselves, so unmounting, lvchange, etc. should just be fine. It might be necessary to regenerate the initramfs, though. I’ll tell you when I fuck up my hard disk try it. To be continued…

The Gnu Pricacy Guard seems to have the default option of setting an expiry date when the signed key has such a date. My gpg key used to have such an expiry day when I went to the first keysigning parties (LinuxTag, DebConf 3). I have removed the expiry date later, but still a lot of signatures are now no longer valid: While I have signed 310 keys, my key is currently (pre-DebConf 7) only has 170 valid signatures.Tonight I wanted to find out what keys are these exactly, and after quite some trial and error I found a way to get that data, using only real old unix tools such as awk, sort, join:

gpg  --list-options show-sig-expire --fixed-list-mode --with-colons --list-sigs nomeata > my-sigs-col
cat my-sigs-col  awk -F: '$1 == "sig" && $7 != ""   print  '   sort -k 5,5 -t : -u > expired
cat my-sigs-col  awk -F: '$1 == "sig" && $7 == ""   print  '   sort -k 5,5 -t : -u > non-expired
join -v 1 -t : -j 5  expired non-expired > only-expired

Maybe I will use that list some time after DebConf7 to mail all these people, asking them to remove or update the expiry date from their signature.

The Gnu Pricacy Guard seems to have the default option of setting an expiry date when the signed key has such a date. My gpg key used to have such an expiry day when I went to the first keysigning parties (LinuxTag, DebConf 3). I have removed the expiry date later, but still a lot of signatures are now no longer valid: While I have signed 310 keys, my key is currently (pre-DebConf 7) only has 170 valid signatures.Tonight I wanted to find out what keys are these exactly, and after quite some trial and error I found a way to get that data, using only real old unix tools such as awk, sort, join:

gpg  --list-options show-sig-expire --fixed-list-mode --with-colons --list-sigs nomeata > my-sigs-col
cat my-sigs-col  awk -F: '$1 == "sig" && $7 != ""   print  '   sort -k 5,5 -t : -u > expired
cat my-sigs-col  awk -F: '$1 == "sig" && $7 == ""   print  '   sort -k 5,5 -t : -u > non-expired
join -v 1 -t : -j 5  expired non-expired > only-expired

Maybe I will use that list some time after DebConf7 to mail all these people, asking them to remove or update the expiry date from their signature.

Okay, so with the “Bits from the DPL” talk that Sam and I shared now out of the way, that hopefully means finishing off the whole dunc-tank thing won’t imply any more conflicts of interest, potential or otherwise. And since we’ve now had the post-etch release team report, there shouldn’t be much to wait around for. I’d been kinda hoping to be able to come to debconf with a pre-prepared report of similar quality to the debconf6’s final report (or debconf5’s for that matter); but while I was going back through the discussion threads, it just got harder and harder to find a way of writing the report that didn’t seem like it’d just inspire the same arguments again. I think that’s avoidable – I know some of the people who were against the way dunc-tank ended up have indicated there are other approaches they might support, and it ought to be possible to summarise the dunc-tank experience in a way that lets us have a better idea what to do in the future based on that experience, than have the same disagreements we’ve already had. Anyway, my best guess is to try crafting the report via blogging or put it on a wiki or something. Anyway, there are three things that I thought we could usefully have in a report that wouldn’t devolve into the same old arguments (after all, if we at least have new arguments, that’s some sort of progress!). First of all is trying to have a real understanding of what happened with etch – getting a release out is complicated with lots of potential blockers beyond just “release management” work: d-i, kernel, toolchain, CDs, builds, bug tracking, whatever. Presumably dunc-tank only ever had a chance of removing release management as a blocker, and since we can’t redo the etch release while changing parameters like dunc-tank or Vancouver, at least having as clear as possible understanding of which blockers happened when, and what other influences might have been important the release process seems like a good way of understanding the influences that were and weren’t important, so we can have a better idea what to do in future. I’m figuring a reasonable structure might be something like:

• Release blockers between July 2006 and April 2007
• Analsyis of bug discovery/resoution rates between July 2006 and April 2007
• Comparison of major statistics for etch with previous releases (freeze length, etc)
• Comparison of etch with Ubuntu’s edgy release (October 2006, roughly the time dunc-tank started), and feisty release (April 2007, roughly the same time as etch’s release)
• Summary of other release timliness projects: Vancouver proposal, release assistants, BTS version tracking and changes to RC bug monitoring, transition monitoring through the entire release cycle, binNMU changes, improvements in experimental support, BSP marathons
• Summary of release quality projects: DFSG-free content, LSB support, security support for testing, QA meetings, piuparts runs, whole archive rebuild testing

The theory is that the above should provide the information to make a good judgement of what (if any) effect dunc-tank (or the other projects) had on the release, without implying a preconceived conclusion of whether dunc-tank’s “good” or “bad” or whatever. As well as looking at the release stuff in some depth, it seems to me to be pretty sensible to do a report on how dunc-tank worked financially. Anyone doing any similar sort of “let’s raise funds, then spend them on free software” is going to have similar issues in getting publicity, collecting funds, and actually spending them, no matter how they end up deciding on the harder questions of deciding who or what to fund. There’s not a lot to say there, and I don’t think there’s much that’s controversial there either, but there are a few problems and approaches to solving them that are worth reporting on. I think that’s about as far as you can get trying to cleverly avoid the controversy though. And presuming we rule out the idea of choosing a position on some of the controversial questions and advocating for that (“dunc-tank was good, paying release managers was awesome, we should do it again for everyone!”), the best idea I’ve had so far is to just try summarising some of the controversial questions that remain open, which Debian, or dunc-tank, or whoever can then address in whatever way we like – by GRs, or further debate, or more experiments, or leaving it to other people to do for us, or whatever. So having the questions and some of the arguments for various answers and summarising what happened with dunc-tank seems like a plausible way to go. The questions I’ve got:

• Is paid work compatible with volunteer work?
  • How the release managers and assistants reacted to doing volunteer work while someone was paid to do similar work
  • Other developers – some donated; others resigned or reduced work
  • Dunc-bank and QA initiatives
• How much distance should there be between Debian and whoever’s funding development?
  • Involvement of DPL and high-profile developers
  • Involvement of SPI
  • Statements from press reports about the relationship
  • Votes to support/endorse/recommend against
  • Donors being confused as to where to donate funds
• How should work or developers to be funded be chosen?
  • Potential for favouritism
  • Potential for misalignment with Debian’s goals
  • Choosing based on least controversy or most benefit
• How should paid “volunteer” work be priced?
  • Covering living expenses?
  • Comparable rate to similar proprietary/commercial development?
  • “Market” rates?
  • Full-time versus part-time work
  • Contracts versus prizes or bounties or rewards

No promises as to whether they’re the best questions, the best sub-points, or that we’re going to be able to come to an answer on any of them. :) Anyway, that’s my theory so far, no I need to grab some lunch before the next lot of talks – feel free to mail me, or talk to me in person, or blog responses or whatever…

Transcript of the call for participation on the Gathering of the Indigenous People of the Americas. If you don’t mind, feel free to skip it. Considering that 515 years after the first invasion of our ancestral territories, the war of conquest, the plunder, and capitalist exploitation have not altered their course, but rather have become a new war of extermination, a war designed for the destruction and utter plunder of all the original peoples of the Americas; Considering that the long history of wars of independence and numerous revolutions that have taken place on our continent have not yet changed the condition of colonized people, nor have they allowed for a full recognition of our rights in the nation states that formed in the last two centuries; Considering that despite a long history of domination, our resistance has sustained itself and that our people continue to live and struggle, and that this struggle has been encouraged by the uprising of the Zapatista National Liberation Front (EZLN) in the first days of 1994; Considering that the strategies of the national governments of the Americas have always sought the division of our people through the establishment of borders, reservations, and legislation designed to fragment us and to neutralize our efforts of autonomy; Considering that the strengthening and the unity of the struggles of liberation of the indigenous people of the Americas will be possible as our peoples meet and come to know each other; Considering that for our people it has proven necessary that their struggles be made known world wide in order to be supported by all honest movements that struggle for a society effectively just, free, and democratic; All indigenous communities, tribes, and nations of the Americas are called to

THE GATHERING OF THE INDIGENOUS PEOPLE OF THE AMERICAS Which will take place the 11st, 12nd, 13rd, and 14th days of October, 2007 in the community of V cam, Yaqui Territory, Municipality of Guaymas, State of Sonora, Mexico, to address the following

TOPICS 1 The war of capitalist conquest in the indigenous communities in the Americas.
2 The resistance of the indigenous peoples of the Americas and our defense of
mother earth, our territories, and our cultures.
3 Why do we, the indigenous peoples of the Americas, struggle?

ORGANIZATIONAL PRINCIPLES 1st Principal. Delegates, observers, and communications media workers attending the Gathering will abide by the regulations of the Yaqui Tribe devised by their Traditional Authorities in order to govern their presence while in attendance.
2nd Principal. All participating indigenous communities, tribes, and nations are invited to demonstrate their clothing, songs, dances, and other traditional forms of expression.
3rd Principal. Authorities and Representatives of indigenous communities, tribes, and nations of the Americas are invited to participate as delegates with voice and vote.
4th Principal. Individuals and members of social movements and civil organizations in solidarity with the struggles of indigenous peoples of the Americas invited by the organizing commission will be able to participate as observers.
5th Principal. National and international communications media workers that are accredited by the organizing commission will be able to cover the event.
6th Principal. All agreements adopted at the Gathering will be made by consensus of the participants and not through the method of voting.
7th Principal. Organization of the Gathering, the national and international distribution of its publicity, as well as accreditation of participating delegates, invitees, and any unforeseen issues will be resolved only by the organizing commission in conformity with the Traditional Authorities of the Community of V cam, Yaqui Tribe, The National Indigenous Congress (CNI), and the Sixth Commission of the Zapatista National Liberation Front. Information: 1. Office of the Sixth Commission of the EZLN, located on Zapotecos St. No 7 Col.Obrera, C.P. 06800 Mexico, D.F. Telephone: (01 or 011) 55 67 61 42 36 Monday thru Friday, 10 AM 8 PM., Saturday, 10 AM 6 PM.
webpage: www.encuentroindignea.org e-mail: informes@encuentroindigena.org
2. Official Precinct Office of the Traditional Guard of the Yaqui Tribe, Vicam Station ,Colonia Yaqui, Municipio de Guaymas, Sonora, Mexico. Telephone: (045 or 001) 64 49 98 94 08.
3. Office of the Organization of Indigenous and Campesino Communities, de Tuxpan, Jalisco, in Nicolas Bravo, No. 65 , Tuxpan, Jalisco,. Telephone: (01 or 011) 371 41 764 15 from Monday Saturday, 10 AM 8 PM. e-mail: comunidad_tuxpan@hotmail.com Written - April 22nd, 2007 in Tohono O Odham Territory of the Tohono O Odham Nation, Sonora, Mexico. Attentively,

Wa-a itom yo Lutu-ria yoribetchibo, intoka a Wiutinepo Amani
(Por el respeto a nuestros usos y costumbres, nuestra identidad y cultura y nuestra verdad que permanezcan en el tiempo y no se olviden)

Never Again, a Mexico Without Us ~ Liberty, Justice, and Democracy

 

Nde Cultural Historical Organization
(Organizaci n Cultural e Hist rica de la Naci n Nde/Apache)

 

Tierra y Libertad / Chicana Indigenous Organization, Tucson, Arizona
(Tierra y Libertad / Organizaci n Xicana Ind gena, Tucson, Arizona

 

Native and Immigrant Indigenous Development Organization
(Desarrollo de Los Pueblos Indios Inmigrantes y Nativos, A.C.)

 

Michelle Cook (Dene Nation)
(Nacion Dene/Navajo)

 

Tohono O Odham Nation Mexico United States
(Nacion Tohono O Odham M xico - Los Estados Unidos de America)

 

Traditonal Authorities of Vicam Community, Yaqui Tribe
(Autoridades Tradicionales del Pueblo V cam, Tribu Yaqui)

 

National indigenous Congress (Congreso Nacional Ind gena)

 

Clandestine Revolutionary Committee, General Command of
the Zapatista National Liberation Army (EZLN)
(Comit Clandestino Revolucionario Ind gena-Comandancia General del
Ej rcito Zapatista de Liberaci n Nacional)

Once more, I'm not going to the Annual Debian Developers' Conference; I've wanted to go to previous editions, but I've always found excuses not to do it: too far away, bad dates, expensive if not going alone, ... and lately I'm too busy to do Debian related activities, mainly because I still have two jobs and my free time is devoted to my son (code named The Beast), that will be two and a half years old in four days. Anyway, this time I'm a little bit sad, because this edition is closer to home and I could have taken my family with me and be away from a stressing period at work. Oh, well, maybe next year I'll change my life and will be able to go. Let's hope so.

I’m using firefox as a web browser (well, Iceweasle in reality, but it is mostly the same, no ?). One thing I like, is that there is a spell checker that can spell check what one enter into a web form. But this spellchecker is not the one I would have choose. For exemple, when you are typing in French, and you type “ame”, the iceweasel spell checker propose as possible correction :

• rame
• lame
• came
• dame
• aime

but not “ me”, which is the obvious correction. I’m not sure of what spell checker iceweasel/firefox use, but for sure, the first corrections to propose in French are the one where only difference is the presence or absence of accent. Even good old ispell in this case propose “ me”, even if it can’t find “ t ” as a possible correction for “ete”
Aspell do that for French, but I cannot find a way to use Aspell for Firefox version 2 or latter.
It seem that there are other browser that could use Aspell (at least Opera and Konkeror), but I don’t really want to change of web browser for this… And I don’t want to install kde (I’ve learnt to use gnome, don’t want to change), and there is no debian package for opera (is opera even free? I haven’t follow the evolution there, and is there an amd64 build somewhere? on opera.com they give me an i386 version it seem). Is there any solution ? Another think I lake, is the equivalent of M-TAB when one is using flyspell-mode in emacs : when one have just enter a word, it can use M-tab to go through the possible correction of the misspelled word. But there might be a keybord binding I don’t know about there.

hi, planet && iso8859-15 to utf8 with netatalk magic Hi, Planet!
Welcome to my blog, and thanks to Martin for subscribing me. I m not going to bother you with any details about me, if you have any questions feel free to ask me on freenode or oftc (in case you re wondering: my nick is bzed).

After a longer time of planning we ve upgraded our main file server at work: New machine, new storage (7TB - woohoo!), Etch instead of Sarge and UTF-8 for file names instead of ISO8859-15. Everything not a too complicated thing, except for renaming the about 10,000,000 files from ISO8859-15 to UTF-8 in a proper way.
In theory this is easy, as convmv happily takes care of the problem. But this did not work for us unfortunately, as we re not only providing NFS and Samba services, but also AFP (Apple_Filing_Protocol) via netatalk. Years ago you had to choose ISO8859-15 if you wanted to allow your Apple and Windows users to see the same, non-broken file names - not only because AFP didn t support UTF8 before version 3, which came out with OSX1). Normally you d think that a server would not allow a client to write a file with a file name which includes not supported characters - again, OSX and netatalk go their own way here:
Characters which are not supported by the underlying file system are character(!)-wise encoded using the CAP encoding. So we had to find a way to

• convert file names from ISO8859-15 to UTF-8
• replace CAP encoded characters by the appropriate UTF-8 character

Luckily netatalk ships the uniconv(1) util2) which is able to take care of this problem. It also updates the CNID3) Database at the same time. There s two things you need to take care of in case you re using the dbd CNID backend:

• uniconv assumes that cnid_metad is listening on port 4700, you need to start it first. Don t run afpd at the same time!
• If you ve specified external CNID dbpaths in your AppleVolumes.default (normally they re located in the root of each volume, prone to be deleted by users accidentally), you need to move the .AppleDB directory into your volume s root directory, otherwise uniconv will create a new CNID database. Just move it back as soon as uniconv is finished.

Be careful: uniconv assumes that all colons in a file name are part of a CAP encoded character. That s not a problem as long as you have only Mac and Windows clients accessing your server, as both systems do not allow colons in file names. In our case we found a number of cache files from Nautilus (which is running on our sunray/Solaris web-surf terminals as default browser) with colons in the file name. I hope no user will miss them Including a final synchronisation of all files to the new storage, about two hours it took uniconv to convert the file names and a aunch break, we were finished after about 14 hours of work. Until now it seems that our users didn t notice the change at all, except for those who realized that they have much more free space now. Looks like a successful weekend so far! 1) and uses a NFD variant of Unicode normalization (UTF-8#Mac_OS_X), but netatalk takes care of this and translates it into the NFC form, which is used on most other platforms 2) it s called netatalk-uniconv in Debian and shipped with netatalk 3) Catalog Node ID, another crust from old Mac OS versions

I wanted to spill the discussion of fonts from Mark’s blog and share what little I know on the topic. So I am localising Webconverger for each locale and learning as I go. My aim is for each locale version to meet the requirements of that locale. That allows some focus, as making a distro that can hit every locale is really hard. On my unstable system, I have somehow have 179 fonts installed.

monty$ fc-list   wc -l
179

Ok, 50 of them are ttf fonts. ttf fonts are what get used on the Web, so they count the most to Webconverger. I am going to ignore bitmap fonts et al for some simplicity and sanity. I am also not going to talk about hinting and anti-aliasing, though I do have opinions. :) Anyway, 50 ttfs is still too many considering the “free fonts” often don’t really replace the msttcorefonts, which I think they should really try do. The leading free fonts like Bitstream Vera Sans and DejaVu Sans look the same to me (~sans-serif) and are seriously boring. I guess they’re comparable to msttcorefont’s Verdana. Great… one down. :) Though DejaVu Sans has really good i18n coverage, another thing I’m interested in. Unfortunately i18n coverage esp. with China/Japan/Korea (CJK) markets is difficult.
AFAIK there is no CJK msttcorefonts, so general ‘free software’ compatibility with proprietary font faces out there is really bad. There are free CJK fonts though many of these free fonts are really bad quality. Some are OK quality, though they aren’t comparable to the proprietary fonts and hence too often the Web page renders badly. Ouch. Then some people in China prefer some font over another so this problem gets quite complex. Add Korea and Japan to the mix who also use Chinese characters in their own “special way”. I’ve been testing out sina.com.cn and taking screenshots. So I have a Chinese friend and asked him to try a special Chinese specific Debian live build of Webconverger. He says all the Ubuntu geeks use VeraSansYuanTi. However that’s non-free, sigh…. My friend also writes:

sorry for Chinese there is no best font..
because China government never try hard for this;
all kinds of fonts is some company create;
and u know China had so log history so many great beautiful font style
.. but not become computer free fonts...

I think the opensource evolution model doesn’t work very well with fonts. They need to be super high quality from the outset. Throwing a bunch of fonts at the problem gets us in our current confused situation. I think it would be great if governments could settle on some core fonts instead dictated to by Microsoft, or by whatever proprietary font gains acceptance in other markets.

I wanted to spill the discussion of fonts from Mark’s blog and share what little I know on the topic and learn from the lazyweb. :) So I am localising Webconverger for each locale and learning as I go. My aim is for each locale to have it’s own Webconverger i18n version. I hope this approach allows some focus, as making a distro that can hit every locale well is really hard. On my unstable system, I have somehow have 179 fonts installed.

monty$ fc-list   wc -l
179

Ok, 50 of them are ttf fonts. ttf fonts are what get used on the Web, so they count the most to Webconverger. I am going to ignore bitmap fonts et al for some simplicity and sanity. I am also not going to talk about hinting and anti-aliasing, though I do have opinions. :) Anyway, 50 ttfs is still too many considering the “free fonts” often don’t really replace the msttcorefonts, which I think they should really try do. The leading free fonts like Bitstream Vera Sans and DejaVu Sans look the same to me (actually they are the same, thnks Panthera!) and are seriously boring. I guess they’re comparable to msttcorefont’s Verdana. Great… one down. :) Update: Mike Day informs me this is what Red Hat are doing, developing a set of fonts that are metrically equivalent to the key Microsoft fonts. DejaVu Sans has really good i18n coverage, though unfortunately i18n coverage esp. with China/Japan/Korea (CJK) markets is difficult.
AFAIK there is no CJK msttcorefonts, so general ‘free software’ compatibility with proprietary font faces out there is really bad. There are free CJK fonts though many of these free fonts are really bad quality. Some are OK quality, though they aren’t comparable to the proprietary fonts and hence too often the Web page renders badly, as it was designed with some other proprietary font in mind. Ouch! Web designers please use ‘em’ units and this problem such as over lapping text could be avoided. Then some people in China prefer some font over another so this problem gets quite complex. Add Korea and Japan to the mix who also use Chinese characters in their own “special way”. I’ve been testing out sina.com.cn and taking screenshots. So I have a Chinese friend and asked him to try a special Chinese specific Debian live build of Webconverger. He says all the Ubuntu geeks use VeraSansYuanTi. However that’s non-free theft, sigh…. My friend also writes:

sorry for Chinese there is no best font..
because China government never try hard for this;
all kinds of fonts is some company create;
and u know China had so log history so many great beautiful font style
.. but not become computer free fonts…

I think the opensource evolution model doesn’t work very well with fonts. They need to be super high quality from the outset. Throwing a bunch of fonts at the problem gets us in our current confused situation. Update: Henri Sivonen wisely points out that “in the case of DejaVu, adding glyphs to Vera as an open source project seems to work”. I think it would be great if governments could settle on some cultural core fonts for their locality instead of being dictated to by Microsoft, or by whatever proprietary font gains acceptance in other markets.

We've just released eds-sync 2.20070531, with some hot new features.

• Allow building against a stock evolution-data-server 1.10
• Handle duplicate contacts correctly
• Batch contact commits if possible (requires eds-dbus)
• Don't work on the same contact multiple times
• Remove code that exists in Glib
• Expire the inspect cache when handles are removed
• Set the alias when adding a contact to the roster

I've uploaded a tarball to Maemo Garage which should be visible soon, but for now there is a tagged release in Subversion.

The good thing about having kept a blog these last few years, is I have a searchable record of hardware changes. Recently, I noticed a few disk errors on caesar, and I was wondering how old that disk was. I disk some searching, and found via this post that the disk in it was a hand-me-down from when I decommissioned daedalus Mk I. One of the reasons I decommissioned it when I did, was the RAID-1 was exhibiting some strange errors during resynchronisation, and I didn't want to leave the country leaving behind an ageing problematic server with my stuff on it. I'd previously been unable to fault either 120 Gb disk under some light testing with badblocks, so I just reused them, and so around October 2005, I put the larger disk in caesar. So this disk has seen a fair bit of use, originally in daedalus, and then in caesar for another 20 months. It's allowed to fail, it's served me well. Anyway, so I bought a 750 Gb drive a couple of weeks ago, because one thing that I've wanted to do with caesar was do remote backups of daedalus, and there just wasn't enough space to do that, so I went for as big as I could find, and essentially repeated the same trick as last time, to upsize:

1. Power down, attach new disk as a slave
2. Boot into single user mode
3. Unmount everything, remount root read-only
4. sfdisk -d /dev/hda sfdisk /dev/hdb to copy the partition table (I use a root, swap, LVM, three partition layout)
5. dd if=/dev/hda1 of=/dev/hdb1 bs=1M to copy the root filesystem to the new disk
6. mkswap /dev/hdb2 to put the swap signature on what will become the new swap device
7. cfdisk /dev/hdb and blow away the third partition and recreate it using all of the available space (there's probably a programmatic way of doing this, but hey, it's not exactly hard to do in cfdisk either)
8. pvcreate /dev/hdb3 to put an LVM signature on the new LVM physical volume
9. vgextend base /dev/hdb3 to tell LVM there's two disks in the volume group for the time being
10. Remount the root filesystem read-write
11. rm /boot/grub/device.map and run /usr/sbin/grub-install /dev/hdb to install GRUB in the MBR of what will become the new boot disk
12. Power down, swap the disks around so that the new disk is now primary and the old disk is now the slave
13. Boot into single user mode again, unmount everything except root
14. pvmove /dev/hdb3 /dev/hda3 to relocate all of the allocated physical extents off the old disk onto the new one
15. Twiddle thumbs for a couple of hours. Admire artwork on computer's motherboard (did you know Compaq EVO's have a picture of a car on their motherboards? A map of Texas?)
16. vgreduce base /dev/hdb3 to take the old disk out of the volume group
17. pvremove /dev/hdb3 to remove the LVM signature from the old disk
18. Power down and remove the old disk
19. Reboot into multi-user mode and profit from all the additional free physical extents

I absolutely love how I can do this with LVM, with a minimum of fuss. Last time I did this, I had a kernel panic during the pvmove, but that didn't even stop it, I just rebooted and reran the pvmove and it carried on from the last checkpoint. No such excitement this time. I'm sure Solaris' ZFS does all of this with substantially less manual steps, but hey, I like my Linux :-)

A few days ago, some people in Venezuela were shocked by Mark Shuttleworth’s claims against Hugo Ch vez Government and supporting Mar a Corina Machado, a minor opposition leader in the Country. Today, at 23:59:59, the nationwide open air TV broadcaster RCTV will cease operations. In Venezuela, as in several other countries in the World, the economically profitable portion of the electromagnetic spectrum is State’s property, thus requiring a concession to make use of it. RCTV concession was granted in 1987 for twenty years, and the Government has decided to deny the renewal, which is quite unusual (right-wing governments in the past simply would have shut up the antennas) Of course, RCTV is not a pro-Government channel, accounting between 90 and 100 percent of the airtime with anti Government news and opinions, which is not bad per se, though in Venezuela there’s a law requiring equilibrium in times devoted to political ads. Both situations (Mark’s comments and the Government’s decision) pose the threats of a now common situation in our Country: people taking very valid decisions with somewhat doubtful consequences. In the first case, Mark is using his right to freely write what he likes in his blog but it seems that it has damaged a little bit Ubuntu’s image in Venezuela (not that it’s Mark’s fault, of course) In the second case, the Government is taking a lawful decision, but it has actually raised an important discomfort in the population (polls, which are not always accountable, point out as much as 80% rejection rate), the journalists of other media and most foreign observers, which state that they’re actually closing a TV channel and destroying freedom of speech in the Country. Expressing political POVs is quite critical nowadays in Venezuela. Some famous etiquette bookwriters, such as the one at Emily Post’s, always tend to promote a be honest, be respectful position while listening or expressing your points of views in order to keep a productive attitude. I do think that politics are important: I just think mixing them with practical daily tasks, including FLOSS development and promotion, is becoming obnoxious and dangerous. Cheeseconf On June 15th., I’ll be departing to DC7 in Edimburgh. I’ll stay a night in Frankfurt, probably in a hostel, and then I’ll stay at one of the hostels in EDI. I plan to carry white cheese to the Cheese and Wine BoF, and I’d love any comments on how to transport them without posing a biological threat for Europe. I’ll be carrying a 65 liters backpack. How long will it take for the cheese to become a lifeform? Any special tips and tricks to avoid that? Copying files to embedded devices I’ve been trying to copy some binary files into an Axesstel D800 EvDO modem which runs Linux over a tiny ARM processor on less than 16 MB. of memory. I’m actually trying to get netcat in so I can get more useful stuff such as snmpd in. The modem has telnetd, but it has no zmodem utilities, particularly rz. Neither it has wget, nor ftpd. It has thttpd to serve the administrative webpages, but it can’t process shell CGI programs so I can’t send the file through a CGI form. Any ideas? I would love to read comments from hackers who have already managed their ways into embedded devices such as 802.11 devices with Linux, and any generic ideas on how to provide new firmware images for devices such as this Axesstel modem.