I've spent some quality upstream time lately with man-db. Specifically, I've been upgrading its locale support. I recently published a pre-release, man-db 2.5.0-pre2, mainly for translators, but other people may be interested in having a look at it as well. I hope to release 2.5.0 quite soon so that all of this can land in Debian. Firstly, man-db now supports creating and using databases for per-locale hierarchies of manual pages, not just English. This means that apropos and whatis can now display information about localised manual pages. Secondly, I've been working on the transition to UTF-8 manual pages. Now, modulo some hacks, groff can't yet deal with Unicode input; some possible input characters are reserved for its internal use which makes full 32-bit input difficult to do properly until that's fixed. However, with a few exceptions, manual pages generally just need the subset of Unicode that corresponds to their language's usual legacy character set, so for now it's good enough to just recode on the fly from UTF-8 to some appropriate 8-bit character set and use groff's support for that. man-db has actually supported doing this kind of thing for a while, but it's been difficult to use since it only applies to /usr/share/man/ll_CC.UTF-8/ directories, while manual pages usually aren't country-specific. So, man-db 2.5.0 supports using /usr/share/man/ll.UTF-8/ instead, which is a bit more appropriate. Also, following a discussion with Adam Borowski, man-db can now try decoding manual pages as UTF-8 and fall back to 8-bit encodings even in directories without an explicit encoding tag; if this fails for some reason, you can put a '\" -*- coding: UTF-8 -*- line at the top of the page. I'm still debating whether Debian policy should recommend installing UTF-8 manual pages in /usr/share/man/ll.UTF-8/ or just in /usr/share/man/ll/. Initially I was very strongly in favour of an encoding declaration, but now that man-db can do a pretty good job of guesswork I'm coming round to Adam Borowski's position that people should be able to forget about character sets with UTF-8. Opinions here would be welcome. One thing I haven't moved on is that any design that assumes that the encoding of manual pages on the filesystem has anything to do with the user's locale is demonstrably incorrect and broken; I'm not going to use LC_CTYPE for anything except output. However, maybe "UTF-8 or the usual legacy encoding provided that the latter is not typically confused for the former" is a good enough specification, and that still has the desirable property of not requiring a flag day. I'll try to come down from the fence before unleashing this code on the world.

As I wrote last week to debian-devel@l.d.o I started the again a mass bug filling against all packages which fails to build twice and three times in a row. Furthermore we checked this time if the package content has changed between the builds. Today I completed these bug reports finally :). I wrote today about ~400 bug reports against several packages. Some packages which are very well known are:

• aptitude
• php5
• debian-policy (my favourite ;-))
• ...

All failures concering doublebuild or three time build issues are filled with Severity: Important and all bugs against packages which have a different content after the builds are filled with Severity: Serious. Please fix all of them as soon as possible (To fix this issues is a release goal for lenny ;-)) To view all bug reports please have a look here and here.

FYI, my new Miro packages (formerly known as Democracy Player) have now reached unstable. After lots of ugly, ugly trouble with even getting a successful build (boost/python/dbus related, you don't want to know) the packages are back in shape now, with tons of fixed (or no longer reproducible) bugs and lots of upstream impovements and new features. If you reported a bug against Democracy Player, please try the latest Miro package and check if it still occurs, thanks! The upgrade should be seamless, your existing config and videos will be migrated from ~/.democracyplayer to ~/.miro automatically upon the first start of Miro. Some of the new/fixed things in this release include:

• HTTP proxy support (uses the GNOME proxy settings, use gconf-editor to change them).
• Flash videos now play fine (non-jerky) and with sound!
• You can search various video sites (Youtube, Google video, etc.) online, and even save searches as channels.
• You can export your channel list into an OPML file (and also import OPML files, of course). I've been waiting for this for a very long time (it's a good way to backup your channel list, or move it to another machine)...
• Lots and lots of bugfixes and small enhancements, as usual.

Over the past couple of weeks I’ve started getting messages like this from the SourceForge:

Your membership in the mailing list whatever has been disabled due to excessive bounces The last bounce received from you was dated 15-Sep-2007.

It’s true - the systems that handle my mail do generate a number of SMTP time rejects for open posting SourceForge lists. My mail server does SMTP time rejection of some spam and these lists don’t appear to have much in the way of spam filtering. It’s a bit depressing since the main problem appears to be limited filtering on the SourceForge side which renders the lists in question rather difficult to read but it’s unsurprising given the nature of the service.

My Python development environment consists of a GVim window for editing code and a gnome-terminal for running the unit tests. This works out well, but leaves some integration to be desired. In particular, jumping to a test failure point is a little cumbersome. I used to do it this way: copy the filename from the traceback, switch to vim, then type :e, paste the filename with middle-click (now I know I can use * instead) and press Enter, then enter the line number and press G. That is barely acceptable; this particular wart has even contributed to pushing a ex-co-worker towards Emacs (the horror!). I have written a small vimscript function to make the aforementioned use case slightly easier. Now to jump to a failure point, just triple-click the corresponding line in the traceback (that selects the whole line), switch to Vim and invoke the function, which scrapes the clipboard for a filename and line number and opens it in the active buffer. Installation: paste the OpenFileFromClipboard function in your ~/.vimrc and bind it to a key by adding the following lines to your .vimrc:

nmap  :call OpenFileFromClipboard()
imap  :call OpenFileFromClipboard()

These commands bind the key F4 to our function. The first command handles normal mode, and the second handles insert mode (hence the to momentarily escape the insert mode). You can of course change the shortcut to whatever you like. The function should handle following types of references from the clipboard:

• /etc/passwd
• John saved his work to ~john/doc.txt and left.
• File "/home/gintas/devel/project/tests.py", line 123, in tests.py
• /usr/include/stdio.h:333:extern int sprintf (char *__restrict __s,

I hope someone finds this useful.

Satellite TV commented:

"Wow, all that seems pretty complex to me. The U.S is so much more simple, we just chose Dish Network or DirecTV or some kind of cable and call it a day. Then again I don't know if thats a bad or a good thing."

That's a bad thing. Look at the situation: In the US, there seems to be very little free-to-air (FTA) satellite TV for most of the country, except PBS and religious channels. The main US FTA FAQ answers "Should I drop cable/Dish/DirecTV and switch to FTA?" with "Probably not." I'd say "Probably yes" to similar questions in Europe, as long as you aren't dependent on one of the few incompatible pay-TV channels. Channels like MTV, Nick, TVEi and DW are all pay-TV in the US, but FTA in Europe. But if you're in the US, look on the bright side: Canada seems even more behind on this. In Europe, you can still call whatever satellite broadcaster (such as Canal Satellite, Premiere or even Sky) and get a system installed if you choose, but you can also install your own. That's what my FAQ is about: giving people in "the land of the free TV" some starting info on how to do that. Some broadcaster-supplied systems only really work well with one broadcaster's "approved" channels (Sky are particularly bad for that), are more expensive to upgrade than standard systems and don't give viewers the equipment to repair it when it breaks, which encourages them to keep paying the broadcaster's agents. Some of those agents do some pretty dishonest tricks when installing dishes, such as putting them far higher up than necessary, to discourage viewers from fixing it themselves and to advertise satellite TV to the neighbours. It's pretty easy to install a standard satellite system... if you can put up a wall bracket and tune in a video, you can probably do it. Alternatively, there are independent installers who will do it for you, who might seem like they charge more than the broadcaster's agents, but remember that a good one won't keep charging you every month or try to sell stuff to you forevermore!

Gunnar wrote that he recently had to explicitly accept the GPL via some dialog box when doing an OpenOffice installation on windows and users are also prompted with the license agreement on the first start of the application.

Why should someone want this?
Are you also FREE to refuse this and still use the software? I mean the GPL is about distributing the software, not using it, so wtf?

It’s a common story. A housemate starts torrenting some American television series of dubious artistic quality and/or legality, or perhaps starts calling their girlfriend on some fluffy non-free privacy invading spyware and your SSH sessions start to lag. Why does this happen? When the upstream channels become saturated, data must overflow into the modem’s buffers. This causes TCP acknowledgement (ACK) packets originating from your SSH client to be delayed, resulting in an increase in the overall round trip time. This phenomena is usually worse on cable connections as their modems’ buffers are generally larger. Avoiding the problem can be pretty simple — limiting your upstream bandwidth just below its maximum helps, and some smarter solutions involving prioritising ACK packets work even better. Although these are best achieved at your router, a number of applications have their own limiting features: every BitTorrent client will have one, and other tools such as scp (via the ‘-l‘ option) have them too.

I completely agree with Adrian's comment. I've recently installed a lab full of OpenOffice for Windows. Of course, the installation is easy (but you have to explicitly accept the GPL to get on with it - why?), but what's really annoying is that the first time each and every user starts the program for the first time, they are also prompted for license agreement. Even worse, they are expected to fill out a quite lengthy survey (which I filled once, of course, but still is too much to ask for the vast majority of our users!). Free Software should be exempt from the stupid click-through nonsense!
Anyway... also echoing Adrian and Otavio: Robert Millan's work on win32-loader just kicks ass. If you are not reading me through Planet Debian, please take a look at this ogg video file (Windows users: you can use VLC) showing what happens if you insert the (Sid) Debian installer CD into a Windows system. No more fiddling around trying to get the stupid machine to boot off your favorite media! Good job!

Argh... I know it seems like I've given up on blogging entirely, but that's not the case. I've just been spending a lot of time working on Evan Prodromou instead of on evan.prodromou.name, which has been a whole different kind of world. I'm mostly poking my head up to point to an article in the Montreal Gazette about my new project, Vinismo. Montreal entrepreneur uncorks Wikipedia of wine, says the headline. I guess that "Montreal entrepreneur" is little ol' me. Go figure! The article is nice and balanced -- it covers the obvious fact that one month after launch Vinismo is not the definitive end-all be-all of wine information... yet. Roberto Rocha, the article's author, is doing a great job covering local entrepreneurs and the startup Internet scene, and I'm glad that our site got his vote of approval. And check out the photo they took of me and Niko at the excellent Pullman Wine Bar. How sophisticated. Glad to be getting some press about the site, and good news for Vinismo. tags: wine vinismo montreal gazette me press

BarCampCanada1 I managed to convoke the very busy trio of Sylvain Carle, Simon Law and Fred Ngo yesterday afternoon at La ka to nail down some plans for this fall. (We were lucky enough to get the funny and insightful Madame Woo to help out, too.) The cement's not dry yet, but it looks like we're going to have BarCampCanada1 here in Montreal the 9-10 of Novemeber. That gives us about 8 weeks to plan out what we hope will be the biggest BarCamp ever in Canada. tags: barcamp barcampcanada1 barcampmontreal

From time to time we have the problem that a mail posted to a list triggers an autoresponse (vacation, bounce, tdma) from an unexpected source. Sometimes it isn't possible to identify the subscriber who is causing this. One example for this was 'petsupermarket'.

So i wrote a small tool, which takes Mailadresses as input and resolves the domains until it reaches IP-level, so it is possible to identify 'related' addresses to such an incident.

I now ran this tool for all 38656 domainparts mailadresses that are currently subscribed to some list at our listservers:

Unresolveable Domains                             :   443
Grounded Domains                                  :     3
Domains with A                                    :  1058
Domains with CNAME, A                             :    59
Domains with CNAME, CNAME, A                      :     3
Domains with MX, A                                : 36202
Domains with MX, CNAME, A                         :  1199
Domains with MX, CNAME, CNAME, A                  :    22
Domains with MX, CNAME, CNAME, CNAME, A           :     5
Domains with MX, PTR                              :    36
Domains with unresolvable Hosts                   :   361

(You may have noticed that those lines doesn't sum up to 38656. It is possible to have more than one MX-Host, and it is also possible to have more than one IP in an A-Record. If one of those combinations falls into another category it is counted twice)

so we have 36528 domains that are completely configured correctly according to RfC 2821 with a MX-Record pointing to an A-Record, or without a MX-Record and an A-Record. These are 94.5% of all domains.

but we have 443 domains in our list, that didn't resolve at the moment i ran my script, those have to be investigated.

We also have 3 domains that are configured with 'IN MX 0 .' as decribed in this expired Draft, those also have to be investigated, and thrown out.

We have 1288 domains, which use CNAMEs (or CNAMEs pointing to CNAMEs) in their MX or directly on their domainname. RFC1034 says:

Domain names in RRs which point at another name should always point at the primary name and not the alias. This avoids extra indirections in accessing information.

then there are 36 domains which point their MX directly to an IP-Number. RFC 1035 says:

3.3.9. MX RDATA format

    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
                       PREFERENCE                    
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    /                   EXCHANGE                    /
    /                                               /
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

where:

PREFERENCE      A 16 bit integer which specifies the preference given to
                this RR among others at the same owner.  Lower values
                are preferred.

EXCHANGE        A <domain-name> which specifies a host willing to act as
                a mail exchange for the owner name.

MX records cause type A additional section processing for the host
specified by EXCHANGE.

So according to that an MX has to point explicitely to a Full Qualified Domain Name, and it has to be an A-Record it points to.

However: Most MTAs (including ours) these days forgive this, and figure out the right thing.

At last: we have 361 Domains which MX and/or CNAME-Records point to Hostnames that are currently unresolvable. A quick check shows that often the problems only appear on one MX-Record, while another is correct, so the service is functional.

So maybe now it is a good idea and check your own DNS-Setup and the Mail-related Data.

I cannot build the new upstream rtorrent package (at least not as should been built) because of libcurl4-openssl-dev and the dependency on libssh2-0-dev that's a virtual package right now on sid (#442198). But I couldn't wait until curl get fixed so I downloaded libssh2-0-dev for lenny and installed it in my machine so I could build it (not with pbuilder but is built! :-) ) So you can find libtorrent 0.11.8 and rtorrent 0.7.8 binaries (and source of course) for amd64 right now on my server, in some minutes I will have the i386 binaries as well. The most important option added in this new release is the peer exchange option and a lot of commands for XMLRPC that in debian we can't enjoy by the moment. BTW, if someone can take care of xmlrpc-c which seems to be forgotten (or with a MIA maintainer) I would appreciate it (and finally be able to build rtorrent with XMLRPC support!!) Anyway, having rtorrent 0.7.8 in debian unstable could take at least a week more :-/ (while curl get fixed) so if you really want it grab them from my server or built it with my dsc diff.gz and orig.tar.gz and dpkg-source -x package_version.dsc and remember if you're building rtorrent you need libssh2-0-dev installed.

In the past couple of weeks I’ve:

• Started a new job at Aplix UK in Guildford
• Moved to Woking, Surrey
• Started to ride about 15 miles a day in my commute to work. My ass!

Really interesting projects at work. I helped put together a position paper for an OpenAJAX/W3C meeting. It was accepted and now I’m off to San Francisco for the first time for the last week of September! It’ll be my first time in North America on my travels. Btw my employers are hiring! As for Debian work. I am still struggling to a get a stable security update of wordpress in. :/
I made an update to flup, which will hopefully be one of the last. Please consider moving your Web application to modwsgi. I was surprised to see Webconverger in the Linux Format magazine #97, which is great news. There should be big announcements soon concerning Webconverger. Busy, busy, busy! :)

Since it's not really my primary focus these days - there are simply too many things like it out there - I decided to release Stuff To Do as open source under the GPL. I chose that license, because as a complete application, I don't see it being that much of an issue, as compared to, say, a library, where I find the GPL annoying. If you would like to use some portion of the code under a more liberal license, feel free to ask - I'll almost certainly say yes, even if it's for your own proprietary project. The code isn't very elegant, I was in a hurry to create the whole thing, and was still learning with Rails, but it does work. It currently has zero documentation, in terms of describing the code. If all that sounds exciting, the RubyForge project is here: http://rubyforge.org/projects/stufftodo/

Okay, apart from finishing the move from the laptop back to the Pegasos (I compiled d2x-xl yesterday, today is reserved for the more serious stuff), this evening will be dedicated to cooking and my other pet project, xcontrol. So, it will be risotto, wireless networks and autobuilders, in no particular order. In other news, I've been running an archive rebuild to find out which packages actually implement the build-arch and build-indep targets. Out of the first 70, I've counted 11. Since so many packages would be affected, a mass filing of wishlist bugs is probably not the way to go, so I guess I'll set up a web page with the results (after making my autobuilder more robust; it still gets too easily confused) and try to shame people into updating their packages.

In other words: Sex sells. We need blue LEDs. Blinking lights. More gadgets. The new Debian loader to boot into Debian installer from a Windows machine is totally cool. And it has become very obvious that we need a bootsplash in the installer (and if the user selects Desktop or Laptop in tasksel or whatever it's called these days, in the default install, too.) The people we are trying to reach via such migration tools are not technical people but average computer users - and they won't understand why Debian boots DOS at first because the graphical part comes up. (If you don't do tech support to non-technical users: any text mode screen will automatically be referred to as DOS. I've seen that again and again, and it's always difficult to explain to these people that “up to date operating system” doesn't automatically mean “GUI” Is this important? Of course it's not important for the current average Debian user (who knows quite a bit around Linux, and who also, I guess, has moved to Debian after having tried and discarded Fedora or SuSE.) But it's important for new users who migrate to Debian directly from Windows, who have a hard time using Synaptic with the default settings and who were just lucky (instead of having made a choice) when they bought it that their Internet access uses an Ethernet based device instead of a cheap USB modem. Do we want those users? They may be a burden to support, but overall I think that the wider our user base, the wider is the pool of potential contributors (translators, or even just users who help other users.) (... and incidentally: please, please, please let's skip the “accep the GPL” dialog. Or at least check the box by default. After all, the big difference to our proprietary competitors is that we don't restrict use and redistribution of the software at all, and modification only moderately, so if a user has decided to burn this CD-ROM and try this system, we shouldn't annoy him.)

So, this time it was for real! AMD really meant it when they announce the release of specifications of ATI boards last weeks. During XDS2007 at Cambridge, we received the actual specs, 2 PDFs files, 460 pages for the M56 (Mobility X1600, r5xx board) and 436 pages for rv630. The files are now online, but I am not going to give the link here since the web server already suffered too much today (and the URL is easy to find anyway). People should NOT understand this as if we were going to get an open-source driver for all ATI boards next weeks. First, these are only 2D specifications for r500 and r600. 3D specs are expected in a couple weeks once AMD fixes the remaining issues. r300 specs might arrive later. Also, these specifications are pretty hard to use, as expected. There are something like 9000 registers in r6xx boards, and these PDF files are just description of these registers. It's of course far away from being a "How-to write a driver?". So if you want to help, there will be lots of things to do. We also got a demo from some Suse guys of the driver there are working on. They got these specs about 2 months ago. And they already have some part of modesetting working. It means you might get your X server to start. There are about one hundreds different chipsets with their own quirks, so still lots of work to do. And then lots of performance things, 3D, ... to do. This driver is called "Radeon HD" so far. It will remain separated for now, but it might end up being merged with the upstream ATI driver later since the 3D engine are very similar (I personally hope it will happen). The reverse-engineered Avivo driver is dead now then. Fortunately, Jerome Glisse has been aware of the event for a couple months now, so he didn't waste to much time working on it. xserver-xorg-video-avivo will remain in Debian experimental until the new driver arrives. The good thing about Avivo is that it (and the ongoing Nouveau driver for Nvidia) led to the development of very nice reverse-engineering tools (revenge, renouveau, mmio trace, ...). They will be very useful for other graphic boards, or even some network hardware. Of course, many thanks to AMD for making this happen!

After one day off, we watched Argentina-Georgia yesterday evening. Interesting game because it directly affect France's chances to qualify after Argentina's win on Friday. Argentina had a big disadvantage by playing only 4 days after their historic battle against France. As a logical consequence, they appeared quite tired and very imprecise. On the other hand, Georgia fought very courageously with a strong pack of forwards, encouraged by most of the crowd in Lyon. Finally, Argentina won *and* scored the bonus point by scoring a try in the very last minute (which might become very important to qualify in this pool. Good game even if Argentina was not that impressive, but I still think this is mostly because of the 4-days delay. Today, 3 games were scheduled. We didn't watch USA-Tonga (15-25). USA was slightly favourite for that one but they also were quite tired by playing 4 days only after their battle against England on Saturday. This was the first match for Tonga and obviously the island players were less tired and played a good rugby to win that one. Japan-Fidji (31-35) seems to have been the most uncertain match as of now. I somewhat regret not watching it as these two are among the teams I like watching and 7 tries in the match probably made it very interesting. Tonight, we watched Italy-Romania (24-18). The italians compete with Scotland for the remaining qualifying spot in that pool, if one assumes that New Zealand will obviously win all its matches. So, Italy *had* to win that match. The quality of romanian rugby has decreased in the last 10 years, but they still have a good set of great players, most of them playing...in France (just like one third of the italian team, indeed). The match was quite indecise with italy keeping the ball all first half long, but only leading 8-0 on half time....and being surprised by two romanian tries at the beginning of second half (8-12). After a strong domination of Italy, Romania was logically sanctioned by one expulsion and a penalty try (for successive voluntary faults), but, until the end, Italy was still in danger. They finally made it, which was the most important for them, 4 days after they were sweeped out by the All Blacks. We're now eagerly waiting for *THE* game of this week, England-South Africa in the Stade de France on Friday. Elizabeth found out that a few tickets are still available, but for 253 euros each, we decided to watch it on TV. So, between 21:00 and 23:00UTC that day, don't even think about pinging me on IRC. I still haven't decided who I'll support the most as I'm not a huge fan of either team....but they're still among the best in the world, of course.

Trying to post the movies for Tour of Britain stage 2: Taunton left me in a whole world of pain. I'd decided to take movies rather than stills because there were lots of photographers there with considerably better cameras than mine, and my blog is far more popular for TV coverage than for photos. My video camera is a four-year-old Nisis Pocket DV4 which appears as a usb-storage device and contains AVI files. Apparently, those AVI files contain Video as mjpeg, yuvj422p, 320x240, 9.04 fps(r) and Audio as pcm_s16le, 11025 Hz, mono, 176 kb/s. I wanted to post the movies as Ogg Theora for various reasons, including how well it seemed to work for debconf this year. I'm a total clubie at this video stuff, but I like freedom and I've seen what I want, so I know it's possible. First of all, I tried mplayer's mencoder. It seems that mplayer can play Ogg Theora, but mencoder can't encode it. Did I miss something? What's with that? So, I tried installing VLC because several people had recommended it to me in the past. It seemed to have an easier interface than mplayer and had a wizard for converting files. Unfortunately, it seemed to crash on some of the AVI files and other times hang. When I upgraded from 0.8.6 to 0.8.6c (which also meant a wxWindows upgrade, which surprised me for such a seemingly-small upgrade), it crashed out with a segfault rather than hanging. I couldn't get the command-line to work either. I tried ffmpeg, but damned if I can get that to produce a good theora (or is it libtheora?) file. Sometimes it works, but the output is much larger than I expected. It is very annoying that there are no releases, but at least there are snapshot tarballs available. I was trying export-2007-09-12 early this morning. Finally, I tried ffmpeg2theora and (after upgrading ffmpeg), the command

"ffmpeg2theora -aq-1 -v 0 -o outfile.ogg infile.avi"

produced useful results, which I've now uploaded. Thanks to BBLUG members for their comments. There seemed to be others who shared my confusion and frustration with video processing. Why is this still so hard? Is it, as one member suggested, the chilling effect of software patents and so on?