This is a follow-up on a previous post: I used a bit of a rainy Sunday to read some docs and write some Python, and I now have a script that converts events from a Guitar Hero World Tour drums controller to MIDI. A brief explanation of how it works: in its standard configuration, the drums controller has five pads and a pedal, and it uses a Wiimote to communicate the events (what pad was hit with what force, or what button was pressed/released, or the position of the mini-stick) to the Wii itself. This communication happens over Bluetooth and not over a dedicated cable or protocol, so it is possible to remove the Wii and use a standard computer with Bluetooth (built-in or via a dongle) instead, and the computer will receive messages. There's some decoding to do, but fortunately people have reverse-engineered the frames, and come up with the libcwiid library and its Python binding, which hides the complexity of the decoding and just delivers structured data to the programs that use it. Then we analyze this data and extract the relevant info from it (with some juggling of bits): "the X pad was hit with force Y". From there, it's a simple matter of mapping the pads to a MIDI note and the force to a MIDI velocity, and of sending the appropriate MIDI message to the target, for instance with the libportmidi library. What you do with the MIDI output is your call: I use it to add more pads to my electronic drum kit, but it could equally be used to drive a software synthetizer, or an external sound module. Or a light-show machine. Or fireworks. Or a model railway. Or whatever. Here's the current version of the wiigh2midi.py script (for updated versions, see the Bazaar repository).

#! /usr/bin/python
import cwiid, time, asyncore, pypm
class Wiimidi:
    wiimote = None
    midiout = None
    lastaction = time.time()
    wii2color =   27: 'kick',
                  25: 'red',
                  17: 'yellow',
                  15: 'blue',
                  14: 'orange',
                  18: 'green'  
    color2midi =   'kick': 36, # Bass drum
                   'red': 38, # Snare drum
                   'blue': 48, # High tom
                   'green': 43, # Low tom
                   'yellow': 42, # Closed hi-hat
                   'orange': 52, # Crash cymbal
                    
    def connect_wiimote(self):
        print "Press 1+2 on the Wiimote..."
        try:
            self.wiimote = cwiid.Wiimote()
        except:
            self.wiimote = None
            return None
        print "Connected."
        self.lastaction = time.time()
        self.wiimote.rumble=1
        time.sleep(.2)
        self.wiimote.rumble=0
        return self.wiimote
    def disconnect_wiimote(self):
        if self.wiimote is not None:
            print "Closing connection to the Wiimote."
            self.wiimote.close()
            self.wiimote = None
        return
    def connect_midi(self):
        for i in range(pypm.CountDevices()):
            interf,name,inp,outp,opened = pypm.GetDeviceInfo(i)
            if outp == 1:
                if name == 'UM-1G MIDI 1':
                    self.midiout = pypm.Output(i, 0)
                    return
        for i in range(pypm.CountDevices()):
            interf,name,inp,outp,opened = pypm.GetDeviceInfo(i)
            if outp == 1:
                print i, name," ",
                print
        dev = int(raw_input("Type output number: "))
        self.midiout = pypm.Output(dev, 0)
    def wiimote_callback(self, messages, timestamp):
        self.lastaction = time.time()
        m = messages[0][1]
        # rx = m['r_stick'][0]
        ry = m['r_stick'][1]
        l = m['l']
        # r = m['r']
        which = ( (l << 1) & 16) + (ry >> 1)
        softness = l & 7
        if softness == 7 or which == 0:
            return
        # print "rx=%d\t ry=%d\t l=%d\t r=%d\t rest=%d" % (rx,ry,l,r,rest)
        # print "which = %d, softness = %d" % (which, softness)
        strength = 7-softness
        color = self.wii2color[which]
        midi_vel = 16*strength
        midi_note = self.color2midi[color]
        self.midiout.WriteShort(0x9a,midi_note,midi_vel)
        self.midiout.WriteShort(0x9a,midi_note,0)
        # print "%s %d -> %d %d" % (color, strength, midi_note, midi_vel)
    def main(self):
        self.connect_midi()
        while True:
            if self.wiimote is None:
                self.connect_wiimote()
                if self.wiimote:
                    # Tell Wiimote to display rock sign
                    self.wiimote.led = cwiid.LED1_ON   cwiid.LED4_ON
                    # We only care for the extension data
                    self.wiimote.rpt_mode = cwiid.RPT_EXT
                    self.wiimote.enable(cwiid.FLAG_MESG_IFC   cwiid.FLAG_REPEAT_BTN)
                    self.wiimote.mesg_callback = self.wiimote_callback
                    self.lastaction = time.time()
                else:
                    print "Retrying... "
                    print
            asyncore.loop(timeout=0, count=1)
            if self.lastaction < time.time() - 3600:
                print "An hour passed since last action..."
                self.disconnect_wiimote()
            time.sleep(0.05)
        print "Exited safely."
# Instantiate and run
inst = Wiimidi()
inst.main()

There are a few obvious improvements to be made: first, handling guitar controllers in addition to drums. Second, handling the buttons could provide extra "pads", to trigger sound effects for instance (thunder, lasers, voices ), or to initiate program changes, or to change the mappings of the pads. Maybe storing the mutable parts (default MIDI adapter, pad-to-note mapping) in a configuration file. Maybe add a user interface to ease the configuration. Patches welcome, otherwise maybe I'll do that myself on another rainy Sunday. Notes and references (in other words, the giants on whose shoulders this program stands):

• As mentioned, this script uses the pyPortMidi binding, available in Debian Wheezy as python-pypm (and easily backported to Squeeze).
• It also uses the Python bindings to CWiid, available in Debian as python-cwiid.
• I based the pad-to-MIDI mapping on the relevant part of the General MIDI specification.
• The bit-mangling operations used to map cwiid messages to plain data were worked out by comparing the tables from the Wiibrew wiki on the classic controller and the drums controller.
• The skeleton of the script was initially based on MythPyWii, although not much of it still remains.

More cowbell!

I must admit that I ve never been a big fan of the dash as /bin/sh change. I have three main problems with the switch: POSIX compliance as an argument Complying to standards is a really good thing. But when everybody is ignoring the standard because they want the comfort of newer features, maybe it s a sign that the standard should be updated to include those newer features. Most of the bashims used everywhere in scripts were signifiant improvements, like the ability to write:
cp short1/path1,short2/path2 /very/long/common/path/to/a/file
instead of:
cp short1/path1/very/long/common/path/to/a/file short2/path2/very/long/common/path/to/a/file The option to improve bash was not fully explored We started with the premise that bash is bloated, slow, and cannot be improved. Maybe you can help me with that, but I could only find a few simplistic benchmarks comparing dash and bash, and I could not find any analysis of why bash is slow, and why it cannot be improved.
One of the obvious problems is that bash is also an interactive shell, and is linked to ncurses and terminfo, which increases the startup time. But have we investigated the possibility to have a /bin/bash-non-interactive binary that would not be linked to ncurses? The change was brought to users While it is OK for Debian (or Ubuntu, in that case, since that change was done in Ubuntu first) to force its developers to use POSIX-compliant scripts, the switch could have been made only to Debian-created scripts (by switching them from a /bin/sh shebang to a /bin/dash shebang, for example). I have trouble justifying that this change was forced on users as well. Next: linker changes and we are doing it again. A set of linker changes (see also the Ubuntu page) was already done in Ubuntu, and is very likely to be done in Debian as well. This switch requires deep changes in some buildsystems (it requires ordering of libraries and forbids indirect dependencies), and is rather painful (it was reverted before the Ubuntu 11.04 release because it was not possible to fix all the packages during the natty release cycle, but is done in the 11.10 release). Of course, there are justifications for this change. But I m not sure that it s worth all the trouble created for users.

I recently wanted to buy some MP3 files from Amazon (a whole album in my case, but you can also just buy single MP3 files if you want). Digital music downloads from Amazon are often much cheaper than buying the physical CD (from Amazon), and you can also instantly get the stuff within seconds, without having to wait for the physical CD to be shipped to your place. The good thing about Amazon's MP3 downloads is that the files are not infested with any DRM-crap (if that were the case I wouldn't spend a single penny on such useless junk, of course). This allows you to burn the MP3 files on CDs and/or play them on any device you like (MP3 player of choice, laptop, hifi-system, car, e-book reader with MP3 playback support, etc. etc). Granted, you can not re-sell the digital files on eBay later, this is the one little drawback you have when compared to physical CDs, but I guess most people can usually live with that. Also, it would be great if Amazon would provide Ogg Vorbis files instead (or in addition to) MP3 files, of course. Anyway, in order to download the MP3 files you buy from Amazon, they suggest to install the Amazon MP3 Downloader, which (surprisingly) is even available in a Mac and Linux version (only 32-bit though), but is (unsurprisingly) closed-source. This is no-go, of course, but luckily there is an alternative. The clamz tool (GPL, version 3 or later) allows you to easily download single Amazon MP3 files, or whole albums. First, you need to login to your Amazon account and then visit a certain Amazon page (which sets a special "congratulations, the Amazon MP3 Downloader has been successfully installed" cookie in your browser). See the clamz website for the respective URL for your country. For Germany, use this URL. The clamz installation is easy enough on Debian:

  $ apt-get install clamz

IMPORTANT: It seems you need at least version 0.5 for recent Amazon files as they apparently changed something, see #647043. Current Debian unstable as of today already has 0.5, though. After that is done, the rest is easy: In Amazon, click on "Buy MP3" or "Buy MP3 album", which will download a special AmazonMP3-1234567890.amz file. You can then let clamz download all the MP3s by typing:

  $ clamz AmazonMP3-1234567890.amz

Wait a few minutes, and you'll have a bunch of non-DRM MP3 files in your current directory. Easy. See the manpage for a bunch of options which let you configure clamz to your preferences.

It's about time that Rapha l is interviewed in the "People behind Debian" series he initiated on his blog. Indeed, when he interviews people, Rapha l asks about other people they could suggest for next interviews. So, during mine, I suggested him to be a next "victim". As he couldn't interview himself, I volunteered for this. As you'll see below, Rapha l (who's a friend of mine as I'm a friend of his) likes to speak and that shows in the length of his answers :-) but you always know more about Debian when reading his blog posts, books, mails, etc. I personnally think that he is among the best promoters of the project for years and it was a pleasure for me to conduct this interview. My questions are in bold, the rest is by Rapha l. Who are you? Hi, Rapha l Hertzog, I'm a 32 years old French Debian developer who is married and who has a 2-year old son. I'm running my own company (Freexian) since 2005, I started it 3 years after the end of my computer science studies. I'm also a very proud author of the Cahier de l'Admin Debian, a French book about Debian. You often wrote about your attempts to make your living partly, if not completely, out of your Debian work. Can you describe the way you're trying to do this? My first try has been with Freexian. I always advertised this company as being specialized in Debian GNU/Linux. While Freexian is successful enough to provide me a decent income, I'm not really satisfied with the result because very few of my contracts are about improving Debian. I use Debian daily for the benefit of my customers, writing new customer specific (embedded) software, deploying a service on Debian servers, etc. But except for the occasional bugfix, all this work does not improve Debian (the only exception has been the dpkg multiarch implementation work sponsored by Linaro). The positive side is that I don't need to fill my entire schedule to earn enough money to live. So I'm regularly taking some days off work to be able to contribute to Debian. This is a freedom that I enjoy... My French book has also been a bestseller and depending on the years the royalties represented between 1 and 2 months of supplementary time that I can spend on Debian (that is between 2000 and 4000 EUR of income). Now since last year, I decided to actively work towards my goal of making a living out of my Debian work. I want to build on what has been most successful for me up to now, that is my book. My strategy has been to build an audience around my blog: with a direct contact with my readers I have the opportunity to sell e-books, and without any intermediary taking the biggest part of the price, I don't need a very large audience to be successful. I have also been experiencing micro-donations with Flattr, people who are enjoying my articles on my blog can use it to give a few cents for each article they find useful. With a large enough userbase, this could fund free documentation and would avoid the need for commercial e-books but we're not there currently and I don't know if it will ever reach the critical mass. Last but not least, I'm soliciting donations for my Debian work on the sidebar of my blog, and I have the chance to a have a few (regular) donators. You're a proud father since last year. How do you manage your commitment to the project with your family life? There are few things that I put above Debian in my life, but my family certainly is. I try to handle most of my Debian duties during work hours so that I can spend time with my family on evenings and during week-ends but in truth I never really disconnect from Debian. It happens quite often that I say to my wife I'll come in a few minutes, let me finish this and then I end up responding to a Debian mail, or an IRC query and take 30 minutes instead of the 5 expected ones. I try hard to avoid this but it's difficult. Luckily for me, my wife is very supportive of my Debian involvement and knows me well... By the way my wife is using Debian on her computer, and my son has already played with DoudouLinux (a Debian derivative!). Have you already been accused of self-promotion in your writings? If that would ever happen, what would you answer to that? Yes, more than once. I am proud of what I do for Debian, I enjoy sharing the result of my work. Because of this, some people believe that I'm selfish and egocentric. And this has somewhat increased since I have been soliciting donations: for me it's important to be transparent towards donators so that they see what I really do for Debian. But some people have the feeling that I'm getting undeserved attention and that I bring everything towards my own person. On the other side, as an author, I'm a public figure who is definitely seeking some attention... I don't have any miraculous answer, we are a large and diverse community, it's next to impossible to please everybody. I listen to all the concerns that people bring forward, I take them into account as much as possible, in particular when I believe they are reasonable/well justified, or when they come from people that I highly respect. But sometimes I have to plainly ignore them too... in particular when they are trying to impose their own political view on a topic that's not directly related to the only value that we all share: the social contract. Contributing to Debian is a challenge, we all have to make efforts to put aside our differences and to concentrate on the work that brings us closer to the best free software operating system ever built. You recently launched a campaign to free out the soon-to-be-published "Debian Administrator Handbook", an English version of your well-known book about Debian in French. Can you tell us more about this project? My French book has been very successful at helping people to get started with Debian, and like I already explained, it was also effective to fund a part of my Debian work. So I wanted to make it available world-wide by publishing an English translation of it. I tried to find an English-speaking editor willing to take on the challenge but I found none interested. Not put off by a setback, Roland (my co-author) and I decided to negotiate with our French editor Eyrolles to recuperate the necessary rights to translate the book into English. Handling everything ourselves represents a lot of work, but it also means that we have the freedom required to decide of the license of the resulting book. We would love to see it under a license compatible with the Debian Free Software Guidelines. But at the same time we firmly believe that we deserve a reasonable monetary compensation for the work on the book, so we conditioned its liberation to a predefined amount of money (25 K ) in what we call the "liberation fund". And since we wanted to be sure that we would have the required means to complete the translation, we used a crowfunding platform to seek support of people interested by the book. With such a platform you're only debited if the minimal requested amount is reached. Anyone can participate, pre-order the book and/or put some money in the liberation fund. As of today, we already reached the minimal funding goal (15 K ) so the book translation will happen. But the liberation target has not yet been reached so we don't know yet if the book will be free from the start... you can follow the progress right on the fundraising page or on the website dedicated to the book. PS: If you want to contribute to this project and also make a donation to Debian at the same time, you should check out this page. You're one of the main developers of dpkg, a critical tool for Debian systems. Can you tell us more about the current development challenges it is facing? What will be the new dpkg features for wheezy? The current challenges are not really technical. dpkg is a relatively mature piece of software and it will continue to work for the foreseeable future without needing much maintenance work. The real challenge is trying to setup a healthy developement community around it so that we can keep tackling new interesting problems (there are many listed in the roadmap and in the 225 wishlist bugs). There is a real problem of leadership and communication in the current team. We used to be three, and we're only two nowadays. Guillem is the legitimate leader since he's involved in dpkg's developement since early 2006 while I joined only in late 2007. But in the last 4 years, we did not manage to recruit anyone else on the team. Some persons tried to contribute significant new features (like Sean Finney with a rework of the way we handle configuration files) but they gave up frustrated after a while because we did not manage to review their work (and discuss the design) in any reasonable timeframe. Another famous case is Ian Jackson with his trigger work. His work got merged, but so late that in the mean time he blew up while trying to hijack the maintenance of dpkg. For a long time I was concentrating my work on the Perl part of dpkg (aka dpkg-dev mainly), so I did not feel qualified to review and merge work related to the C part and I was just a worried observator of this situation. I tried to improve it by setting up some basic review infrastructure, it should have brought some lisibility to the status of each change left to be reviewed... but it has not been used and it changed nothing. Over the years, I became much more interested in the C part. My first big contribution in C has been the rewrite of update-alternatives (from Perl to C). I made other small changes in between, but at the start of this year I had this great opportunity to work on the multiarch implementation (FYI, multiarch is the possibility to mix packages from several architectures on a single system). This really forced me to jump into the C codebase and learn a lot about how dpkg is implemented. Thanks to this I have been able to tackle other small projects (like the improved triggers). This would be all great if my multiarch work was already merged, but it's not. It's a large work, I do not mind waiting a bit in particular since Guillem is a highly skilled C programmer. His sharp analysis of new designs are invaluable, when he reviews code he always finds something to improve. I learnt a lot just by reviewing the code he wrote over the years. That said I have been waiting since April without almost no updates from him. With the release team asking us to hurry up, the situation is getting somewhat strained as I really want to see multiarch in Wheezy and I do not really want to short-circuit Guillem. Hum, I may have drifted a bit from your original question... what great new features can people expect? Well multiarch is supposed to be the big new feature, apart from that there aren't many things that matter to the end users. But there are already quite a few changes that are of interest to package maintainers (like hardened build flags, source package improvements, improved triggers, ). What's the biggest problem of Debian? Manicheism and a tendency to quickly polarize the discussions. In reality, there are very few situations where everything is all good or all bad. Ever since I have read The 7 Habits of Highly Effective People I try hard to put into practice the habits of interdependence . Instead of having only my point of view in mind, I try to understand the motivations from the other party ( Seek First to Understand, Then to be Understood ) in order to be able to put forward solutions acceptable to both parties ( Think Win-Win ). I highly recommend this book to anyone. And I invite everybody to at least try to follow those simple advices. Is there someone in Debian you admire for their contributions? There are many and I can't give an exhaustive list... here are some that I would like to highlight (in no particular order):

• Niels Thykier for the work he puts into Lintian, and his recent work on a britney test-suite (the software that creates testing out of unstable);
• Joachim Breitner for his britney reimplementation using a SAT solver;
• Adam D. Barrat for his work on britney2;
• All the release team members who take care of an endless stream of transitions;
• Christoph Berg for his continuous work over the years on various QA infrastructure (DDPO, Mole, );
• Enrico Zini for the way he helped reform the New Member process without any big disruption;
• Russ Allbery for his Debian Policy work and for being a model in terms of how we should interact on Debian mailing lists;
• Stefano Zacchiroli for being the most active leader that I have ever seen, and for the way he tries to steer important discussions to a constructive conclusion.
• etc.

Most of those people are working on improving Debian's infrastructure so that we can all be more effective and do an even better work. This kind of work is not always very visible but it's crucial to Debian's future. Thank you to Rapha l for the time spent answering my questions. I hope you enjoyed reading his answers as I did. And, anyway, it was fun to just play the game "the other way".

Debian has applied to the Google code-in program as mentoring organization. In the Google code-in, pre-university students (ages 13-17), have the opportunity of contributing to Debian, trying to complete different tasks. We are hoping to get accepted and improve our outcome from last year. To achieve this, we need the help of more Debian contributors proposing and mentoring simple tasks for the students. Please take a take a look at the archives in the mailing list and join us with your proposals.
The list of tasks is at http://wiki.debian.org/GoogleCodeIn2011/Applying, we are specially missing translation and training tasks! You can also join us in the IRC channel #debian-soc (irc.oftc.net)

Revision one in the Subversion repository of the FAI project dates back to the 27th of June 2000. Over the last few months we were discussing the switch to Git. At the last FAI meeting we decided to finally switch to Git and I volunteered to drive the migration. Today I officially did the migration. I ve been using git-svn to handle the FAI svn repository for at least as long as I m wearing the FAI stable release manager hat. But the tags, author information, inside my git-svn checkout weren t ready for official publishing yet There are at least two different well known and referenced svn2git implementations out there. I decided to give the one hosted on github a shot because it s mentioned at http://help.github.com/import-from-subversion/. It basically uses git-svn and just does the few annoying steps for you that I d had to do manually otherwise. For more complex repositories the other svn2git implementation might be worth a try, especially since it s supported by the svneverever tool. svneverever supports more flexible rules regarding the repository layout (it was used for switching Gentoo s Portage from Subversion to Git). Luckily we didn t need it in our situation and I had experience with git-svn on the FAI repository already, so Converting the repository was as simple as running:

% svn2git svn://svn.debian.org/svn/fai --authors ../fai-svnauthors

The ../fai-svnauthors file contains all the svnuser = Realname <mail@example.org> mappings to get the author information right. Some 41 minutes later svn2git finished its work. Finally I pushed the result to the FAI repository at github. If you re interested in the repository size of svn vs. git:

• svn repository (server side): 155MB
• svn checkout (client side): 318.3MB (4.3MB trunk, 301MB tags, 13MB branches)
• git-svn checkout: 13MB
• git repository/checkout: 7.1MB

So, there is some history of organisations doing a poor job at managing security bugs. We saw the This is not really a security hole jokes just to avoid having bad statistics in the front page. We saw the OMFG you must update to the latest version RIGHT NOW and no I m not telling why panic. We still frequently see security fixes hidden in unrelated public commits, just to make them harder to backport for distributors. But really, there is absolutely no match for that. Kudos for setting a new standard in the worse way of dealing with security issues, guys. Update: one of the developers has started insulting a pair of professional IT security experts who came and tried to educate him. Awesome reading, don t forget the popcorn.

See also parts 1, 2, and 3. I am a person that enjoys food that s different from what s at home, and Rhodes didn t disappoint. Terah and I used to live close to an excellent Greek restaurant in Indianapolis, so we were already familiar in some way with the food. But there isn t any Greek restaurant at all in the Wichita area, so we missed it. Our favorite restaurant on Rhodes was Kalypso of Lindos. Everything there was just excellent, starting with the saganaki, one of my favorite Greek appetizers. I had yogurt with honey there for dessert both times we visited, a surprisingly tasty desert. Like many restaurants in Lindos, Kalypso had the option of eating on the rooftop, or at ground level. We ate on the roof, which had a nice view of the Lindos Acropolis. Being outdoors, there were sometimes cats around. This kitten enjoyed playing games with my shoestrings for awhile. Kalypso is at a 17th century captain s house. Here s a view of it from the rooftop: We, of course, had the chance to eat at quite a few different places during our visit, and I d go on way too long if I mentioned them all. Terah particularly enjoyed the gelaterie.gr ice cream shop in the square in Rhodes. We liked our lunch at Maria s Taverna in Lindos and enjoyed chatting with the staff there. I recently talked about shopping in Mexico, and perhaps learned a thing or two from that. I won t say we never buy them, but in general we don t buy souvenirs like t-shirts, plastic things made in China, etc. We prefer to buy local. Those items tend to be higher quality, more interesting, and we like to support the local economy. We also don t have lots of room for things, so we try to choose carefully. So it was something of a surprise to Terah, and perhaps even to me, when I suggested we go shopping one day. Terah typically enjoys shopping a lot more than I do. Anyhow, off we went to Lindos. One of the first things that had caught our eye in Lindos was the shop selling glass. But it wasn t just any glass; it appeared to be made with some sort of layered process, and has a distinctly three-dimensional feel to it. As you move around, it looks like the background shifts. We wound up with this item, which was made in Athens: By the time we visited Lindos specifically for shopping, we had a good feel for when the busy times of day were, so we could avoid them. It gave us the opportunity to visit with people and when they weren t busy, many shopkeepers liked to chat. I enjoy hearing people s stories and we heard several. One ceramics shop the Musa Shop -caught Terah s eye. They had such incredible and beautiful pieces outside that we just had to go in. We wound up with two pieces from there, both in shades of blue: Both remind me of the Aegean Sea and the deep blue sky of Rhodes. And then, as we were walking along, I pointed inside a shop and said to Terah, Hey, those look different. We went in, and eventually wound up buying these: The appearance, and even feel, of them is unlike anything I d seen. Quite interesting. And seeing those particular items in the Lakis Place shop led to making some new friends I ll write about it in the next post.

In last weeks, I've finally managed to create my personal gallery, where I could present selection of my photographs. This task was outstanding on my todo list for pretty long time, the major obstacle being selecting photos which to present there. During my sickness few weeks ago, I finally went through most of my photos and chosen the ones which I think are worth presenting. Still I did not manage to go through photos from half year spent in Japan, so these will be added sometimes later. Also most of the pictures miss descriptions, though I'm (slowly) fixing this. From technical side, the gallery is being generated using lazygal with custom theme. You can check the gallery on photos.cihar.com, I hope you will like it :-).

Filed under: English Photography 1 comments Flattr this!

I care a lot about free network services. Recently, I have been given lots of reasons to be happy with the progress the free software community has made in developing services that live up to my standards. I have personally switched from a few proprietary network services to alternative systems that respect my autonomy and have been very happy both with the freedom I have gained and with the no-longer-rudimentary feature sets that the free tools offer. Although there is plenty left to do, here are four tools I'm using now instead of the proprietary tools that many people use, or that I used to use myself:

• StatusNet/identi.ca for microblogging (instead of Twitter): I have had my account since the almost the very beginning and am very happy with the improvements in the recent 1.0 rollout.
• Diaspora for social networking (instead of Facebook): Diaspora has made important strides forward recently and has become both quite usable and quite useful. Not having used Facebook, I've not managed to totally figure out where the system fits into my life, but I do periodically post updates that are more personal and less polished than the ones on my blog. I still have not set up my own pod but look forward to work that the Diaspora team is putting into making that process easier.
• NewsBlur for feed reading/sharing (instead of Google Reader): NewsBlur can be thought of as a replacement for Google Reader and is, in my opinion, much better even before one considers issues of autonomy. You can install the code yourself or pay the author a small amount to host it for you (he will do it for free if you are following under 64 feeds).
• Scuttle for social bookmarking (instead of Delicious): In the wake of Yahoo's sale and shutdown of Delicious, there is a renewed interest in free tools for social bookmarking. Scuttle, a rather mature project, seems to have been one of several beneficiaries. My Scuttle installation is at links.mako.cc.

In trying to switch away from proprietary services, I have found that there still a lack of good information comparing the different systems out there and giving folks advice on who might be able to help with things like setup or hosting. I really value hearing from other people about what they use and what they find useful but finding this information online still seems to be a struggle. The autonomo.us wiki seems like the natural place to host or summarize this discussion and to collect and share information useful for those of us slouching (or running) toward autonomy in our use of network services. I invite folks to get involved in improving that already useful resource. For example, this week, I spent a few hours researching free social bookmarking tools and produced a major update to the (already useful) social bookmarking page on the autonomo.us wiki. Of course, I can imagine lots of ways to improve that page and to collect similar information on other classes of network services. Please join me in that effort!

In August 2010, Sarah and I had a lovely (but short) week-long holiday in Rhodes, Greece. We thoroughly enjoyed it but I didn't document it here. When John Goerzen was looking for European holiday destinations earlier this year, I had no hesitation in recommending Rhodes. I am surprised and delighted that John took up my suggestion and loved it too! See John's holiday blogging for more on this beautiful place. This year we wanted to take a two-week holiday and have a similar experience again. We ended up choosing Kos, Greece. Kos was, perhaps not surprisingly, quite similar to Rhodes. We were based in "Hotel Oceanis" in Psalidi, on the north-east side, about ten minutes bus journey from Kos town. The hotel was very relaxing with plenty of space. It's based on the sea-side and has two public pools: the large main one and a smaller (but seemingly deeper) one adjacent to a snack bar. We were very lucky to find that the hotel had put is in a room with a private swimming pool! In total, we had four separate places to swim in the hotel. Kos town was similar to Rhodes town, but not quite as "historic" feeling. It was a bit larger/more spread-out. Some areas of the town are quite lively but we always managed to find pleasant and quiet places to eat and drink together. We had twice as much time and took advantage of that by hiring a car and visiting the rest of the island. In particular we loved the beach in Mastichari (middle-North), which had quite turbulent rolling waves. "Sunny beach" on the south coast was absolutely packed with holiday makers, but was none-the-less worth the look. All in all, my first two experiences of Greek islands were wonderful. One day, I'm sure we will return to (at least) Rhodes, but we may leave it for a while and see what the rest of the world has to offer in the mean time.

There are many system administration and configuration management tools available, I've mentioned them in the past and we're probably all familiar with our pet favourites. The "biggies" include CFEngine, Puppet, Chef, BFG2. The "minis" are largely in-house tools, or abuses of existing software such as fabric. My own personal solution manages my home network, and three dedicated servers I pay for in various ways. Currently I've been setting up some configuration "stuff" for a friend and I've elected to manage some of the setup with this system of my own, and I guess I need to decide what I'm going to do going forward. slaughter is well maintained, largely by virtue of not doing too much. The things it does are genuinely useful and entirely sufficient to handle a lot of the common tasks - and because the server-side requirement is a HTTP server, and the only client-side requirement is CRON it is trivial to deploy. In the past I've thought of three alternatives that would make it more complex:

• Stop using HTTP and have a mini-daemon to both serve and schedule.
• Stop using HTTP and use rsync instead.
• Rewrite it in Javascript. (Yes, really).

Each approaches have their appeal. I like the idea of only executing GPG-signed policies, and that would be trivial if there was a real server in place. It could also use SSL because that's all you need for security (ha!). On the other hand using rsync allows me to trivially implement the only missing primitive I actually miss at times - the ability to recursively download and install a remote directory tree. (I solve this problem by downloading a .tar file and unpacking it. Not good. Doesn't cope with template expansion and is fiddlier than I like). In the lifetime of the project I think I've had 20-50 feature requests or comments, which suggests it might actually be used by 50-100 people. (Ha! Optimism) In the meantime I'll keep a careful eye on the number of people who download the tarball & the binary packages... ObQuote: "I have vermin to kill. " - Kill Bill

Some days ago, a hacker group, THC, released a denial of service tool for SSL web servers. As stated in its description, the problem is not really new: a complete SSL handshake implies costly cryptographic computations. There are two different aspects in the presented attack:

• The computation cost of an handshake is more important on the server side than on the client side. The advisory explains that a server will require 15 times the processing power of a client. This means a single average workstation could challenge a multi-core high-end server.
• The use of SSL renegotiation allows to trigger hundreds of handshakes in the same TCP connection. Even a client behind a DSL connection can therefore bomb a server with a lot of renegotiation requests.

Mitigation techniques There is no definitive solution to this attack but there exists some workarounds. Since the DoS tool from THC relies heavily on renegotiation, the most obvious one is to disable this mechanism on the server side but we will explore other possibilities.

Disabling SSL renegotiation Tackling the second problem seems easy: just disable SSL renegotiation. It is hardly needed: a server can trigger a renegotiation to ask a client to present a certificate but a client usually does not have any reason to trigger one. Because of a past vulnerability in SSL renegotiation, recent version of Apache and nginx just forbid it, even when the non-vulnerable version is available. openssl s_client can be used to test if SSL renegotiation is really disabled. Sending R on an empty line trigger renegotiation. Here is an example where renegotiation is disabled (despite being advertised as supported):

$ openssl s_client -connect www.luffy.cx:443 -tls1
[...]
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
[...]
R
RENEGOTIATING
140675659794088:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:591:

Disabling renegotation is not trivial with OpenSSL. As an example, I have pushed a patch to disable renegotiation in stud, the scalable TLS unwrapping daemon.

Rate limiting SSL handshakes Disabling SSL renegotiation on the client side is not always possible. For example, your web server may be too old to propose such an option. Since those renegotiations should not happen often, a workaround is to limit them. When the flaw was first advertised, F5 Networks provided a way to configure such a limitation with an iRule on their load-balancers. We can do something similar with just Netfilter. We can spot most TCP packets triggering such a renegotiation by looking for encrypted TLS handshake record. They may happen in a regular handshake but in this case, they usually are not at the beginning of the TCP payload. There is no field saying if a TLS record is encrypted or not (TLS is stateful for this purpose). Therefore, we have to use some heuristics. If the handshake type is unknown, we assume that this is an encrypted record. Moreover, renegotiation requests are usually encapsulated in a TCP packet flagged with push .

# Access to TCP payload (if not fragmented)
payload="0 >> 22 & 0x3C @ 12 >> 26 & 0x3C @"
iptables -A LIMIT_RENEGOCIATION \
    -p tcp --dport 443 \
    --tcp-flags SYN,FIN,RST,PSH PSH \
    -m u32 \
    --u32 "$payload 0 >> 8 = 0x160300:0x160303 && $payload 2 & 0xFF = 3:10,17:19,21:255" \
    -m hashlimit \
    --hashlimit-above 5/minute --hashlimit-burst 3 \
    --hashlimit-mode srcip --hashlimit-name ssl-reneg \
    -j DROP

The use of u32 match is a bit difficult to read. The manual page gives some insightful examples. $payload allows to seek for the TCP payload. It only works if there is no fragmentation. Then, we check if we have a handshake (0x16) and if we recognise TLS version (0x0300, 0x0301, 0x0302 or 0x0303). At least, we check if the handshake type is not a known value. There is a risk of false positives but since we use hashlimit, we should be safe. This is not a bullet proof solution: TCP fragmentation would allow an attacker to evade detection. Another equivalent solution would be to use CONNMARK to record the fact the initial handshake has been done and forbid any subsequent handshakes. If you happen to disable SSL renegociation, you can still use some Netfilter rule to limit the number of SSL handshakes by limiting the number of TCP connections from one IP:

iptables -A LIMIT_SSL \
    -p tcp --dport 443 \
    --syn -m state --state NEW \
    -m hashlimit \
    --hashlimit-above 120/minute --hashlimit-burst 20 \
    --hashlimit-mode srcip --hashlimit-name ssl-conn \
    -j DROP

Your servers will still be vulnerable to a large botnet but if there is only a handful of source IP, this rule will work just fine¹. I have made all those solutions available in a single file.

Increasing server-side power processing SSL can easily be scaled up and out. Since SSL performance increases linearly with the number of cores, scaling up can be done by throwing in more CPU or more cores per CPU. Adding expensive SSL accelerators would also do the trick. Scaling out is also relatively easy but you should care about SSL session resume.

Putting more work on the client side In their presentation of the denial of service tool, THC explains:

Establishing a secure SSL connection requires 15 more processing power on the server than on the client.

I don t know where this figure comes from. To check it, I built a small tool to measure CPU time of a client and a server doing 1000 handshakes with various parameters (cipher suites and key sizes). The results are summarized on the following plot: For example, with 2048bit RSA certificates and a cipher suite like AES256-SHA, the server needs 6 times more CPU power than the client. However, if we use DHE-RSA-AES256-SHA instead, the server needs 34% less CPU power. The most efficient cipher suite from the server point of view seems to be something like DHE-DSS-AES256-SHA where the server needs half the power of the client. However, you can t really uses those shiny cipher suites:

1. Some browsers do not support them: they are limited to RSA cipher suites².
2. Using them will increase your regular load a lot. Your servers may collapse with just legitimate traffic.
3. They are expensive for some mobile clients: they need more memory, more processing power and will drain battery faster.

Let s dig a bit more on why the server needs more computational power in the case of RSA. Here is a SSL handshake when using a cipher suite like AES256-SHA: When sending the Client Key Exchange message, the client will encrypt TLS version and 46 random bytes with the public key of the certificate sent by the server in its Certificate message. The server will have to decrypt this message with her private key. Those are the two most expensive operations in the handshake. Encryption and decryption are done with RSA (because of the selected cipher suite). To understand why decryption is more expensive than encryption, let me explain how RSA works. First, the server needs a public and a private key. Here are the main steps to generate them:

1. Pick two random distinct prime numbers p and q , each roughly the same size.
2. Compute n=pq . It is the modulus.
3. Compute \varphi(n)=(p-1)(q-1) .
4. Choose an integer e such that 1<e<\varphi(n) and \gcd(\varphi(n),e) = 1 (i.e. e and \varphi(n) are coprime). It is the public exponent.
5. Compute d=e^ -1 \mod\varphi(n) . It is the private key exponent.

The public key is (n,e) while the private key is (n,d) . A message to be encrypted is first turned into an integer m<n (with some appropriate padding). It is then encrypted to a ciphered message c with the public key and should only be decrypted with the private key:

• c=m^e\mod n (encryption)
• m=c^d\mod n (decryption)

So, why is decryption more expensive? In fact, the key pair is not really generated like I said above. Usually, e is a small fixed prime number with a lot of 0, like 17 (0x11) or 65537 (0x10001) and p and q are choosen such that \varphi(n) is coprime with e . This allows encryption to be fast using exponentiation by squaring. On the other hand, its inverse d is a big number with no special property and therefore, exponentiation is more costly and slow. Instead of computing d from e , it is possible to choose d and compute e . We could choose d to be small and coprime with \varphi(n) and then compute e=d^ -1 \mod\varphi(n) and get blazingly fast decryption. Unfortunately, there are two problems with this:

• Most SSL implementation expects e to be a 32bit integer.
• If d is too small (less than one-quarter as many bits as the modulus n ), it can be computed efficiently from the public key.

Therefore, we cannot use a small private exponent. The best we can do is to choose the public exponent to be e =4294967291 (the biggest prime 32bit number and it contains only one 0). However, there is no change as you can see on our comparative plot. To summarize, no real solution here. You need to allow RSA cipher suites and there is no way to improve the computational ratio between the server and the client with such a cipher suite.

Things get worse Shortly after the release the denial of service tool, Eric Rescorla³ published a good analysis on the impact of such a tool. He asks himself about the efficiency to use renegotiation for such an attack:

What you should be asking at this point is whether a computational DoS attack based on renegotiation is any better for the attacker than a computational DoS attack based on multiple connections. The way we measure this is by the ratio of the work the attacker has to do to the work that the server has to do. I ve never seen any actual measurements here (and the THC guys don t present any), but some back of the envelope calculations suggest that the difference is small. If I want to mount the old, multiple connection attack, I need to incur the following costs:

1. Do the TCP handshake (3 packets)
2. Send the SSL/TLS ClientHello (1 packet). This can be a canned message.
3. Send the SSL/TLS ClientKeyExchange, ChangeCipherSpec, Finished messages (1 packet). These can also be canned.

Note that I don t need to parse any SSL/TLS messages from the server, and I don t need to do any cryptography. I m just going to send the server junk anyway, so I can (for instance) send the same bogus ClientKeyExchange and Finished every time. The server can t find out that they are bogus until it s done the expensive part. So, roughly speaking, this attack consists of sending a bunch of canned packets in order to force the server to do one RSA decryption.

I have written a quick proof of concept of such a tool. To avoid any abuse, it will only work if the server supports NULL-MD5 cipher suite. No sane server in the wild will support such a cipher. You need to configure your web server to support it before using this tool. While Eric explains that there is no need to parse any SSL/TLS messages, I have found that if the key exchange message is sent before the server send the answer, the connection will be aborted. Therefore, I quickly parse the server s answer to check if I can continue. Eric also says a bogus key exchange message can be sent since the server will have to decrypt it before discovering it is bogus. I have choosen to build a valid key exchange message during the first handshake (using the certificate presented by the server) and replay it on subsequent handshakes because I think the server may dismiss the message before the computation is complete (for example, if the size does not match the size of the certificate). With such a tool and 2048bit RSA certificate, a server is using 100 times more processing power than the client. Unfortunately, this means that most solutions, except rate limiting, exposed on this page may just be ineffective.

---

1. However, since this rule relies on source IP to identify the attacker, the risk of false positive is real. You can slow down legitimate proxies, networks NATed behind a single IP, mobile users sharing an IP address or people behind a CGN.
2. Cipher suites supported by all browsers are RC4-MD5, RC4-SHA and 3DES-SHA. Support for DHE-DSS-AES256-SHA requires TLS 1.2 (not supported by any browser).
3. Eric is one of the author of several RFC related to TLS. He knows his stuff.

As you will know, every file on a unixoid system has some meta-data like owner, group and permission bits. This is of course also true for files part of some Debian package. And it is not very surprising that different files should have different permissions. Or even different owners or groups. Which file has which settings is of course for the package maintainer to decide and Debian would not be Debian if there were not ways for the user to give their own preferences and have them preserved. This post is thus about how those settings end up in the package and what is to be observed when doing it. As you will also have heard, a .deb file is simply a tar archive stored as part of an ar archive, as you can verify by unpacking a package manually:

$ ar t reprepro_4.8.1-1_amd64.deb
debian-binary
control.tar.gz
data.tar.gz
$ ar p reprepro_4.8.1-1_amd64.deb data.tar.gz   gunzip   tar -tvvf -
drwxr-xr-x root/root         0 2011-10-10 12:05 ./
drwxr-xr-x root/root         0 2011-10-10 12:05 ./etc/
drwxr-xr-x root/root         0 2011-10-10 12:05 ./etc/bash_completion.d/
-rw-r--r-- root/root     19823 2011-10-10 12:05 ./etc/bash_completion.d/reprepro
drwxr-xr-x root/root         0 2011-10-10 12:05 ./usr/
drwxr-xr-x root/root         0 2011-10-10 12:05 ./usr/share/
--More--

(For unpacking stuff from scripts, you should of course use dpkg-deb --fsys-tarfile instead of ar gunzip. Above example is about the format, not a recipe to unpack files). This already explains how the information is usually encoded in the package: A tar file contains that information for each contained file and dpkg is simply using that information. (As tar stores numeric owner and group information, that limits group and owner information to users and groups with fixed numbers, i.e. 0-99. Other cases will be covered later.) The question for the maintainer is now: Where is the information which file has which owner/group/permissions in the .tar inside the .deb, and the answer is simple: It's taken from the files to be put into the .deb. This means that package tools could simply be implemented first by simply calling tar and there is no imminent need to write you own tar generator. It also means that the maintainer has full control and does not have to learn new descriptive languages or tools to change permissions, but can simply put the usually shell commands into debian/rules. There are some disadvantages, though: A normal user cannot change ownership of files and one has to make sure all files have proper permissions and owners. This means that dpkg-deb -b (or the usually used wrapper dh_builddeb) must be run in some context where you could change the file ownership to root first. This means you either need to be root, or at least to fake being root by using fakeroot. (While this could be considered some ugly workaround, it also means upstream's make install is run believing to be root, which also avoids some -- for a packager -- quite annoying automatisms in upstream build scripts assuming a package is not installed system wide if not installed as root). Another problem are random build host characteristics changing how files are created in the directory later given to dpkg-deb -b. For example an umask which might make all files non-world-readable by default. The usual workaround is to first fix up all those permissions. Most packages use dh_fixperms for this, which also sets executable bits according to some simple rules and has some more special cases so that the overall majority of packages does not need to look at permissions at all. So using some debhelper setup, every special permissions and all owner/group information for owner groups with fixed numbers only needs to be set using the normal command line tools between dh_fixperms and dh_builddeb. Everything else happens automatically. Note that games is a group with fixed gid. So it is not necessary (and usually a bug) to change group-ownership of files withing the package to group games in maintainer scripts (postinst,...). If a user wants to change permissions or ownership of a file, dpkg allows this using the dpkg-statoverride command. This command essentially manages a list of files to get special treatment and ownership and permission information they should get. This way a user can specify that files should have different permissions and this setting is applied if a new version of this file is installed by dpkg. Being a user setting especially means, that packages (that means their maintainer scripts) should not usually use dpkg-statoverride. There are two exceptions, though: Different permissions based on interaction with the user (e.g. asking question with debconf) and dynamically allocated users/groups with dynamic id. In both cases one should note that settings to dpkg-statoverride are settings of the user, so the same care should be given as to files in /etc, especially one should never override something the user has set in there. (I can think of no example where calling dpkg-statoverride --add without dpkg-statoverride --list in some maintainer script is not a serious bug: Either you override user settings or you are using debconf as a registry. Moral To recap, your package is doing something wrong if:

• Your maintainer scripts unconditionally change permissions of files within the package (Those should either be set within the package or you miss user configuration in dpkg-statoverride).
• Your maintainer scripts call dpkg-statoverride --add without having checked first whether the user already has done so with different values.

Many people around me switched to Chrome or Chromium. I also used it for a bit, but I was a bit disappointed about the extensions available. To show why, here's a list of the extensions I've currently installed:

• Adblock Plus: I guess everyone knows this. It sits quitely in the background and removes quite a bunch of eye-distracting stuff from web pages. You see the web differently and are always confused when viewing pages on your smartphone.
• British English Dictionary: Pretty self-explaining. Firefox has an integrated spell checker and that one needs a dictionary.
• BugMeNot: You can right-click on a login field and let it insert BugMeNot data automatically. I don't use it often enough to remember that it's there.
• Certificate Patrol: SSL's X.509 trust model is weak, to say the least. This extension implements the "save on first visit" trust model and warns you if the certificate or the CA of a URL changes.
• Download Statusbar: If you're used from, say, Chromium to see your download progress above the status bar, this extension will give you that. I don't like the separate window, especially with Awesome as my window manager.
• Firebug: Most people know this, too. Very useful when you do web development. You can see the effects of CSS as you type, for instance.
• Flashblock: YouTube has its HTML5 trial, so you don't need Flash for it. Sadly I had to give in recently (for live streaming sites like the German Parliament, go figure) and have it installed for "emergency" cases. But really nobody needs Flash advertisements or other silly Flash animations, so Flashblock will conveniently refuse to load the Flash plugin unless you tell it to.
• German Dictionary: As above.
• Google Translator for Firefox: Translates marked sections on a web page in place. It lets Google guess the source language, so you really only mark the area, click the button and be done.
• Greasemonkey: Modifies web pages in place to make them more sane, using little scripts. I use two currently:
• Better Outlook Web Access (OWA): For odd reasons I'm forced to use OWA 2007 at a company of Windows and Mac users. This makes it a little bit more bearable, by allowing you to save your password and by adding a message preview pane. You really don't want to use OWA Light 2007 without this.
• Tagesschau.de - video tag: A simple script that lets you watch videos on tagesschau.de without resorting to Flash. They ship Theora videos alongside H.264, which is supported by Firefox out of the box.
• Lazarus: Form Recovery: This already saved me many, many times. Sometimes I hit the stupid Thinkpad page back/page forward keys, or the browser crashes or I'm torn away from a page in another way. This extension keeps your form content mostly save, so that you don't need to start from scratch. A must have.
• Live HTTP headers: curl -D usually works. But sometimes you need authentication and cookies and stuff, hence doing it in the browser makes sort of sense.
• Modify Headers: I don't really use it anymore. There was this legend that YouTube lets you watch videos that are blocked in your country (Hello, GEMA, I'm looking at you!) if you provide a X-Forwarded-For header with an IP from another country. Never worked for me.
• Mozilla Archive Format: If you need to archive a web page or want to collect a bunch of pages for offline reading, this is the way to go. You can conveniently select either single pages or several tabs to be stored.
• Perspectives: I'm paranoid, so here's the second SSL certificate check. Perspectives uses a bunch of network notaries hosted on PlanetLab to check if everyone sees the same certificate. Together with Certificate Patrol that means if that you save a self-signed certificate on first visit if the notaries all agree that it's the currently installed one.
• PwdHash: Password re-use on different web sites is bad. There are sites where I'm not very concerned about the strength of my password, but where I don't want to leak my main ones. PwdHash uses the domain name and the password I give as components to a hash function. So if I'm at a computer without access to my saved passwords, I can easily reconstruct the hash. If need be, I can use the JavaScript on the developer's web site to do that.
• RequestPolicy: It's currently installed but I don't think it will stick around much longer. Too many sites are using embedded cross-site requests, which this extension will allow you to review. As an example every Blogger site that's on a custom domain (like my blog) will trigger it. Instead half of your web will have red flags everywhere full of blocked requests. Not that helpful. Interestingly enough it also polices Flash's outgoing web requests. Which breaks even more often than normal web pages.
• Secure Login: This gives you a button (or a keyboard shortcut) that allows you to choose from several logins and then goes on and posts those credentials securely to the page in question, bypassing any JavaScript that might want to see them. You also get auto-login bookmarks for those sites which don't allow you to remain logged-in across browser sessions. (Like OWA.)

If Firefox on Android were quicker to start and faster overall, I might even use it there. But as-is it's not very useful. Sadly this also means that I can't use Firefox Sync on my phone and as I don't use Chrome on my desktop I also can't use Chrome to Phone. So I usually go and build a QR code on my laptop and read that with Android's Barcode Scanner.

Of course I'm actually using Iceweasel and I'm very grateful for Mike Hommey's efforts to track the release channel on mozilla.debian.net.

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (130.30 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg work The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

• #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
• #642656: dpkg-source s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) 2.0 source format.
• #644412: the Makefile snippet buildflags.mk did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell ). Fixed that by manually exporting the required variables.
• I also disabled dpkg-buildpackage s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix. Multiarch progress I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu s dpkg. The first one (LP #863675) was rather severe since an installed package ended being disappeared in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures. But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far. While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already. Python-django security update and RC bug Since python-django s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates. Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear. Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced. In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer WordPress packaging WordPress sounds similar to python-django. I m also only a backup maintainer but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it. I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of opportunistic replacement with symlinks . WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic. The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It s somewhat of a pain since it s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker. Dropbox packaging work A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.). I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream. The Debian Administrator s Handbook Update I m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters. Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book. With (less than) 27 days left, it s going to be a challenge to meet the goal, but we do like challenges, don t we? Misc work

• I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
• I merged lots of patches from Luca Falavigna in the developers-reference.
• I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
• I updated publican s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
• I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

Rage is the latest game by Id Software, a company whose prior work I have an immense fondness for. I picked it up at the same time as buying a refurbished Playstation 3 a few weeks ago. Sadly, I'm not too impressed. The graphics are very pretty, most of the time, but the PS3 suffers the same problem which has plagued the PC version, namely, texture pop-up: A noticeable delay for a low-resolution texture to be swapped in for a high-resolution one. I'm only about 45 minutes in but my experience mirrors that of Rock Paper Shotgun's: It's desperately trying to disguise it's own linearity. Finally, and most crucially, I'm suffering majorly from motion sickness after more than a few minutes of play. This might be related to the fact that Rage's framerate is apparently double that of most games at 60 frames per second. Whatever the explanation, I can't play it if it makes me sick to do so! I may trade it in for the similarly-themed Fallout: New Vegas, which I held off buying simply because I thought two post-apocalyptic FPS games was too many. I loved Fallout 3 (albeit on Xbox 360), logging perhaps 50+ hours of play with plenty more to do; but I don't think I could return to the beginning of that game.

Volker Theile is well known for his work on FreeNAS. But at one point, he decided to give up on it and to restart from scratch but this time based on Debian (instead of FreeBSD). OpenMediaVault is the name of this new Debian based product. Here s how he defines the project:

OpenMediaVault is the next generation network attached storage (NAS) solution based on Debian Linux. It contains services like SSH, (S)FTP, SMB/CIFS, DAAP media server, RSync, BitTorrent client and many more. Thanks to the modular design of the framework it can be enhanced via plugins. OpenMediaVault is primarily designed to be used in home environments or small home offices, but is not limited to those scenarios. It is a simple and easy to use out-of-the-box solution that will allow everyone to install and administrate a Network Attached Storage without deeper knowledge.

Even though all the work was private, he managed to attract an important following, and I must say that I was looking forward to this project. He regularly blogged on his progress, sharing some good-looking video of the resulting product (example here). The first public release (Version 0.2, codenamed Ix) happened on October 17th. I have yet to try it but I took a look on the website. As a Debian developer, I was keen on seeing the source code and how the project was managed. The GPLv3 license is presented as an important feature and I was expecting a well-managed open source project. The fact that it was a private one-man project up to now did not bother me, we re quite used to the scratch your itch kind of start for free software projects. Enough to say is that I have been very disappointed. First you come across a contributor agreement, it s frowned upon by many free software developers. But why not, maybe he bought the argument of Mark Shuttleworth and wants to give it a try. But then I looked at the subversion repository, it s obvious that it s just a dumping ground of files that are managed somewhere else in another repository. A bit like Android which is not developed in the open but released as a whole from time to time. But the worst was yet to be found on the licensing page:

Beside being freely available for personal end-users, System Builders and System Integrators, in general Installers, require a commercial license for OpenMediaVault.

Besides the fact that I m still not sure what this means, I would like to know how he reconciles this requirement with the terms of the GPLv3. It s a shame that a so promising project ends up being a disaster from a free software perspective. Have you tried OpenMediaVault already? If yes, I d be glad to read your thoughts in the comments.

9 comments Liked this article? Click here. My blog is Flattr-enabled.

Ron has written an interesting blog post about the US as a lottery economy [1]. Most people won t win the lottery (literally or metaphorically) so they remain destined for poverty. Tim Connors wrote an informative summary of the issues relating to traffic light timing and pedestrians/cyclists [2]. I have walked between Southgate and the Crown Casino area many times and have experienced the problem he describes many times. Scientific American has an interesting article about a new global marketplace for scientific research [3]. The concept is that instead of buying a wide range of research equipment (and hiring people to run it) you can outsource non-core research for a lower cost. Svante P bo gave an interesting TED talk about his work analysing human DNA to determine prehistoric human migration patterns [4]. Among other things he determined that 2.5% of the DNA from modern people outside Africa came from the Neandertals. Lisa wrote an informative article about Emotional Support Animals (as opposed to Service Animals such as guide dogs) for disabled people [5]. It seems that the US law is quite similar to Australian law in that reasonable accommodations have to be made for disabled people which includes allowing pets in rental properties even if such pets aren t officially ESAs. Beyond Zero Emissions has an interesting article about electricity prices which explains how wind power forces prices down [6]. This should offset the new carbon tax . Problogger has an article listing some of the ways that infographics can be used on the web [7]. This can be for blog posts or just for your personal understanding. Petter Reinholdtsen wrote a handy post about ripping DVDs which also explains how to do it when the DVD has errors [8], I haven t yet ripped a DVD but this one is worth noting for when I do. Miriam has written about the Fantastic Park ICT training for 8-12yo kids [9]. It s run in Spain (and all the links are in Spanish but Google Translation works well) and is a camp to teach children about computers and robotics using Lego Wedo among other things. We need to have more of these things in other countries. The Atlantic Cities has an interesting article comparing grid and cul-de-sac based urban designs [10]. Apparently the cul-de-sac design forces an increase in car use and therefore an increase in fatal accidents while also decreasing the health benefits of walking. Having lived in both grid and cul-de-sac based urban areas I have personally experienced the benefits of the grid based layout. Sarah Chayes wrote an interesting LA Times article about governments being taken over by corruption [11]. She argues that arbitrary criminal government leads to an increase in religious fundamentelism. Michael Lewis has an insightful article in Vanity Fair about the bankruptcy of US states and cities [12]. Ben Goldacre gave an interesting TED talk about bad medical science [13]. He starts with the quackery that is published in tabloid newspapers and then moves on to deliberate scientific fraud by medical companies. Geoff Mulgan gave an interesting TED talk about the Studio Schools in the UK which are based around group project work [14]. The main thing I took from this is that the best method of teaching varies by subject and by student. So instead of having a monolithic education department controlling everything we should have schools aimed at particular career paths and learning methods. Sophos has an interesting article about the motion sensors of smart phones being used to transcribe keyboard input based on vibration [15]. This attack could be launched by convincing a target to install a trojan application on their phone. It s probably best to regard your phone with suspicion nowadays. Simon Josefsson wrote a good article explaining how to use a GPG smart-card to authenticate ssh sessions with particular reference to running backups over ssh [16]. C ran wrote a good article explaining how to use all the screen space when playing DVDs on a wide screen display with mplayer [17]. Charles Stross has an informative blog post about Wall St Journal circulation fraud [18]. Apparently the WSJ was faking readership numbers to get more money from advertisers, this should lead to law suits and more problems for Rupert Murdoch. Is everything associated with Wall St corrupt?

• [1] http://rondam.blogspot.com/2011/08/lottery-economy.html
• [2] http://tau-iota-mu-c.livejournal.com/172226.html
• [3] http://www.scientificamerican.com/article.cfm?id=an-ebay-for-science
• [4] http://www.ted.com/talks/svante_paeaebo_dna_clues_to_our_inner_neanderthal.html
• [5] http://chaoticidealism.livejournal.com/99347.html
• [6] http://beyondzeroemissions.org/blog/merit-wind-energy-110913
• [7] http://www.problogger.net/archives/2011/09/16/blogosphere-trends-using-infographics/
• [8] http://people.skolelinux.org/pere/blog/Ripping_problematic_DVDs_using_dvdbackup_and_genisoimage.html
• [9] http://www.miriamruiz.es/weblog/?p=654
• [10] http://www.theatlanticcities.com/design/2011/09/street-grids/124/
• [11] http://www.latimes.com/news/opinion/commentary/la-oe-chayes-corruption-20110925,0,517490.story
• [12] http://www.vanityfair.com/business/features/2011/11/michael-lewis-201111
• [13] http://www.ted.com/talks/ben_goldacre_battling_bad_science.html
• [14] http://www.ted.com/talks/geoff_mulgan_a_short_intro_to_the_studio_school.html
• [15] http://nakedsecurity.sophos.com/2011/10/20/iphone-spyware-snoop-desktop-typing/
• [16] http://blog.josefsson.org/2011/10/11/unattended-ssh-with-smartcard/
• [17] http://www.carbon-project.org/MPlayer__and_no_black_bars.html
• [18] http://www.antipope.org/charlie/blog-static/2011/10/going-down-hard.html

Saturday started well, reading two stimulating articles on Planet_Debian. One about GitTogether2011, and the other about the use of Git in the Baserock company. I like a lot the idea of using Git beyond source code management. This article is itself distributed via a network of Git repositories. For the Debian package I develop and that are maintained with Git, I started some time ago to store their build logs in a branch that I called meta. Since sbuild can filter some variable chains in its logs, the command git diff --word-diff=color became a nice tool to inspect the difference between two package builds. Here is for instance the update I made this morning for bedtools: we can see that -D_FORTIFY_SOURCE=2 is not passed anymore, and that consequently the -Wunused-result warnings disappeared. Another advantages of storing build logs is simply to make available information that was not previously: buildd.debian.org contains the build logs for every architecture except the one used by the package maintainer, often one of the most popular architectures, because the uploads combine source and binary packages. This problem will be solved when our archive will automatically rebuild the binary packages uploaded with the source packages, but it will not be entirely solved as it is still possible to upload binary packages alone. In the meta branch of my Git repositories, the logs are kept side-to-side for every architecture. I am unsure if that the best layout, but for the moment I am uncomfortable with having too many parallel branches. I am starting to automate the work with the logs, for instance with that small script to gather them from buildd.debian.org.

#!/bin/bash
BASEURL=buildd.debian.org:/srv/buildd.debian.org/db
PACKAGE=$1
shift
for version in "$*"
do
  scp $BASEURL/$ PACKAGE:0:1 /$PACKAGE/$ version /* .
  for arch in $(ls *bz2   sed 's/_.*//g')
    do bzcat $ arch *bz2 > $ arch .log
    rm $ arch _*_log.bz2;
  done
done
exit