During Debconf11, I've tried a couple of times to eat Cevapi, but for one reason or another I didn't manage to. On my way from Banja Luka to Split, I've stopped my motorbike in the proverbial middle of nowhere for lunch (the fact it was raining cats & dogs helped in not having much of a choice). As a consequence, the waiter I ended up facing didn't speak a single word of any language I know. The only three words of Bosnian I managed to say were luckily the right ones: evapi, pivo, and voda. The only missing ingredient for a perfect DebConf in the Balkans evapi has hence been acquired before leaving the country. Everything else was already there: intense hacking, truckload of emotions, tiny teeny bit of stress, feeling of family, and physical exhaustion. No sign of DebConf flu (yet). In case you missed a specific event or the conference all together, hail the Videoteam for having already made available the videos on video.d.n. They can provide useful entertaining for the forthcoming weeks. For what is worth, I plan to spend the next two weeks elsewhere in the Balkans with very little Internet (as in: none at all). Talk to you later.

There are certain times when I m really glad that we have Linux on the house for our boys to play with. I ve already written how our 4-year-old Jacob has fun with bash and can chain together commands to draw ASCII animated steam locomotives. Today I thought it might be fun to install cw, a program that can take text on standard input and play it on the console speaker or sound card as Morse code. Just the sort of thing that I could see Jacob eventually getting a kick out of. But his PC was mute. We opened it up and discovered it didn t have a console speaker. So we traipsed downstairs, dug out an external speaker, and I figured out how to enable the on-board audio chipset in the BIOS. So now the cw command worked, but also there were a lot of other possibilities. We also brought up a microphone. While Jacob was busy with other things, I set to work getting things hooked up, volume levels adjusted, and wrote some shell scripts for him. I also printed out this reference sheet for Jacob: He is good at reading but not so good at spelling. I intentionally didn t write down what the commands do, hoping that this would provide some avenue for exploration for him. He already is generally familiar with the ones under the quiet category. I wrote a shell script called record . It simply records from the microphone and drops a timestamped WAV file in a holding directory. He can then type play to simply play back whatever he recorded most recently. Easy enough. But what he really wanted was sound for his ASCII steam locomotive. So with the help of a Google search for steam train mp3 , I wrote a script ssl (sound steam locomotive) that starts playing the sound in the background if it isn t already going, and then runs sl to show the animation. This was a big hit. I also set it up so he can type play train to hear that audio, or play song to play our favorite train song (Always a Train in My Dreams by Steve Gillette). Jacob typed that in and sat still for the entire 3 minutes listening to it. I had to hook up an Ethernet cable to his machine to do all this, and he was very interested that I was hooking his computer up to mine in some way. He thought all the stuff about cables in the walls was quite exciting. The last thing I did was install flite, a speech synthesis program. I wrote a small shell script called talk which reads a line at a time from stdin and invokes flite for each one (to give more instant feedback rather than not starting playback until after having read a large block from stdin). He had some fun hearing it say his name and other favorite words, but predictably the most fun was when he typed gibberish at it, and heard it try to pronounce or spell nonsense words. In all, he was so excited about this new world of computer sound opened up to him. I m sure there will be lots of happy experimentation and discovery going on.

In the past week, I ran a lot of benchmarks using a Spirent Avalanche which is an appliance providing performance testing of network related products (a load balancer, a router, a web server, ). The reporting module does not provide a lot of flexibility and the plots are not the most beautiful ones. Fortunately, the results are also exported as CSV. matplotlib is a python plotting library which produces great figures without making things hard when they should be easy. It is a good replacement for GNU Plot and you don t need a lot of Python knowledge to use it. The documentation includes a fine user s guide that should get you started in less than 20 minutes.

Quick introduction You can use matplotlib from IPython to experiment:

$ ipython -pylab
Python 2.6.7 (r267:88850, Jul 10 2011, 08:11:54) 
Type "copyright", "credits" or "license" for more information.
  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.
> In [1]: plot([1,1,4,5,10,11])

When you are ready, you can build a simple Python script:

#!/usr/bin/env python
from matplotlib.pylab import *
plot([1,1,4,5,10,11])
savefig("my-plot.pdf")

Grabbing results from CSV Data are contained into a file named realtime.csv. It contains the raw data as well as some general information about the benchmark (test name, description, paramaters ). We need to skip those. Fortunately, csv2rec() allows to load data from a CSV file into a record array and skip the first rows if necessary.

from matplotlib.pylab import *
import sys, os
import gzip
skip = 0
for line in gzip.open(sys.argv[1]):
    if line.startswith("Seconds Elapsed,"):
        break
    skip = skip + 1
ava = csv2rec(gzip.open(sys.argv[1]),skiprows=skip)

Now, the Elapsed seconds column can be accessed with ava['elapsed_seconds'].

General structure I need 4 plots:

• successful, unsuccesful and attempted transactions per second,
• minimum, average and maximum response time per page,
• Avalanche CPU usage,
• incoming and outgoing bandwidth.

The most important plot is the first one. The second one is less important and the last ones are here only to check we did not hit some bottleneck during the benchmark. We want to produce a page like this: matplotlib allows to plot subfigures. We create 4 subfigures sharing the same X axis (which the number of seconds elapsed since the beginning of the benchmark). We save the result to PDF.

# Create the figure (A4 format)
figure(num=None, figsize=(8.27, 11.69), dpi=100)
ax1 = subplot2grid((4, 2), (0, 0), rowspan=2, colspan=2)
# [...]
ax2 = subplot2grid((4, 2), (2, 0), colspan=2, sharex=ax1)
# [...]
ax3 = subplot2grid((4, 2), (3, 0), sharex=ax1)
# [...]
ax4 = subplot2grid((4, 2), (3, 1), sharex=ax1)
# [...]
# Save to PDF
savefig("%s.pdf" % TITLE)

Bandwidth plot Let s start with the easiest plot: bandwidth usage.

# Plot 4: Bandwidth
ax4 = subplot2grid((4, 2), (3, 1), sharex=ax1)
plot(ava['seconds_elapsed'], ava['incoming_traffic_kbps']/1000.,
     'b-', label='Incoming traffic')
plot(ava['seconds_elapsed'], -ava['outgoing_traffic_kbps']/1000.,
     'r-', label='Outgoing traffic')
grid(True, which="both", linestyle="dotted")
ylabel("Mbps", fontsize=7)
xticks(fontsize=7)
yticks(fontsize=7)

Here, in the suplot positioned in (3,1), we plot the number of seconds elapsed versus the incoming traffic with a blue line (b-). We also plot the seconds elapsed versus the outgoing traffic with a red line (r-). Here is the result: Most functions of matplotlib are exposed as a method of the object they refer to and as a global function. In the latest case, the function is applied to the latest created figure or plotting area. For example, plot() is called a function and therefore refer to the plotting area ax4. We could have written ax4.plot() instead.

CPU plot The average CPU utilization data available in the CSV file needs to be normalized. We assume the Avalanche to be mostly idle on start. The plotting part is pretty similar to our previous case.

# CPU
max = np.max(ava['average_cpu_utilization'])
order = 10**np.floor(np.log10(max))
max = np.ceil(max/order)*order
cpu = (max - ava['average_cpu_utilization'])*100/max
# Plot 3: CPU
ax3 = subplot2grid((4, 2), (3, 0), sharex=ax1)
plot(ava['seconds_elapsed'],
     cpu,
     'r-', label="Avalanche CPU")
grid(True, which="both", linestyle="dotted")
ylabel("Avalanche CPU%", fontsize=7)
xticks(fontsize=7)
yticks(fontsize=7)

Response time We would like to plot response time. We have three corresponding metrics: minimum response time, average response time, maximum response time. Because those metrics start from 0 ms up to several seconds, a logarithmic scale is used:

# Plot 2: response time
ax2 = subplot2grid((4, 2), (2, 0), colspan=2, sharex=ax1)
plot(ava['seconds_elapsed'], ava['minimum_response_time_per_page_msec'],
     'b-', label="Minimum response time")
plot(ava['seconds_elapsed'], ava['maximum_response_time_per_page_msec'],
     'r-', label="Maximum response time")
plot(ava['seconds_elapsed'], ava['average_response_time_per_page_msec'],
     'g-', linewidth=2, label="Average response time")
legend(loc='upper left', fancybox=True, shadow=True, prop=dict(size=8))
grid(True, which="major", linestyle="dotted")
yscale("log")
ylabel("Response time (msec)", fontsize=9)
xticks(fontsize=9)
yticks(fontsize=9)

This is also the first graphic with a legend.

Transactions per second The most important plot is the number of transactions per second.

# Plot 1: TPS
ax1 = subplot2grid((4, 2), (0, 0), rowspan=2, colspan=2)
plot(ava['seconds_elapsed'], ava['desired_load_transactionssec'],
     '-', color='0.7', label="Desired Load")
plot(ava['seconds_elapsed'], ava['successful_transactionssecond'],
     'g:', label="Successful")
plot(ava['seconds_elapsed'], smooth(ava['successful_transactionssecond']),
     'g-', linewidth=2)
plot(ava['seconds_elapsed'], ava['attempted_transactionssecond'],
     'b-', label="Attempted")
plot(ava['seconds_elapsed'], ava['aborted_transactionssecond'],
     'k-', label="Aborted")
plot(ava['seconds_elapsed'][:-1], ava['unsuccessful_transactionssecond'][:-1],
     'r-', label="Unsuccessful")
legend(loc='upper left', fancybox=True, shadow=True, prop=dict(size=10))
grid(True, which="both", linestyle="dotted")
ylabel("Transactions/s")

The number of successful transactions is plotted twice: when the benchmarked equipment becomes overloaded, we get a lot of noise in this metric and it can be difficult to read. Therefore, we plot a smoothed version with the help of NumPy:

import numpy as np
def smooth(x, win=4):
    s = np.r_[x[win-1:0:-1],x,x[-1:-win:-1]]
    w = np.ones(win, 'd')
    y = np.convolve(w/w.sum(),s,mode='valid')
    return y[(win-1)/2:-(win-1)/2]

np.r_() is just here to extend our data by the size of the window. np.ones() build a weight vector of the size of the window. If the window is 4, we get [0.25, 0.25, 0.25, 0.25]. We use this vector to apply a convolution to the original data. Here is the result: The original data is a green dotted line while the smoothed one is a green thick line. What about the three annotations? matplotlib allows to put annotations on a figure. Here is how this is done:

# Noticeable points
count = 0
def highlight(index, reason):
    global count
    if index and index > 0:
        x,y = (ava['seconds_elapsed'][index],
               smooth(ava['successful_transactionssecond'])[index])
        plot([x], [y], 'ko')
        annotate('%d TPS\n(%s)' % (y,reason), xy=(x,y),
                 xytext=(20, -(count+4.7)*22), textcoords='axes points',
                 arrowprops=dict(arrowstyle="-",
                 connectionstyle="angle,angleA=0,angleB=80,rad=10"),
                 horizontalalignment='left',
                 verticalalignment='bottom',
                 fontsize=8)
        count = count + 1
highlight(np.argmax(smooth(ava['successful_transactionssecond'])), "Max TPS")
highlight(np.argmax(cpu > 99), "CPU>99%")
highlight(np.argmax(ava['average_response_time_per_page_msec'] > 500), ">500ms")
highlight(np.argmax(ava['average_response_time_per_page_msec'] > 100), ">100ms")

np.argmax() returns the index of the first maximum value. The trick here is that when I write ava['average_response_time_per_page_msec'] > 100, I get an array with 1 when the value is more than 100 and 0 otherwise. Therefore, np.argmax() will return the first index where the value is superior to 100 ms. The highlight() function will add a point (plot([x], [y], 'ko')) on the smoothed successful transactions par second plot and add an annotation with some fancy arrow. Look at this benchmark of nginx as SSL termination for a complete output of this script.

NEW YORK, Jul 29, 2011 The crowd of thousands gathered here to attend PreventaConf 2011 were abuzz over the unveiling of the culmination of 3 years of aimless arsing about^ww^w concentrated development which has led to the porting of dpkg to the MINIX 3 operating system. Women fainted, men wept openly, medical personnel were kept busy dealing with the many instances of heart seizure, multisensory hallucination and spontaneous combustion which accompanied this momentous event. Much remains to be done.

• start-stop-daemon needs to be ported. It is not built for now.
• A basic set of essential packages needs to be created. Currently all that stuff lives in /usr/local or /usr/pkg. This has already caused problems when e.g. dpkg-architecture assumed perl was in /usr/bin.
• fakeroot needs to be ported so you don't have to build .debs as root which is currently the case.
• Other support programs such as update-rc.d and ldconfig etc. are needed. Currently .debs have to be installed with --force-all to get around their lack. (ldconfig atleast could just be a noop as Minix doesn't actually support shared libraries yet.)
• There are still some API deficiencies. lchown(2) doesn't exist, getpwent(3) doesn't seem to work reliably. etc. Could this cause problems? We will have to investigate and provide workarounds if necessary. No doubt other problems will crop up as work progresses.

Still this is a major milestone of which the Debian Minix developers are justifiably proud.

While at Debconf11, I have several times during discussions mentioned the issues I believe should be improved in Debian for its desktop to be useful for more people. The use case for this is my parents, which are currently running Kubuntu which solve the issues. I suspect these four missing features are not very hard to implement. After all, they are present in Ubuntu, so if we wanted to do this in Debian we would have a source.

1. Simple GUI based upgrade of packages. When there are new packages available for upgrades, a icon in the KDE status bar indicate this, and clicking on it will activate the simple upgrade tool to handle it. I have no problem guiding both of my parents through the process over the phone. If a kernel reboot is required, this too is indicated by the status bars and the upgrade tool. Last time I checked, nothing with the same features was working in KDE in Debian.
2. Simple handling of missing Firefox browser plugins. When the browser encounter a MIME type it do not currently have a handler for, it will ask the user if the system should search for a package that would add support for this MIME type, and if the user say yes, the APT sources will be searched for packages advertising the MIME type in their control file (visible in the Packages file in the APT archive). If one or more packages are found, it is a simple click of the mouse to add support for the missing mime type. If the package require the user to accept some non-free license, this is explained to the user. The entire process make it more clear to the user why something do not work in the browser, and make the chances higher for the user to blame the web page authors and not the browser for any missing features.
3. Simple handling of missing multimedia codec/format handlers. When the media players encounter a format or codec it is not supporting, a dialog pop up asking the user if the system should search for a package that would add support for it. This happen with things like MP3, Windows Media or H.264. The selection and installation procedure is very similar to the Firefox browser plugin handling. This is as far as I know implemented using a gstreamer hook. The end result is that the user easily get access to the codecs that are present from the APT archives available, while explaining more on why a given format is unsupported by Ubuntu.
4. Better browser handling of some MIME types. When displaying a text/plain file in my Debian browser, it will propose to start emacs to show it. If I remember correctly, when doing the same in Kunbutu it show the file as a text file in the browser. At least I know Opera will show text files within the browser. I much prefer the latter behaviour.

There are other nice features as well, like the simplified suite upgrader, but given that I am the one mostly doing the dist-upgrade, it do not matter much. I really hope we could get these features in place for the next Debian release. It would require the coordinated effort of several maintainers, but would make the end user experience a lot better.

Due to high demand I have prepared official Redis packages for Debian "squeeze":

1. Add the following line to /etc/apt/sources.list:

   deb http://backports.debian.org/debian-backports squeeze-backports main
   

2. Update your local package lists:

   sudo apt-get update
   

3. Install redis-server with:

   sudo apt-get -t squeeze-backports install redis-server
   

The current version number in this repository may be found here. Bugs should be reported in the usual way. For the sections of the web community who insist on hitting problems solved years ago, here is why you should use these packages over installing Redis manually:

• Zero-effort installation: Starts Redis immediately, no compilation or configuration required.
• Improved security: Runs as a dedicated user/group and data files are accessible only to this user.
• Production-ready initscript: All these versions can cause data loss. Server can also be restarted with a standard "/etc/init.d/redis-server restart" instead of manual dance with pidfiles.
• Sensible default locations for data files (/var/lib/redis), logs (/var/log/redis) and utilities (/usr/bin/redis-cli, etc.). Logs are automatically rotated and compressed using logrotate. The log verbosity is additionally reduced from verbose to notice - it is unlikely you care about every new connection.
• Sensible upgrade path: Upgrades to newer versions result in less downtime, handle conflicting changes in configuration options, and can be quickly rolled back in case of incompatibility. The package can also be purged entirely from the system in seconds.
• You don't make your sysadmin cry. If you are the sysadmin and are installing software with a variant of git pull && make && make install you deserve every problem you run into.

After being dropped out of Debian, the m68k was stalled from some time now. There was no real upgrade path and so my machines still are running etch-m68k. Thanks to Thorsten Glaser the port is slowly keeping up with NPTL now ported to kernel and glibc for m68k. He took care to port and compile a lots of packages that are needed for upgrading from etch-m68k. Big thanks for that to Thorsten! Anyway, I'm in the progress of upgrading my m68k machines and buildds with the help and tips from Thorsten and this is the way I'm doing this:

1. Change your /etc/apt/sources.list to include this:
   
   deb http://ftp.debian-ports.org/debian/ unstable main contrib
   deb http://archive.debian.org/debian etch-m68k main contrib non-free
   
2. Get libuuid-perl from snapshot.debian.org:
   
   wget http://snapshot.debian.org/archive/debian/20070128T000000Z/pool/main/libu/libuuid-perl/libuuid-perl_0.02-1_m68k.deb
   dpkg -i libuuid-perl_0.02-1_m68k.deb
   
3. Get kernel & linux-base from unstable
   You need to install a recent kernel like linux-image-2.6.39-2-amiga in my case. Either download it by hand or use apt:
   
   apt-get -d install linux-image-2.6.39-2-amiga linux-base
   cd /var/cache/apt/archive
   dpkg --force-depends -i linux-image-2.6.39-2-amiga_2.6.39-3_m68k.deb linux-base_3.3_all.deb
   
4. If needed, remove linux-bases postinst, when you get this kind of error:
   
   syntax error at /var/lib/dpkg/info/linux-base.postinst line 1275, near "# UUIDs under /dev"
   Can't use global $_ in "my" at /var/lib/dpkg/info/linux-base.postinst line 1289, near " $_"
   rm /var/lib/dpkg/info/linux-base.postinst
   dpkg --configure --pending --force-depends
   
   
5. If everything installed fine you should be ready to boot into your new kernel.
   On Amigas you mostly likely need to edit your boot script and copy your kernel and System.map to your AmigaOS partition. This is how my boot looks like:
   
   amiboot-5.6 -k vmlinux-2.6.39 "debug=mem root=/dev/sda4 video=pal-lace devtmpfs.mount=1"
   
   You can omit the debug=mem. This is just for the case that the kernel crashes. You can then collect the dmesg output under AmigaOS with the dmesg tool. The other parameter devtmpfs.mount=1 is needed because we don't want udev. Using video=pal-lace is necessary because the 2.6.39 kernel crashes on initializing my PicassoII graphics card and I've unplugged the card for the time being necessary to solve the problem.
   
6. Kernel 2.6.39 runs fine, but you can't ssh into machine.
   Because we don't want udevd, there's now a problem when trying to login by SSH:
   
   pty allocation request failed on channel 0
   stdin is not a tty
   
   You can fix this either by installing udev, which most websites recommend when you're looking for this error, because on Xen this is the recommended solution, but as we are on m68k and not under Xen, it's better to run with a static /dev. So you need to create /dev/pts and add the following to your /etc/fstab:
   
   mkdir /dev/pts
   devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
   
7. After kernel boots into 2.6.39 you can dist-upgrade to unstable.
   When you successfully booted into your new kernel, you should be save to proceed with upgrading to unstable. For that you should first let the missing or broken depends from linux-image and linux-base being installed:
   
   apt-get -f install
   
   This should lead to install some dependencies that were missing, because using dpkg --force-depends from above. After that I upgraded dpkg, apt and apt-utils:
   
   apt-get install dpkg apt apt-utils
   
   When this succeeded, you should be save to fully dist-upgrade to unstable:
   
   apt-get -u dist-upgrade
   
   When you get errors during apt-get dist-upgrade, you might run dpkg --configure --pending or apt-get -f install, before proceeding with apt-get -u dist-upgrade. Another problem can occur with apt. When you see this error:
   
   E: Could not perform immediate configuration on 'perl-modules'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
   
   you should add "-o APT::Immediate-Configure=false" to your apt-get command, for example:
   
   apt-get -o APT::Immediate-Configure=false -f install

As stated above, my PicassoII in my A3000 doesn't seem to work under 2.6.39, whereas the PicassoIV in my A4000T does not crash the kernel. Please don't hesitate to add additions, corrections or other kind of feedback by commenting below! P.S.:
Wouter and Thorsten are currently at Debconf in Banja Luka, working on the m68k port. Wouter just finished a first version of a new debian-installer image. He asks for testing it on real hardware. Please volunteer if you can! It's available at: http://people.debian.org/~wouter/di-m68k/

After being dropped out of Debian, the m68k was stalled from some time now. There was no real upgrade path and so my machines still are running etch-m68k. Thanks to Thorsten Glaser the port is slowly keeping up with NPTL now ported to kernel and glibc for m68k. He took care to port and compile a lots of packages that are needed for upgrading from etch-m68k. Big thanks for that to Thorsten! Anyway, I'm in the progress of upgrading my m68k machines and buildds with the help and tips from Thorsten and this is the way I'm doing this:

1. Change your /etc/apt/sources.list to include this:
   
   deb http://ftp.debian-ports.org/debian/ unstable main contrib
   deb http://archive.debian.org/debian etch-m68k main contrib non-free
   
2. Get libuuid-perl from snapshot.debian.org:
   
   wget http://snapshot.debian.org/archive/debian/20070128T000000Z/pool/main/libu/libuuid-perl/libuuid-perl_0.02-1_m68k.deb
   dpkg -i libuuid-perl_0.02-1_m68k.deb
   
3. Get kernel & linux-base from unstable
   You need to install a recent kernel like linux-image-2.6.39-2-amiga in my case. Either download it by hand or use apt:
   
   apt-get -d install linux-image-2.6.39-2-amiga linux-base
   cd /var/cache/apt/archive
   dpkg --force-depends -i linux-image-2.6.39-2-amiga_2.6.39-3_m68k.deb linux-base_3.3_all.deb
   
4. If needed, remove linux-bases postinst, when you get this kind of error:
   
   syntax error at /var/lib/dpkg/info/linux-base.postinst line 1275, near "# UUIDs under /dev"
   Can't use global $_ in "my" at /var/lib/dpkg/info/linux-base.postinst line 1289, near " $_"
   rm /var/lib/dpkg/info/linux-base.postinst
   dpkg --configure --pending --force-depends
   
   
5. If everything installed fine you should be ready to boot into your new kernel.
   On Amigas you mostly likely need to edit your boot script and copy your kernel and System.map to your AmigaOS partition. This is how my boot looks like:
   
   amiboot-5.6 -k vmlinux-2.6.39 "debug=mem root=/dev/sda4 video=pal-lace devtmpfs.mount=1"
   
   You can omit the debug=mem. This is just for the case that the kernel crashes. You can then collect the dmesg output under AmigaOS with the dmesg tool. The other parameter devtmpfs.mount=1 is needed because we don't want udev. Using video=pal-lace is necessary because the 2.6.39 kernel crashes on initializing my PicassoII graphics card and I've unplugged the card for the time being necessary to solve the problem.
   
6. Kernel 2.6.39 runs fine, but you can't ssh into machine.
   Because we don't want udevd, there's now a problem when trying to login by SSH:
   
   pty allocation request failed on channel 0
   stdin is not a tty
   
   You can fix this either by installing udev, which most websites recommend when you're looking for this error, because on Xen this is the recommended solution, but as we are on m68k and not under Xen, it's better to run with a static /dev. So you need to create /dev/pts and add the following to your /etc/fstab:
   
   mkdir /dev/pts
   devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
   
7. After kernel boots into 2.6.39 you can dist-upgrade to unstable.
   When you successfully booted into your new kernel, you should be save to proceed with upgrading to unstable. For that you should first let the missing or broken depends from linux-image and linux-base being installed:
   
   apt-get -f install
   
   This should lead to install some dependencies that were missing, because using dpkg --force-depends from above. After that I upgraded dpkg, apt and apt-utils:
   
   apt-get install dpkg apt apt-utils
   
   When this succeeded, you should be save to fully dist-upgrade to unstable:
   
   apt-get -u dist-upgrade
   
   When you get errors during apt-get dist-upgrade, you might run dpkg --configure --pending or apt-get -f install, before proceeding with apt-get -u dist-upgrade. Another problem can occur with apt. When you see this error:
   
   E: Could not perform immediate configuration on 'perl-modules'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
   
   you should add "-o APT::Immediate-Configure=false" to your apt-get command, for example:
   
   apt-get -o APT::Immediate-Configure=false -f install
   
   Another pitfall might be exim4-daemon-heavy, which currently segfaults. Replace it by exim4-daemon-light in that case, which works.

As stated above, my PicassoII in my A3000 doesn't seem to work under 2.6.39, whereas the PicassoIV in my A4000T does not crash the kernel. Please don't hesitate to add additions, corrections or other kind of feedback by commenting below! P.S.:
Wouter and Thorsten are currently at Debconf in Banja Luka, working on the m68k port. Wouter just finished a first version of a new debian-installer image. He asks for testing it on real hardware. Please volunteer if you can! It's available at: http://people.debian.org/~wouter/di-m68k/

It is half-time for me at DebCamp/DebConf11 in Banja Luka, Bosnia-Herzegovina, and that is a good time to write about my work so far. I started with some finger exercises, such as the HTML copying for gnome-terminal (as blogged previously) and cosmetic changes to the buildd.debian.org/status website. After that, I began my main project: A SAT-based solver for the testing transition problem. In Debian, new software first enters a repository called unstable, where it is tested. Once some requirements are fulfilled (package has been in unstable for usually at least 10 days and has no new release critical bugs), it is entitled to enter the repository called testing, which will eventually form a new stable Debian release. But some packages need to migrate together, usually because either requires the new version of the other. Also, no packages in testing ought to become uninstallable by some seemingly unrelated change. The software that decides these things is called britney. It is sufficiently good in making sure nothing bad happens, but not smart enough to figure out what to do in case more than two packages need to migrate simultaneously. My goal is to improve this. Now, the the various requirements can all be expressed as predicative formulas, and a lot of research has been going into writing good solvers for such problems, called SAT-solvers. Hence my plan was to only specify what we expect from a testing migration, but leave the search for a solution to such a general purpose and highly optimized program. My initial progress was good and I had some result after two days of hacking, and the approach is promising. Because the data sets are pretty large (1.5GB of input data, the final SAT problem has 1.8 million clauses with 250.000 variables), so I learned quite a bit about profiling and optimizing Haskell programs, and that parsec is slower than working with BS.lines and BS.split to parse simple input. I also used the FFI to use dpkg code directly to compare Debian version numbers. At some point I noticed that I actually want to solve PMAX-SAT problems: Given two set of clauses, hard clauses and soft clauses, find a variable assignment that fulfills the hard clauses and as many of the soft clauses as possible. Unfortunately, there are no fast Free PMAX-SAT solvers around. The ones that I found and that were fast enough that I could use them, msuncore by Joao Marques-Silva at the University College Dublin and MiniMaxSat by Federico Heras at the Universitat Polit cnica de Catalunya, are only available as statically linked binaries. It is a shame that this is acceptable by the academic community; imagine mathematicians would stop including the proofs in their papers, and only share their theorems. So if you happen to have written a PMAX-SAT solver and can solve this instance (in weighted DIMACS format) in less than five minutes, and want to brag that the Debian project is using your code, then please release it under a Free license (e.g. GPL or BSD) and tell me about it! The code of my SAT-based solver is, of course, available, though slightly unpolished. Parts of it (notably the interface to picosat and the PMAX-SAT solvers) might be of general interest and I plan to put them on Hackage as a separate license. Update: I have contacted the authors of the SAT solvers mentioned above, and they have reconfirmed that they have no intention of releasing the source. Now I put my hope in maxsatz2009 by Chumin LI, which is GPL but was not able to cope with my large instance directly; I think I need to change the memory management.

Flattr this post

Jacob has been eagerly awaiting this day for at least, well, 2 days now. This morning he started asking questions. All day long he has been wondering how long until the big event. And that event is the arrival of the UPS truck carrying a special $8 package for him. And in that $8 package is not a DVD, or a book, or a toy, but rather 12 rolls of adding machine tape (totaling 1/3 of a mile). Oh yes, our four-year-old can hardly contain his excitement over office supplies. Let me back up a bit to explain. I am always on the lookout for a certain kind of toy. The sort of thing that you don t find in the toy section of a store. The sort of thing that is interesting and engages the curiosity of our boys, that most people would consider would toss, and that usually costs less than $5. Along those lines, Jacob has an old manual typewriter he can use, he and I built a switchbox full of switches and light bulbs that the boys love to play with, we built him a command-line-only PC out of spare parts, the boys have some radios they can use, and one day I salvaged a decades-old adding machine that was going to be thrown out. Jacob liked it he enjoys number and letter games and had some fun, but didn t zealously engage with it until just a few days ago. He sat down at his table and started making cards . He will print out a bunch of numbers, making a strip a couple of feet long, and then say, Dad, I made another card for you! We will then happily accept the card and he will continue making more. Here he is, holding up one of his cards: Jacob had a ton of fun making them, and Oliver wanted in on the action too. Oliver was somewhat interested in the adding machine, but not as much as Jacob, and Jacob didn t want to share. So somehow they decided that Oliver would run and get cards from Jacob and then take them to whatever parent Jacob said. Today Jacob said, Dad, the Oliver delivery service will bring you a card! Saturday morning we had some time to hang around and play, so we played some adding machine tape games. I tucked some long cards under my waistband, and the boys happily chased me all over the house trying to get them back. We also decorated: And here is what the room looked like after some cleanup: That orange bucket on the desk is my card bucket where I can put all the cards he gives me. He even helpfully empties it out into the recycle bin when there are too many old cards in it! But then tragedy struck it was out of paper! I ordered a 12-roll pack off Amazon, and boy was it a long couple of days until it arrived. Today before the UPS truck got here, I showed him how we could feed old paper through and print on the back side an incredible innovation, it seemed! Terah had a conversation with him earlier that went like this: Terah: Jacob, we need to clean up the house. Jacob: Why? Terah: Because some people are coming over. Jacob: Who is OH YES! The UPS delivery man! Yes, let s clean! Terah: Ah, well yes he will be here too, but your great uncle and aunt are coming. Jacob: Oh yay! And the UPS man! It is hard to describe Jacob s excitement when the package arrived. I quickly opened it up, and at his great uncle s prompting, he counted to see if he really did get 12 rolls of paper. IT IS TWELVE ROLLS! YAY! And then we quickly installed it in the machine and off he went. At his great uncle s prompting, he started writing a J on each card . This meant Jacob , or maybe he d put two J s, meaning To John From Jacob , or perhaps the two J s meant To Terah From Jacob he d always have to clue us in. Anyhow, for an $8 investment in paper this has brought hours of joy to our boys. Even though Jacob isn t yet really using it to perform arithmetic, it is setting him up to enjoy that sort of thing and encouraging his curiosity. Both good things.

I ve actually spent a few hours searching around for a good backup solution for my mailbox until I decided to stick with getmail. What you ll be able to achieve after reading this HowTo and deploying the following setup is:

1. A full backup of your e-mail DATA in the Mbox format. (yes, Gmail s labels / folders as well)
2. Prevent getmail to mark all mails as read after delivering them. (this was a pretty bad issue since getmail was marking all my mails as read even if I did not access my e-mail at all)
3. Keep your backups up-to-date with the latest content from your mailbox. (by default getmail grabs all the DATA from your mailbox and fills up the Mbox / Maildir content keeping deleted mails. So let s say I deleted a mail two days ago, well it ll still appear on today s backups. This behaviour is definitely unwanted)

I ll now move to explain a few details about my new configuration but before moving to tweak getmail s main config file, please do the following change on _retrieverbases.py* :

return self._getmsgpartbyid(msgid, (RFC822) ) to return self._getmsgpartbyid(msgid, (BODY.PEEK[]) )

When done grab the following getmailrc and adapt it to your needs**:

[retriever]
type = SimpleIMAPSSLRetriever ## or SimplePOP3SSLRetriever.
server = imap.gmail.com ## or pop.gmail.com for POP3.
username = example@gmail.com
password = password ## so-called Gmail s labels should be listed one by one here for getmail to retrieve mail from them successfully. mailboxes = ( INBOX , [Gmail]/Sent mail ,
ubuntu , gnome/example , linux/example ) [destination]
type = Mboxrd
path = ~/.getmail/backup.mbox [options]
delivered_to = false ## No delivered_to header added automatically.
received = false ## No received header added automatically.
verbose = 2 ## getmail will print messages about each of its actions.

When done we should go ahead setting up getmail s directories and config file:

mkdir $HOME/.getmail cp $HOME/getmailrc $HOME/.getmail/ ## Adapt $HOME/getmailrc to whatever dir you put that file into.

But pretty much all the remaining work will be done by a small shell script I wrote:

#!/bin/sh WORKDIR=$HOME/.getmail
date= date +%d-%m-%Y_%H:%M if [ ! -f $WORKDIR/backup.mbox ]
then
touch $WORKDIR/backup.mbox
fi getmail > $WORKDIR/getmail.log
OUT=$?
if [ $OUT -eq 0 ]
then
mkdir -p $WORKDIR/backups/ && mv $WORKDIR/backup.mbox $WORKDIR/backups/backup_$date.mbox ;
else [ $OUT -eq 1 ]
exit 1
fi ## Cleanup older than 3 days backups
find $WORKDIR/backups/* -mtime +3 -exec rm ;
cd $WORKDIR && rm -rf oldmail-* ;

This script will:

1. Run getmail using the getmailrc config file you previously worked on.
2. If the above command will be successful, it ll create a backups dir into $HOME/.getmail and move the latest Mbox file there appending a date and time to its name. (by doing this we are sure next getmail run will happen on an empty backup.mbox file, thus it will just contain the latest content from your mailbox)
3. It ll re-create a backup.mbox file on $HOME/.getmail to avoid the next getmail run to fail.
4. In the end, it ll clean up older than 3 days backups to avoid a too crowded backups folder. (it removes the oldmail file as well since it is useless in our case)

In the end set up a cronjob that will run the above script and generate the backups for you every one hour:

0 * * * * $HOME/.getmail/getmail_run.sh > /dev/null

Feel free to let me know if you ve encountered any issue while following the above HowTo. Enjoy! * /usr/share/getmail4/getmailcore/_retrieverbases.py on line 901. ** More documentation about the getmailrc file and syntax can be found on getmail s documentation page.

If you don't have any interest in tedious licensing discussion then I recommend not reading this. It's not going to be a great deal of fun.

Having said that:

I'm looking at the Canonical Individual Contributor License Agreement (pdf here). In contrast to the previous copyright assignment, it merely grants a broad set of rights to Canonical, including the right to relicense the work under any license they choose. Notably, it does not transfer copyright to Canonical. The contributor retains copyright.

So here's a thought experiment. Canonical release a project under the GPLv3. I produce a significant modification to it. It's now clearly a derived work of Canonical's project, so I have to distribute it under the GPLv3. I have no right to distribute it under a proprietary license. I sign the CLA and provide my modification to Canonical. The grant I give them includes giving them permission to relicense my work under any license they choose. As copyright holders to the original work, they may also change the license of that work. But, notably, I have not granted them copyright to my work. I continue to hold that.

Now someone else decides to extend the functionality that I added. By doing so they are creating a work that's both a derivative work of Canonical's code and also a derivative work of my code. How can they sign the CLA? I granted Canonical the right to grant extra permissions on my work. I didn't grant anyone else that right.

This wasn't a problem with the copyright assignment case, because as copyright holders Canonical could simply grant that permission to all downstream recipients. But Canonical aren't the copyright holder, and unless they explicitly relicense my work I don't see any way that they can accept derivatives of it. The only way I can see this working is if all Canonical code is actually distributed under an implicit license that's slightly more permissive than the GPLv3. But there's nothing saying that it is at present.

Now, I'm obviously not a lawyer. I may be entirely wrong about the above. But asymmetric CLAs introduce an additional level of complexity into the entire process of contributing that make it even more difficult for a potential contributor to become involved. I've spent far more time than most worrying about licenses and even I don't understand exactly what I'm giving up, which is ironic given that a stated aim is usually that they increase certainty about licensing. Is the opportunity to relicense really worth alienating people who would otherwise be doing free work for you?

If a single man can display so much hatred
think only of how much love we all can display together.

Stine Renate H heim

I wrote a Norwegian post explaining my experience at Ut ya. I had taken this blog for dead, and had entirely forgotten that it was syndicated on Planet Debian. I don t want to let Google Translate make this disaster any worse than it is the translation of bullets into balls being particularly bad so the international attention the massacre has garnered in consideration, I am writing an English translation of my experiences. I feel somehow duty-bound to make people aware of what happened, but I don t want to get into anything else but a sober description of the events and some very brief reflections. There are many details I have chosen to omit. Others have written their experiences of the events at Ut ya. I wanted to write mine down as well, and get it out there . Partly, I want to write this down because I m unsure if I will remember all the details at a later point in time, although I think I d prefer it if I couldn t. I m also writing this because people are asking about my experiences and it s much better to have an URL to give them, lest I have to keep going through the same spiel over and over again. Our former Prime Minister and current labour movement demigod Gro Harlem Brundtland had recently left the island. I had been the cameraman for a video interview of her talking about Ut ya, and I was in the media group room encoding the video into a file suitable for YouTube, when someone else in the room startled and said that Twitter was full of messages about a loud explosion in Oslo. As the newspapers brought us information about the extent of the damages, a consensus arose that an informational meeting was in order. As soon as the current round of talks finished, we were gathered into the main hall. The meeting was duly held, and after the statement was made that a TV feed would be made available, I took it upon myself as the local alpha geek to make it happen. Of course, the situation caused both the wireless network and the GPRS networks to become totally unusable. As I was waiting for someone to set up a password, I took the opportunity to face the consequences of having eaten two bits of a microwavable dish called Hold-It the local equivalent of a Hot Pocket and went to the toilet. As I was in there, I first heard agitated shouting, then screams, then gunshots coming from just outside the toilets. More than anything else, it sounded like a toy gun. I was convinced that someone was making a joke in incredibly bad taste and I stormed out of the booth with the intent of halting it. As I tore the door open, I saw two of my comrades hiding in a recessed corner. Their facial expressions left absolutely no doubt that this was no toy. They signalled for me to get back in the booth. I closed the door, did a mental double-take in utter, complete confusion, and opened it again. They were still signalling. Had they not stood there, I would have run straight into the gunman; they saved my life. I looked out into the hallway, and I made eye contact with a young boy lying in a pool of blood. He was motioning for me to help him. I heard more gunshots from inside the building and retreated back inside. As I was trying to think through my next move, I realized that the decidedly insubstantial wood-fiber door would not resist any kind of bullets. I made my way out into the hallway, with the intent of escaping outside. At that point, I was of course not aware that there was an intention to kill as many as possible, so I thought that the open spaces outside would be a place of relative safety. Of course, this proved to be wrong and my life was probably saved a second time by one of the caf volunteers taking me into a hard-to-spot employee s bathroom. We sat there for ninety minutes. Always ready to make a run for it, ready for just about anything. A peculiar group dynamic arose with these two people with whom I had barely previously spoken. We came to share a strange sense of common destiny and gallows humour. One of them had seen the shooter and described the police uniform. I perceived it to be realistic that we were the only ones aware of the wounded outside the toilet. I tried to reach the emergency services, but all their lines were busy; the terror attack in Oslo had probably clogged their lines. I finally got through to the fire services, who could inform me that the police did know about the situation and were on their way. This was to take 90 minutes and by the time we evacuated, the young boy outside my door had perished. The despair I first saw in his eyes as I passed him, fleeing from one room to the other and the empty, blank stare as we left, are burned into me and they are images I will never in my life forget. Finally, the real police arrived. We walked out. I chose the path through the minor conference hall something I now regret. The sight was simply beyond my capacity to describe fully, and so terrifying that I barely remember the sight only the terror it struck in me. There were several people bunched up in a corner, a big amorphous heap of bodies. Some were conscious and yelled at me not to do anything that could startle the police, others lay still. Their bodies were all covered in blood, and a thick pool of blood extended at least a half-metre in all directions around them. The policeman across the hall was screaming orders at me, but he was screaming so loudly that I couldn t make out his words at first. We were first moved into the camp newspaper s offices. There were about eight of us there, I think, in addition to one girl who lay wounded. Towards the end she was drifting in and out of consciousness. We covered her with sweaters to keep her warm and one of us tried to at least temper her bleeding. The bullet had missed her heart, but by the entry wound it was clear that it was not by far. I do not know who this girl was or how she is now. I sat behind and never saw her face. The wounded were evacuated first. I don t remember how long we remained; I had lost all concept of time. In spite of protests from the group who knew him, one kid was put in handcuffs. At the time I didn t understand why, and the policeman seemed to say something almost to the effect that there was no reason for it at all. I didn t see when they undid his cuffs, but I remember thinking that this treatment made a terrible experience even worse for him. I tried my best to comfort him but knew it would be little help. Later, when things stabilised a little, we were told that he was handcuffed because he had come from an unsecured area. The police was extremely good at carefully explaining what was happening and why; this was a big help and I am grateful for it. Eventually we were moved out into the main corridor of the building, where we joined up into a group of about fifty. Only when I saw the two people who saved my life did any emotion other than mild confusion arise. I broke down shivering in tears in one of their arms. After a few seconds, I came back to my senses and realised that this was not the time. I quickly gathered myself, got the shaking under control, and sat down. We were given some chocolate and soda from the kiosk. I remember making an offhand remark that an inability to find joy in free candy was a sure sign of a bad situation. We all laughed out loud. Gallows humour is a coping mechanism, but in retrospect one almost feels guilty for it. We were shown out in a single file with hands above our heads. I remember an intense concern that someone would slip in the steep, muddy slope and create a misunderstanding. Outside, there were more bodies. Some under improvised covers a tarpaulin from the waffle stand, the deflated bouncy castle but some simply lay there. Everyone I met displayed a courage, a mental discipline and unity of purpose far beyond anything one would ever wish to expect from people this young. Everyone conducted themselves with an attitude that could almost be described as stiff upper lip . Safely across the fjord we were offered blankets. I was asked if I was aware of any injuries, and asked to lift my shirt and show my abdominal region. We were shown into the bus which took us to the hotel used by the survivors and their family. I simply cannot describe in any words the relief I felt when I was able to embrace my living comrades. It was completely unlike anything I had ever felt before in my life. The euphoric feeling was tempered only by the realisation that there would be many I could never see again, comrades whom I had taken great pride in calling my friends, with futures in the service of all mankind, futures I had previously found such great joy in pondering and guessing about. The feeling which continues to upset me the most, is the feeling that so many of my comrades left behind grieving families and friends. Torn away senselessly. I do not know how much more than this relatively sober account of the events on Ut ya I can muster. I would, however, like to offer some reflections. First of all, from the bottom of my heart, I want to thank the police who saved the lives of so many still on the island, the holidaymakers who took aboard swimmers into their boats and the rescue services staffed primarily by volunteers who have spared no effort in trying to soften the blow as much as they can. The opportunity to spend time with those comrades who underwent the same experience as myself has also been an immeasurable aid. I was also so relieved to find my very closest friend among the survivors, which has also been an indescribable help. If I can name a single positive in this tragedy: Had he arrived with his automatic weapon fifteen or twenty minutes prior, he would have arrived during the informational meeting, at a time when the major hall was absolutely jam-packed the death toll would be many times what it ended up being. I am agonisingly aware of the meager comfort this provides to those who have been bereft of their closest, but I do find some solace in this. We cannot sweep under a rug that this was without question! a political attack on the labour movement. But it is thankfully also an attack which has been perceived by everyone as an attack on the Norwegian society, and on a symbol of the wide recruitment to the participatory democracy which lies at our very national soul. I cannot thank the Norwegian people, and indeed the people in other nations who have offered their condolences, enough for their shows of support and shared grief. It really has been a tremendous help to me knowing that so many people feel with us. I also want to thank from the bottom of my heart the rock-steadiness of everyone in both the national and local wings of the Labour Party and Labour Youth in supporting us, and the political milieu in general for their resolute steadfastness saving me from losing yet more that I cherish; our freedoms in a participatory democracy. Our Party has lost many of its very brightest youngsters. Personally I feel an angry spite, a deep restless urge to get the wheels of society going again. I want to show his kind that we will not be broken. We re stronger than that. I will not be frightened into silence and passivity. I want to remember the dead, and honor them by carrying on our common work. I want to end this with a request to everyone who reads this, echoing a statement I read by one of my good friends and comrades: Please, don t let me see any messages of hatred, wishes for the death penalty, anything like that. If anyone should be of the belief that anything will improve by murdering this sad little person, they would be profoundly wrong. All attention now should be plowed into caring for those victims and their relatives who did not share my luck, and not giving an audience to a perpetrator who wants one. Tore Sinding Bekkedal

The trip down from Gratz to Banja Luka was much easier than the day before. After a while you just get used to being sat in a car for ages. Plenty of nice scenery to enjoy through Slovenia. After a while our car's GPS's began to fail, showing us driving through fields, and we were stuck for 1.5 hours in a traffic jam when the 4 lane highway seemed to end. Got around that with some guesswork, and on into Croatia by back roads. The Bosnian border was an interesting experience, all the guards could say in English was "green card! green card!" -- which from an American POV is an unsettling thing to be asked for at a border, especially if they've already taken your passport away -- but at least we were not detained overnight. While drivers were away getting the car insurance settled it descended toward farce as we had to hand roll the cars forward to let trucks get into the country. (Or we thought we did.. one was rolled with the keys in it as it turned out.) Arrived at the Hotel in Banja Luka in the middle of a wedding, which was amazingly loud (I could still hear it from the 5th floor at 2 am). There's also a casino at the hotel, so first impression was garish and loud! ... But now that it's a rainy Sunday, seems much nicer here.

why proprietary relicensing is unacceptable for a Free Software enthusiast The past few months have witnessed a large debate about copyright assignment, which seems to has neared its peak around the time Project Harmony has released version 1.0 of its agreement templates. (Note: I use the expression "copyright assignment" in this post, but I'd happily include into it both actual Copyright Assignment Agreements CCAs and Contributor License Agreements CLAs, as both could be used to permit various forms of proprietary relicensing.) Mark Shuttleworth has been one of the most prominent voices campaigning in favor of copyright assignments (to for-profit companies). A couple of interesting blog posts of his where this is evident are "On balancing economic power in the FLOSS ecosystem" and "The responsibilities of ownership". Voices against the opportunity of increasing the popularity of copyright assignments are aplenty and it is pointless to repeat good arguments which I find fully convincing. If you are interested in the subject, here are a few good reads I recommend:

• "Some thougts on copyright assignment", seminal essay (circa end of 2009) by Michael Meeks that provides a good coverage of copyright assignment basics and effects on the surrounding ecosystem
• Richard Fontana's "The trouble with Harmony"; in particular, I recommend taking a close look at "The illusion of constraint" part

What I would like to add to the debate is the point of view of a Free Software enthusiast, not unlike yours truly. I contribute to Free Software not because it is superior to proprietary software in either its software products or in its development methodology. (It actually is superior in many cases, but that's just not my primary motivation.) I do contribute to Free Software to build a world in which users are in complete control of the software they run. Let's assume we have a broken toaster at home. We are all free to (1) open it up and try to repair it by ourselves. If we don't have the needed knowledge, we are free to (2) hand it over to a person that we trust and who does have the needed knowledge to repair it. That failing, we can (3) ask the toaster producer to repair it for us. Many people like me simply don't accept to live in a world where, once all the toasters will be software-driven, the only repair option will be (3), just because the toaster software is proprietary. (Given various kind of [broken] metaphors have been used to explain copyright assignments, it seemed sensible to propose my broken toaster metaphor in exchange.) When I contribute to some Free Software project, I not only care about being able myself to get a Free Software licensed copy of the whole product (i.e. the original software product + my own contribution). I care that all recipients of the product, present and future, will be able to get that contribution with enough freedoms attached to put them in control of the software that includes it. This desire can be fulfilled with a varying degree of success, depending on the license of the code base I contribute to. For instance, I find more pleasant to contribute to a copyleft licensed code base than to a weak-/non-copyleft licensed one, and more pleasant to contribute to an AGPL-ed web app than to a copyleft web app. It also means that, given the option, I will not allow the code base maintainer to re-license my contribution under terms that allow distributing it without suitable freedoms attached. Different contributors will be ready to accept different re-licensing promises, but the mere fact we are debating all this and that companies like Canonical feel the need of campaigning for copyright assignments is evidence that many contributors like me exists and actually have a problem with allowing downstream relicensing of their contributions. What is feared the most is of course proprietary relicensing. According to my moral horizon this is the case not because the company might end up gaining money out of my code: I wouldn't have a problem with that. The reason is rather that the company will be able to distribute my own code in a way that grants less freedoms to users than those that were implicitly "agreed upon" when the contribution has been written. In that respect, I can't help thinking at harmony agreements as a large scale smoke screen experiment meant to convince volunteers that copyright assignments are something Free Software needs. By exploiting the creative-commons-like "it's easy, just choose one" scheme, Harmony takes the chance of offering misleading options such as the one labeled "any OSI approved license". Via that option, which is seemingly fine, the copyright recipient will be allowed to relicense the contribution under some non-copyleft license (e.g. BSD) and from there do essentially what they want. All in all, companies campaigning for copyright assignments seem to have failed to understand two main points. The first one is that contributors motivated by Free Software ideals want the code to remain free not only for themselves, but for all recipients of the code. I'm not claiming that all contributors to FOSS projects belong to this category, but the fact that there is a considerable amount of such people is undeniable. Hoping that they will be fooled by tricks like the "any OSI approved license" agreement is foolish. The second is that while volunteer Free Software communities do respect companies that provide huge amounts of code contributions, they also expect companies to figure out by themselves which business models make their businesses sustainable. Telling volunteers that current business models are not sustainable without proprietary relicensing is worthless. Volunteers just don't care. They want software freedom guarantees and are not willing to renounce to them in exchange of the wealth of companies, not even under the (implicit) threat that otherwise those companies will be forced to reduce the amount of their code contributions.

---

Update: explain in the initial note why my reasoning here encompasses both CCAs and CLAs. Thanks Tollef for noticing that it wasn't clear.

There was an interesting issue in D-Bus, related to unicast vs broadcast signals, which led [edit: typo] to a small change in specs and which might be of some interest for D-Bus developers.

Unicast signals are not widely known and probably even less used, but they are possible.
They are useful, for instance, when you need to trigger an action from a single client, among your listeners.

Until some days ago, when a unicast signal was emitted, it was actually received by everyone listening to the signal's interface (unless a strict rule was added, unusual), waking up a number of processes which actually weren't interested in the signal.
Collateral effect, waking up cost apart: those processes might actually consider the signal as they were the actual recipient and take some action upon it. Bad.

Typical rule having this problem is "sender="org.foo,interface=org.bar".
Imagine several clients using this rule to listen to org.foo, but org.foo wanting to send a signal to :1.23 only.
Specifying destination=:1.23 for the signal object didn't really work since no dest=val was specified in the rule, allowing any destination actually matching it and all the listeners to be woken up anyway.

The problem with this situation was fixing the bad behaviour without filter out eavesdroppers, which actually wanted to receive the message even if not for them.

The solution is a sort of "eavesdrop opt-in", as Thiago proposed to add a keyword to DBusMatchRule, "eavesdrop=true false" which defaults to false and with which the listener declares that it really wants to eavesdrop, enabling it to receive messages (including signals) not meant for it (AKA eavesdropping).
Otherwise any message (again, including signals) with a specified destination won't match a filter with no or different destination. Also a rule with a specific destination won't match broadcast messages.


In other words, by default a match rule filter will match only broadcast messages or the ones specifically for you, unless you declare your very nature of eavesdropper by adding "eavesdrop=true" to your filter rule.

This is a behavioural change and consequently means a small amend to the D-Bus spec for 1.5.x.

It also means that if you maintain some code which acts as an eavesdropper, you should fix the code adding "eavesdrop=true" to your filter.
Note that it's only for 1.5.x branch; adding the "eavesdrop" keyword to a filter sent to a 1.4.x bus will fail as the keyword is not recognized.
An example on how to deal with this change keeping compatibility toward 1.4.x (stable branch) and 1.5.x (devel branch) at the same time is shown by this bustle fix. It checks for the feature presence and prepends the keyword if supported.

Currently Debian/Unstable is going through a transition to using /run instead of /var/run. Naturally any significant change to the filesystem layout requires matching changes to SE Linux policy. We currently have Debian bug #626720 open about this. Currently the initscripts package breaks selinux-policy-default in Debian/Unstable so that you can t have initscripts using /run if the SE Linux policy doesn t support it. A patch has been suggested to the policy which uses a subst file, basically that causes the SE Linux labeling programs to treat one directory tree the same way as another. The problem with this is that it depends on a libselinux patch that is not in any yet released version of libselinux (and certainly won t be in a Squeeze update). The upside of such a fix is that it would work for policy that I package as well as custom policy, so if someone wrote custom policy referring to /var/run it would automatically work with /run without any extra effort. I think that the only way to do this is to just have regular expressions that deal with this in the file contexts. It s a bit ugly and slows the relabel process down a little (probably no more than about 10%) but it will work and work on Squeeze as well. One thing I really like to do is to have the SE Linux policy for version X of Debian work with version X+1. This makes upgrades a lot easier for the users. Ideally upgrading a server could be a process that involves separate upgrades of the kernel, the SE Linux policy, and user-space in any particular order because upgrading everything at once almost guarantees that something will break and it may be difficult to determine the cause. At this time I m not sure whether I ll add a new policy using the subs file before the release of Wheezy (the next stable release of Debian) or just keep using regular expressions. I can have the Wheezy policy depend on a new enough libselinux so it won t be a problem in that regard (a new upstream version of libselinux with the subst feature should be released soon). In any case I need a back-port to Squeeze to use regular expressions to make an upgrade to Wheezy easier. for n in $(find . -name "*.fc" xargs grep var/run cut -f1 -d: uniq) ; do
sed -e "s/\/var\/run/\/(var\/)?run/" < $n > $n.new
mv $n.new $n
done
for n in $(find . -name "*.fc" xargs grep var/lock cut -f1 -d: uniq) ; do
sed -e "s/\/var\/lock/\/((var\/run) (run) (var))\/lock/" < $n > $n.new
mv $n.new $n
done
for n in $(find . -name "*.fc" xargs grep dev/shm cut -f1 -d: uniq) ; do
sed -e "s/\/dev\/shm/\/((var\/run) (run) (dev))\/shm/" < $n > $n.new
mv $n.new $n
done I used the above fragment of shell code to change /var/run to /(var/)?run , /var/lock to /((var/run) (run) (var))/lock , and change /dev/shm to /(var/run) (run) (dev))/shm . It involves a reasonable number of changes to policy (mostly for /var/run), but hopefully this will be acceptable to the release team for inclusion in the next Squeeze update as the changes are relatively simple and obvious and the size of the patch is due to it being generated code. There is one final complication, Squeeze currently has selinux-policy-default version 2:0.2.20100524-7+squeeze1, but initscripts in Unstable breaks versions <= 2:0.2.20100524-9. So I guess I could submit a proposed version 2:0.2.20100524-9+squeeze1 to the release team to fix this. I would really like to have the Squeeze policy work with initscripts from Unstable or Wheezy. Any suggestions for how to deal with this? Update: I wrote the above before testing the code, and it turned out to not work. I ve written another post describing a better solution that I have now uploaded to Unstable. I still have to sort something out with an update for Squeeze.

... or TDDD and DEP8 context As a nice byproduct of the huge "rolling" discussion we had back in April/May, various people have brainstormed about applying Test-Driven Development (TDD) techniques to Debian development. Here is a brief summary of some opinions on the matter:

• a seminal conversation on identi.ca
• a couple of blog posts by Lars, discussing the general problem of how to use TDD in distribution development and to a lesser extent its instantiation to Debian
• a separate post by Daniel stressing the specific TDD advantage of increasing the confidence of maintainers in making far reaching changes

... and hey, they've also coined the cool TDDD acronym, which I hereby took the freedom to re-target to Test-Driven Development in Debian. Having a cool acronym, we are already half-way to actually having the process up and running *g*. more testing I believe Debian needs more testing and I've been advocating for that since quite a while e.g. at DebConf10, as one of the main goals we should pursue in the near future. Of course advocating alone is not enough in Debian to make things happen and that is probably why this goal has been (thus far) less successful that others we have put forward, such as welcoming non-packaging contributors as Debian Project members. There are important reasons for increasing testing in Debian. quality assurance Quality Assurance has always been, and still is, one of the distinctive traits of Debian. I often say that Debian has a widespread culture of technical excellence and that is visible in several places: the Debian Policy process, lintian, piuparts, periodic full archive rebuilds, the EDOS/Mancoosi QA tools, the cultural fact that maintainers tend to know a lot about the software they're packaging rather than only about packaging, the we release when it's ready mantra, etc. But caring about quality is not a boolean, it's rather something that should be continuously cherished, refining quality requirements over time. By simply maintaining the status quo in its QA tools and processes, Debian won't remain for long a distribution who could claim to care about package quality. Others will catch up and are in fact already doing that. In particular, we surely have room for improvements in our quality tools and processes for:

• Build time package testing. Several packages run their build-time test suites during package build. This aspect is somewhat supported by the Debian policy; we also do a best effort attempt to run build-time test suites via debhelper (starting from version 7). At the same time we could probably campaign more to encourage maintainers to look for build-time test suites which do not follow common Makefile target naming or, fwiw, which don't rely on make at all. TodoList.add("investigate and possibly file bug report against policy to encourage maintainers to run build-time test suites")
TodoList.add("investigate and possibly file bug report against lintian to spot the presence of [non-invoked] build-time test suites")
• Runtime package testing (AKA "as-installed package testing"). Some kinds of test-suites are difficult to run at build-time (e.g. complex applications that don't offer easy way to re-configure on the fly their filesystem paths) and some others are plainly impossible to run without having the packages to be tested properly setup (e.g. running services that are accessible only through the network) or an isolated, throw-away testbed (e.g. bootloaders, kernels, etc.). TTBOMK this kind of testing is not currently in use neither in Debian nor, indirectly, in any of its derivatives. autopkgtest (AKA DEP8) is a step in this direction; I will get back to it later on in this post.
• Integration testing. When applied in the context of distributions, integration testing is about testing package combinations, task (as in tasksel) combinations, basic system functionalities, automated installations, etc. This is the aspect that Lars has discussed the most (and in which he seems to be interested the most) so I don't feel like adding much. Still, it's interesting to note that on this front Debian has an important potential to exploit. The large base of its developers and the fact that the average Debian user is technical savvy mean that we can potentially collect a lot of test cases. It's "just" (with well-deserved double quotes) a matter of deciding a standard interface for test case submissions and developing a most likely virtualized test runner. At that point a continuous integration service that periodically run and publishes results can be setup independently from the rest of the Debian infrastructure and offered to interested eyes, such as Release Team's. Any taker?
• Automated code analysis and in particular automated bug finding is something which is becoming more and more common and finally, after decades of research viable. Successful proprietary tools and businesses are aplenty (a nice and very fun read on the subject is the story of Coverity), as well as FOSS solutions. For instance, the Linux kernel is routinely using Coccinelle to automatically find bugs and even produce the corresponding patches. Add to the above the fact that Debian is one of the largest software collection in existence (and quite probably the largest collection of Free software in existence) and is upstream of the packaging of that software for the most part. That turns Debian into a very peculiar platform for large scale automated code analyses. Not that we can aim at fixing all the bugs we are going to find, or even hope to tame all the false positives we are going to find. But we can offer an important service to the whole ecosystem of Free software and use it to pinpoint important class of bugs from a distribution point of view, such as nasty security bugs (as long as they can be automatically found ). DACA is a very good step in the right direction, even though it is still in its infancy.

reducing inertia Inertia is a recurring topic among Debian lovers (and haters). It is often argued how difficult it is to make changes in Debian, both small and large, due to several (alleged) hindrances such as the size of the archive, the number of ports, the number of maintainers that should agree before proceeding with a cross-package change, etc. It's undeniable that the more code/ports/diversity you have, the more difficult is to apply "global" changes. But at least for what concerns the archive size, I believe that for the most part it's just FUD: simply debunking the self-inflicted culture about how "dangerous" doing NMUs is might go and has already gone, imho a long way to fight inertia. Adding per-package and integration tests will make us go another long way in reducing inertia when it comes to performing archive-wide changes. Indeed if a package you are not entirely familiar with has extensive test suites, and if they still pass after your changes, you can be more confident in your changes. The barrier to contribution, possibly via NMU, gets reduced as a result. And if your change will turn out to be bad but still not spot by the test suites, then you can NMU (or otherwise contribute) again to extend the test suite and make the life easier for future contributors to that package. It smells a lot like an useful virtuous cycle to me. autopkgtest / DEP8 how you can help Of all the above, the topic that intrigues me the most is as-installed package testing. Work on that front has been started a few years ago by Ian Jackson when he was working for Canonical. The status quo is embodied by the autopkgtest package. At present, the package contains of various tools and the following two specs:

1. README.package-tests provides a standard format to declare per-package tests using the new debian/tests/control file. Tests come as executable files which will be run by the adt-run tool in a testbed where the package(s) to be tested is already installed. This part of the specs has been reified as DEP8 which I'm (supposedly) co-driving with Iustin Pop and Ian (for well-deserved credits).
2. README.virtualisation-sever describes the interface among the test runner and the testbed. A nice separation is provided among the runner and the testbed, enabling different testbed environments with a varying degree of isolation: you can have a "null" testbed which runs tests on your real machines (needless to say, this is highly discouraged, but is provided by the adt-virt-null tool), a chroot testbed (adt-chroot), or a XEN/LVM based testbed (adt-virt-xenlvm).

The specs allow for several runtime testing scenarios and look quite flexible. The tools on the other hand suffer a bit of bitrot, which is unsurprisingly given they haven't been used much for several years. At the very minimum the following Python development tasks are in need of some love:

• The usage of python-apt needs to be ported to recent API, as several used methods and attributes are now gone.
• Porting from dchroot to schroot is needed for the adt-virt-chroot backend.
• A kvm backend for the test runner would be nice.

If you are both interested in TDDD and grok Python, the above and many others tasks might whet your appetite. If this is the case don't hesitate to contact me, I'll be happy to provide some guidance.

---

Note: this post is the basis for the TDDD BoF that I will co-host with Tom Marble at DebConf11. If you plan to come, we will appreciate your thoughts on this matter as well as your help in getting the autopkgtest toolchain up and running again.

From time to time I get e-mails about my wine-unstable packages being uninstallable. Most often it turns out, that the problem stems from using e.g. oldstable (Lenny) instead of at least stable (Squeeze). The same problem arises, if you wish to install the wine-unstable packages on a platform for which I don't offer binaries. In either case you need to build the package from source, which isn't as difficult, as many think. This entry in my ongoing tips mini-series of posts is thus going to detail how you setup pbuilder and afterwards use it to build a package. Please note, that I'm just focusing on pbuilder here and ignore things like cowbuilder. Preparations The following steps are needed just once, in general. Some would be needed again, if you want to create a second base.tgz, e.g. for a different distribution.

1. Install pbuilder with your favourite package manager, e.g. aptitude: aptitude install pbuilder
2. Become root
3. Configure pbuilder. Have a look at /etc/pbuilderrc, /usr/share/doc/pbuilder/examples/pbuilderrc and the manual page for pbuilderrc for reference. If you want to change some settings made in /etc/pbuilderrc, create a /root/.pbuilderrc and override any settings you don't like.
4. Create the base.tgz (the chroot environment for your builds). In the easiest case all your settings are made in the configuration files and you just need to run pbuilder --create, which creates the base.tgz in /var/cache/pbuilder, unless you changed the configuration.
   You should also note, that you can override certain settings, like the distribution or the path to (and name of) the base.tgz file created. See the manual page for pbuilder for all options and when they can be used.

Build a package The following steps are needed for every build:

1. Download the source package, you want to build. No matter how you fetch them (dget, apt-get source, ), you should end up with three files, one of them ending in .dsc.
2. Become root.
3. Update your base.tgz file: pbuilder --update. Again, see the manual page for further options. (You can skip this step, if you just created the base.tgz or you just updated it for another build.)
4. Build the source package: run pbuilder --build /path/to/source-package.dsc (obviously you need to replace the last argument with a real path)
5. If you haven't changed the configuration of pbuilder in that regard, then your binaries end up in /var/cache/pbuilder/result. Install the binaries, you want, from there (with dpkg -i).

Now you should be able to build your own packages from source! And to answer one question, that some of you might have now ("Why use pbuilder at all and not just build the package with dpkg-buildpackage directly?"): because pbuilder allows you to reliably build in a clean environment. That can safe you a lot of trouble, especially if you modified the source package. And it helps to keep the number of installed packages on your system down.