Christian Robert, whose blog I commented-on here once before, had followed up on a recent set of posts by Radford Neal which had appeared both on Radford's blog and on the r-devel mailing list. Now, let me prefix this by saying that I really enjoyed Radford's posts. He obviously put a lot of time into finding a number of (all somewhat small in isolation) inefficiencies in R which, when taken together, can make a difference in performance. I already spotted one commit by Duncan in the SVN logs for R so this is being looked at. Yet Christian, on the other hand, goes a little overboard in bemoaning performance differences somewhere between ten and fifteen percent -- the difference between curly and straight braces (as noticed in Radford's first post). Maybe he spent too much time waiting for his MCMC runs to finish to realize the obvious: compiled code is evidently much faster. And before everybody goes and moans and groans that that is hard, allow me to just interject and note that it is not. It really doesn't have to be. Here is a quick cleaned up version of Christian's example code, with proper assigment operators and a second variable x. We then get to the meat and potatoes and load our Rcpp package as well as inline to define the same little test function in C++. Throw in rbenchmark which I am becoming increasingly fond of for these little timing tests, et voila, we have ourselves a horserace:

# Xian's code, using <- for assignments and passing x down
f <- function(n, x=1) for (i in 1:n) x=1/(1+x)
g <- function(n, x=1) for (i in 1:n) x=(1/(1+x))
h <- function(n, x=1) for (i in 1:n) x=(1+x)^(-1)
j <- function(n, x=1) for (i in 1:n) x= 1/ 1+x 
k <- function(n, x=1) for (i in 1:n) x=1/ 1+x 
# now load some tools
library(Rcpp)
library(inline)
# and define our version in C++
l <- cxxfunction(signature(ns="integer", xs="numeric"),
                 'int n = as<int>(ns); double x=as<double>(xs);
                  for (int i=0; i<n; i++) x=1/(1+x);
                  return wrap(x); ',
                 plugin="Rcpp")
# more tools
library(rbenchmark)
# now run the benchmark
N <- 1e6
benchmark(f(N, 1), g(N, 1), h(N, 1), j(N, 1), k(N, 1), l(N, 1),
          columns=c("test", "replications", "elapsed", "relative"),
          order="relative", replications=10)

And how does it do? Well, glad you asked. On my i7, which the other three cores standing around and watching, we get an eighty-fold increase relative to the best interpreted version:

/tmp$ Rscript xian.R
Loading required package: methods
     test replications elapsed relative
6 l(N, 1)           10   0.122    1.000
5 k(N, 1)           10   9.880   80.984
1 f(N, 1)           10   9.978   81.787
4 j(N, 1)           10  11.293   92.566
2 g(N, 1)           10  12.027   98.582
3 h(N, 1)           10  15.372  126.000
/tmp$ 

So do we really want to spend time arguing about the ten and fifteen percent differences? Moore's law gets you those gains in a couple of weeks anyway. I'd much rather have a conversation about how we can get people speed increases that are orders of magnitude, not fractions. Rcpp is one such tool. Let's get more of them.

End of the second day in the off-grid house. Three things strike me about being here: quiet, rhythm, and awareness. In the city, I am constantly annoyed by noise. Here, there's a empty, echoing feeling to the quiet. It's not lack of noise exactly -- I can hear cicadas and birds right now -- but it's still quiet. At first I wanted to fill it, but now I can feel my hearing instead expanding outward. Catching a faint engine noise, or a dog barking in the distance. Here there's a rhythm to the day driven by the sun. Get up, connect the batteries to the solar panels. (Only necessary because the charge controller is broken.) Check and record the battery levels, make sure the panels are producing well. Think about adjusting them at solar noon. Check the cooler to see how the ice is holding out. Do weekly chores: Refilling the kerosine lanterns and replacing the rock salt used to control the humidity, which is the downside to a earth-sheltered house that stays 70 degrees cool on a 90 degree day. In the evening, watch the sun go down, disconnect the solar panels, record battery levels, and light lanterns. That's the shape of the day. Besides being aware of distant sounds and time time of day, being here brings to the fore awareness of consumables. Until I get a fridge sorted out, I have a cooler full of ice that I have to monitor. There's propane for the stove, and kerosine for the lanterns. And of course always the state of the batteries. So far, the first battery bank seems likely to last longer than the ice. I'll know better tomorrow. Unfortunatly I burned out the 5v power suppzly I was using to run my NSLU2 in a wiring mishap, so I have to run that on an inverter for now, and the batteries, though not low, can barely power both the inverter and my laptop. This has made staying online tricky. But I'm finding dialup psurpisingly tolerable, and it helps to generally slow down, too. Despite all the above, I got about as much real work done today as I normally would. ?tag lay offgrid

A few days ago I talked about how noisy my new Dell Studio 15 was but I can now report back with the beginnings of a solution to that problem, and it doesn't appear to be ACPI related. The first clue I needed was that if I switch to the proprietary fglrx drivers for the Radeon the fan quite quickly drops off to a much more reasonable level. It seems the fglrx drivers have issues, however, in particular I get big black patches on my screen. This video corruption happens especially in Firefox, but sometimes in other applications as well. They also appear to screw up my suspend/resume, which is probably even more annoying to me. The second clue that I needed was that Radeon power management support has only just made it into recent kernels. Thanks to Michael Kirkland for providing me with both of those clues :-) Looking under /sys/class/drm/ I find a whole bunch of stuff, but in particular there are /sys/class/drm/card0/device/power_method and /sys/class/drm/card0/device/power_profile. Looking through the kernel source code I can see that power_profile can be set to low, mid, high, auto and default, while power_method can be set to either dynpm or profile. Trying out all of these values, it seems I get the quietest result with the profile method and either the low or mid profile. The dynpm method is nearly as good, and I would think it should really be the default for a 'Mobility' chipset. From the detailed benchmarking that Phoronix did I wonder if it shouldn't be the default for everyone. For myself, I see some small 'tearing' artifacts occasionally when running with the low profile. These disappear when I run with the mid profile, and since that seems to have pretty much the same temperature (and noise) results I'll go with that one. Though the laptop often does still make more noise than I would prefer it to, it is no longer annoying everyone in the room. Not unexpectedly this seems to have a huge impact on power use, too. It appears that the laptop should now give me around 4.5 hours when I do everything I can think of to lower the power use, whereas before it was more like 2.5 hours. Now I guess I can get back to hacking on DAViCal...

After a bit of work, I got the sheeva plug working the way I wanted it to. First of all, I grabbed a spare 1G USB flash disk I had laying around and installed Debian squeeze to it by following tbm s instructions here: http://www.cyrius.com/debian/kirkwood/sheevaplug/install.html After debian was installed to the USB disk, I removed it from the plug and used dd on my laptop to create an image of the filesystem. I mounted the filesystem as a loopback device and created a jffs2 partition image from it after doing a bit of minor tweaking. I placed the USB disk back in the plug and booted to it. Using this intermediate filesystem and the mtd-tools package, I wrote the new jffs2 image to the NAND mtd device. I then modified the uboot environment to suit, saved, and now have a working setup. Detailed instructions are below. pre-requisites Connect the plug to your network with the rj-45/cat5 cable. Connect a USB flash drive to the plug. I used a 1G drive, but 512M should be sufficient. Install the tftpd package on a machine on your network. You ll also need screen on the machine that you use to connect to the plug s serial terminal. I ll assume these are the same host. You need the IP address of the tftp server. In this example, we will assume that the IP is 192.168.1.2 and that you will assign 192.168.1.200 to the plug. $ sudo apt-get install tftpd screen Place uImage, uInitrd, and uBoot in the /srv/tftp directory: $ sudo wget -O /srv/tftp/uImage http://www.cyrius.com/tmp/beta1/marvell/sheevaplug/uImage
$ sudo wget -O /srv/tftp/uInitrd http://www.cyrius.com/tmp/beta1/marvell/sheevaplug/uInitrd
$ sudo wget -O /srv/tftp/uBoot http://wp.colliertech.org/cj/wp-content/uploads/2010/09/07/uboot.bin Connecting to the serial console We are now ready to boot the plug. Note that you have to be quick with the screen command. I recommend you type it out and get ready to press the enter key. You have to interrupt the bootloader in order to enter the u-boot console. Attach the power and immediately enter the following command: $ screen -S sheeva /dev/ttyUSB0 cs8,ixoff,115200 If you did this right, you should see a prompt like this: Marvell>> update uboot As per tbm s instructions, you can now update the uboot if needed. Instructions for that are here: http://www.cyrius.com/debian/kirkwood/sheevaplug/uboot-upgrade.html I ll put them inline here for completeness.

Marvell>> setenv serverip 192.168.1.2 # IP of your TFTP server
Marvell>> setenv ipaddr 192.168.1.200 # IP of the plug
Marvell>> bubt uboot.bin

be sure to answer n to the question about env parameters. Now reset the device. Don t forget to interrupt the bootloader so we can get back to the prompt:

Marvell>> reset

Let the device know that you will be running a mainline kernel and set the arcNumber:

Marvell>> setenv mainlineLinux yes
Marvell>> setenv arcNumber 2097
Marvell>> saveenv
Marvell>> reset

Install Debian to the USB disk Interrupt the boot process once more. We are now ready to run the debian installer. I ll leave the details as an exercise for the reader. The following should put you into the installer, which you are probably quite familiar with by now. NB: I will assume that you will use a single ext3 partition for the entire system. This will make it easier to build a jffs2 image out of the resultant partition on the USB disk.

Marvell>> setenv serverip 192.168.1.2
Marvell>> setenv ipaddr 192.168.1.147
Marvell>> tftpboot 0x01100000 uInitrd
Marvell>> tftpboot 0x00800000 uImage
Marvell>> setenv bootargs console=ttyS0,115200n8 base-installer/initramfs-tools/driver-policy=most
Marvell>> bootm 0x00800000 0x01100000

build a jffs2 image Once the installation has completed, power down the plug and remove the USB disk. Put the USB disk in the machine you used to get the console on the plug and note what device the kernel assigns to it. In my case, it was given sdc, so debian is installed to /dev/sdc1. I will use these values for this example. If the filesystem was automatically mounted, unmount it. Create a disk image of the partition and mount it as a loopback device.

$ sudo umount /dev/sdc1
$ dd if=/dev/sdc1 of=/tmp/debian.img
$ mkdir /tmp/mnt
$ sudo mount -o loop /tmp/debian.img /tmp/mnt

You can now remove the USB disk and return it to the plug. Modify the /etc/fstab file. The root filesystem will be /dev/mtdblock2 and of fs type jffs2. My fstab file looks like the following:

proc            /proc           proc    defaults        0       0
/dev/mtdblock2 /               jffs2    errors=remount-ro 0       1

We can now create a jffs2 image from the mounted and altered fresh install:

$ sudo mkfs.jffs2 -l \
  -e 0x20000 \
  -X zlib \
  --eraseblock=128KiB \
  --pad \
  --output=/tmp/rootfs.jffs2 \
  --compression-mode=priority \
  -n \
  --squash \
  -r /tmp/mnt

write jffs2 to nand Now that we have an image to flash to the NAND, let s boot off of the USB disk. It should now be attached to the plug. Power on the plug and use screen to get a console. Interrupt the bootloader and enter the following commands:

Marvell>> setenv bootargs_console console=ttyS0,115200
Marvell>> setenv bootcmd_usb 'usb start; ext2load usb 0:1 0x01100000 /boot/uInitrd; ext2load usb 0:1 0x00800000 /boot/uImage'
Marvell>> setenv bootcmd 'setenv bootargs $(bootargs_console); run bootcmd_usb; bootm 0x00800000 0x01100000'
Marvell>> run bootcmd

Log in as root with the credentials you configured during the install. You will need to install a few packages in order to complete the nand flash. You can then ssh to the host on which you created the jffs2 image and cat it to stdout, piping this to nandwrite:

$ sudo apt-get install mtd-utils
$ ssh user@192.168.1.2 cat /tmp/rootfs.jffs2   sudo nandwrite /dev/mtd2 -p -

You now have a jffs2 image on the nand. Configure u-boot One more reboot to set u-boot s environment, and you will be done. Power down and remove the USB disk. Power on and get the serial console using screen. Break into the bootloader and enter the following commands:

Marvell>> setenv bootargs_console console=ttyS0,115200
Marvell>> setenv mtdpartitions mtdparts=orion_mtd:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs)
Marvell>> setenv bootargs_root root=/dev/mtdblock2 rw rootfstype=jffs2
Marvell>> setenv bootcmd 'setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root); nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000'
Marvell>> saveenv
Marvell>> reset

You should now be good to go.

The recent work by MWR Labs does a reasonable job showing Debian s poor pro-active security and why I am so frustrated about it: we have not been able to move very quickly at getting it enabled. While my hardening-includes package is available to maintainers that want to turn on protections for their builds, it s still a far cry from having it be distro-wide, and it doesn t protect people that build stuff by hand. We were able to solve this in Ubuntu very directly a while ago by improving the compiler itself. Since SSP and FORTIFY_SOURCE can only be confirmed (it s not possible without source analysis to see if it should have been enabled), it would be nice to see what binaries differed between distros on this. Most of the SSP disabled stuff are binaries that lack character arrays on the stack to begin with, and the FORTIFY_SOURCE stuff may have done all compile-time protections. The comments about other distributions could potentially enable it for a few more binaries is a bit misleading since, for all but Debian, both SSP and FORTIFY_SOURCE are enabled for all builds. I did appreciate the nod to Ubuntu for being the only distro without by-default PIE that built Firefox with PIE. Given that Firefox is the #2 most vulnerable piece of software in a desktop distro, it was important to do it. (The #1 most vulnerable is the kernel itself I m counting number of fixed CVEs for this stat.) The kernel analysis by MWR seems rather incomplete. Also, it s not clear to me which distros were running a PAE kernel, which would change some of the results. I didn t see any mention of several other userspace protections that the kernel can provide, for example:

• symlink and hardlink protections (Gentoo Hardened and Ubuntu 10.10 only)
• PTRACE protections (Gentoo Hardened and Ubuntu 10.10 only)

And a ton more that only Gentoo Hardened could boast, due to their use of grsecurity. I d also be curious to see performance comparisons, too. They compared 4 general-purpose distros against a tuned-specifically-for-security-hardening distro, which seems a bit unfair. How about comparing against vanilla Gentoo instead? I can tell you who would be best then. :)

Since the introduction of the 3.0 (quilt) source format, it is now possible to integrate multiple upstream tarballs in Debian source packages. This article will show you how to do the same with your own package shall you need it. It s quite useful to easily integrate supplementary plugins, translations, or documentation that the upstream developers are providing in separate tarballs. Step by step explanation We ll take the spamassassin source package as an example. The upstream version is 3.3.1. The main upstream tarball is named as usual (spamassassin_3.3.1.orig.tar.gz) and contains the top directory of our source package. We already have a debian directory because the package is not new. Upstream provides spamassassin rules in a separate tarball named Mail-SpamAssassin-rules-3.3.1.r901671.tgz. We grab it, rename it to spamassassin_3.3.1.orig-pkgrules.tar.gz and put it next to the main tarball. The pkgrules part is the component name that we choose to identify the tarball, it s also the name of the directory in which it will be extracted inside the source package. For now that directory doesn t exist yet so we must create it.

$ mv Mail-SpamAssassin-rules-3.3.1.r901671.tgz spamassassin_3.3.1.orig-pkgrules.tar.gz
$ cd spamassassin-3.3.1
$ mkdir pkgrules
$ tar -C pkgrules -zxf ../spamassassin_3.3.1.orig-pkgrules.tar.gz

This is already enough, the next time that you will build the source package, the supplementary tarball will be automatically integrated in the generated source package.

$ dpkg-buildpackage -S
[...]
 dpkg-source -b spamassassin-3.3.1
dpkg-source: info: using source format  3.0 (quilt)'
dpkg-source: info: building spamassassin using existing ./spamassassin_3.3.1.orig-pkgrules.tar.gz ./spamassassin_3.3.1.orig.tar.gz
dpkg-source: info: building spamassassin in spamassassin_3.3.1-1.debian.tar.gz
dpkg-source: info: building spamassassin in spamassassin_3.3.1-1.dsc

The supplementary tarball is now part of the source package but we re not making anything useful out of it. We have to modify debian/rules (or debian/spamassin.install) to install the new files in the binary package. A special case: bundling related software In very rare cases, you might want to create a bundle of several software (small perl modules for example) and you don t have any main tarball, you only have several small tarballs. Rename all the tarballs following the same logic as above and when building the source package you can ask dpkg-source to create an empty (and fake) main archive for you with the option --create-empty-orig:

$ dpkg-buildpackage -S --source-option=--create-empty-orig

Use with care as the version number you give to the bundle is what users will see and it s likely unrelated to the version number of each individual software. Common mistakes Forgetting to extract the supplementary tarball If you forget to extract the content of the supplementary tarball in the pkgrules directory, dpkg-source will emit lots of warnings about those files being deleted. In fact, you did not delete them but you only forgot to create them in the first place.

dpkg-source: warning: ignoring deletion of directory pkgrules
dpkg-source: warning: ignoring deletion of file pkgrules/20_fake_helo_tests.cf
dpkg-source: warning: ignoring deletion of file pkgrules/60_shortcircuit.cf
[...]

Using a bad version number for the supplementary tarball Sometimes the supplementary tarball has a version of its own that does not match the upstream version. You must still name the file in a way that matches the upstream version of the main tarball otherwise it will not be picked up by dpkg-source and it will generate a new patch in debian/patches/ containing the whole new directory. It s possible to encode the version number of the supplementary tarball in the component name (in our example above we could have picked pkgrules-r901671 as component name) but this means that the name of the associated directory will regularly change and you must adapt your packaging rules to cope with this. However this last trick has the benefit of being able to update the additional tarball without bumping the upstream version. A sourceful upload of a new revision of the package will be accepted by the archive: the main tarball is ignored since it s unchanged but the supplementary tarball is taken since it s a new file for the archive (it has a different filename). Be sure to move away old versions of the additional tarball when you do that if you don t want to upload several versions of the same tarball by mistake! Misextracting the supplementary tarball dpkg-source is very smart when it extracts the supplementary tarball and you should be as well when you manually extract it. If the tarball contains only a single top-level directory, that directory is extracted, renamed to match the component name and moved in the source package directory.
If the tarball contains several top-level files or directories, then the target directory is first created and the content of the archive is directly extracted into that directory. Here are commands to install the files in both cases (we re already in the source package directory):

$ mkdir pkgrules
# Archive contains a single top-level directory
$ tar -C pkgrules --strip-components=1 -zxf ../spamassassin_3.3.1.orig-pkgrules.tar.gz
# Archive contains many top-level entries
$ tar -C pkgrules -zxf ../spamassassin_3.3.1.orig-pkgrules.tar.gz

2 comments Support my work

I've been using IPv6 on my rootserver for some time now. Last week I migrated to a new rootserver and copied my domU/VM instances over to the new hardware. Everything is working fine so far - except IPv6. The network setup is the same as on the old server: the external interface is eth0. The domU/VMs are hooked up to a bridge, called xenbr0. There's another bridge for internal communication: xenbr1. The dom0/Xen host itself seems reachable via IPv6. The /48 subnet is provided by Sixxs and is using a static 6-to-4 tunnel. But the VMs are not reliable reachable, although it is setup in the same way as it was on the old server and where it was working like a charme. The configs of the hosts are these:

Xen dom0

config	old server	new server
System	Debian Etch 2.6.18-6-xen-amd64 linux-image-2.6.18-6-xen-amd64 linux-modules-2.6.18-6-xen-amd64 xen-hypervisor-3.0.3-1-amd64 xen-ioemu-3.0.3-1 xen-linux-system-2.6.18-6-xen-amd64 xen-tools xen-utils-3.0.3-1 xen-utils-common	Debian Squeeze 2.6.32-5-xen-amd64 libxenstore3.0 linux-image-2.6.32-5-xen-amd64 xen-hypervisor-4.0-amd64 xen-linux-system-2.6.32-5-xen-amd64 xen-tools xen-utils-4.0 xen-utils-common xenstore-utils xenwatch
/etc/sysctl.conf	net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1 net.ipv4.conf.default.proxy_arp=1	net.ipv4.ip_forward=1 net.ipv4.ip_syncookies=1 net.ipv4.conf.all.rp_filter=1 net.ipv4.icmp_echo_ignore_broadcasts=1
/etc/xen/xend-config.sxp	(network-script network-route) (vif-script vif-bridge) (dom0-min-mem 196) (dom0-cpus 0)	(vif-script vif-bridge) (network-script network-route) (dom0-min-mem 196) (enable-dom0-ballooning yes) (total_available_memory 0) (dom0-cpus 2) (vncpasswd '')
/etc/network/interfaces	# device: eth0 auto eth0 iface eth0 inet static address 85.10.209.30 broadcast 85.10.209.31 netmask 255.255.255.224 up route add -net 85.10.209.0 netmask 255.255.255.224 gw 85.10.209.1 eth0 true up ip route add 85.10.209.0/28 via 85.10.209.1 src 85.10.209.30 true up route add default gw 85.10.209.1 true auto xenbr0 iface xenbr0 inet static address 85.10.209.30 netmask 255.255.255.224 pre-up brctl addbr xenbr0 up route add -net 78.47.85.144/29 dev xenbr0 true up ip -6 r a 2001:6f8:90e:145::1/64 via 2001:6f8:90e:1:216:3eff:fe55:197c dev xenbr0 true up ip -6 r a 2001:6f8:90e:146::1/64 via 2001:6f8:90e:1:216:3eff:fe2f:481d dev xenbr0 true up ip -6 r a 2001:6f8:90e:147::1/64 via 2001:6f8:90e:1:216:3eff:fe60:68be dev xenbr0 true up ip -6 r a 2001:6f8:90e:a100::1/64 via 2001:6f8:90e:1:216:3eff:fe70:be dev xenbr0 true auto xenbr1 iface xenbr1 inet static address 192.168.x.254 netmask 255.255.255.0 pre-up brctl addbr xenbr1 # some internal IPv4 routing auto sixxs iface sixxs inet6 v4tunnel address 2001:6f8:900:c6e::2 netmask 64 endpoint 212.224.0.188 local 85.10.209.30 ttl 64 up ip link set mtu 1280 dev sixxs true up ip route add default via 2001:6f8:900:c6e::1 dev sixxs true up ip -6 route flush dev eth0 true up ip -6 r a 2001:6f8:90e::/48 dev xenbr0 true down ip -6 route flush dev sixxs true	# device: eth0 auto eth0 iface eth0 inet static address 178.63.83.84 broadcast 178.63.83.127 netmask 255.255.255.192 gateway 178.63.83.65 # default route to access subnet up route add -net 178.63.83.64 netmask 255.255.255.192 gw 178.63.83.65 eth0 auto xenbr0 iface xenbr0 inet static address 178.63.83.84 netmask 255.255.255.192 pre-up brctl addbr xenbr0 up route add -host 178.63.83.104 dev xenbr0 true up route add -host 178.63.83.105 dev xenbr0 true up route add -host 178.63.83.106 dev xenbr0 true up ip route add 178.63.123.128/26 dev xenbr0 true up ip -6 r a 2001:6f8:90e:145::1/64 via 2001:6f8:90e:1:216:3eff:fe89:6c31 dev xenbr0 true up ip -6 r a 2001:6f8:90e:146::1/64 via 2001:6f8:90e:1:216:3eff:fedc:af5f dev xenbr0 true up ip -6 r a 2001:6f8:90e:147::1/64 via 2001:6f8:90e:1:216:3eff:fe08:8f40 dev xenbr0 true up ip -6 r a 2001:6f8:90e:a100::1/64 via 2001:6f8:90e:1:216:3eff:fe70:be dev xenbr0 true auto xenbr1 iface xenbr1 inet static address 192.168.x.254 netmask 255.255.255.0 pre-up brctl addbr xenbr1 # some internal IPv4 routing auto sixxs iface sixxs inet6 v4tunnel address 2001:6f8:900:c6e::2 netmask 64 endpoint 212.224.0.188 local 178.63.83.84 ttl 64 up ip link set mtu 1280 dev sixxs true up ip route add default via 2001:6f8:900:c6e::1 dev sixxs true up ip -6 route flush dev eth0 true up ip -6 r a 2001:6f8:90e::/48 dev xenbr0 true down ip -6 route flush dev sixxs true

The config of one of the Xen domUs is this:

sample domU config

config	old server	new server
system	Debian Lenny 2.6.18-6-xen-amd64	Debian Lenny 2.6.32-5-xen-amd64
/etc/sysctl.conf	net.ipv4.conf.default.rp_filter=1 net.ipv4.tcp_syncookies=1 net.ipv4.conf.default.forwarding=1 net.ipv6.conf.default.forwarding=1 net.ipv4.conf.eth0.proxy_arp=1 net.ipv4.conf.default.proxy_arp=1 kernel.shmmax=268435456	net.ipv4.conf.default.rp_filter=1 net.ipv4.tcp_syncookies=1 net.ipv4.conf.default.forwarding=1 net.ipv4.conf.all.forwarding=1 net.ipv6.conf.default.forwarding=1 net.ipv4.conf.eth0.proxy_arp=1 net.ipv4.conf.default.proxy_arp=1 kernel.shmmax=268435456
/etc/network/interfaces	# The primary network interface auto eth0 iface eth0 inet static address 78.47.85.145 netmask 255.255.255.248 up ip route add 85.10.209.0/27 dev eth0 up route add default gw 85.10.209.30 dev eth0 up ip -6 address add 2001:6f8:90e:145::1/64 dev eth0 true up ip -6 route add default via 2001:6f8:90e:1::1 true up iptables -t nat -A POSTROUTING -s 192.168.x.96/27 -o eth0 -j MASQUERADE true iface eth0 inet6 static address 2001:6f8:90e:145::1 netmask 64 gateway 2001:6f8:90e:1::1 auto eth1 iface eth1 inet static address 192.168.x.1 netmask 255.255.255.0	# The primary network interface auto eth0 iface eth0 inet static address 178.63.83.104 gateway 178.63.83.84 netmask 255.255.255.192 broadcast 178.63.83.127 up ip -6 address add 2001:6f8:90e:145::1/64 dev eth0 true up ip -6 route add default via 2001:6f8:90e:1::1 src 2001:6f8:90e:1:216:3eff:fe89:6c31 true up iptables -t nat -A POSTROUTING -s 192.168.x.96/27 -o eth0 -j MASQUERADE true iface eth0 inet6 static address 2001:6f8:90e:145::1 netmask 64 gateway 2001:6f8:90e:1::1 auto eth1 iface eth1 inet static address 192.168.x.1 netmask 255.255.255.0

The dom0 seems to be reachable via IPv6 just perfectly fine. When trying to reach the domU I see packets going through the xenbr0 bridge and reaching the domU eth0 interface. The ICMP6 echo request packets (proto 58) are unanswered there, no echo replies. Strange enough it seems to work from time to time, but mostly not. I've also tried to set a generic default route like ip -6 route add default dev eth0, but still no improvement. Did I miss something when migrating to the new server? Is there any mistake in the configs? Any suggestions are appreciated! :-) UPDATE:
This is a tcpdump from the domU, doing a ping from dom0:

07:35:29.308261 IP6 fe80::e46d:25ff:fe1f:317b > ip6-allnodes: ICMP6, router advertisement, length 56
07:35:29.329411 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 45, length 64
07:35:30.329439 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 46, length 64
07:35:30.451991 IP6 2001:418:4001:3::c657:b0c5 > vserv.windfluechter.net: ICMP6, echo request, seq 52836, length 64
07:35:30.481124 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:31.329906 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 47, length 64
07:35:31.481645 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:32.329284 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 48, length 64
07:35:32.481114 IP6 fe80::e46d:25ff:fe1f:317b > ff02::1:ff70:be: ICMP6, neighbor solicitation, who has 2001:6f8:90e:1:216:3eff:fe70:be, length 32
07:35:33.329951 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 49, length 64
07:35:34.329705 IP6 gate-hro.ip6.windfluechter.net > vserv.windfluechter.net: ICMP6, echo request, seq 50, length 64

UPDATE #2:
The issue is finally solved. The solution can be found here.

Two updates for Debian today.

I'm the upstream for psmisc and 22.13 finally got released, which also meant 22.13-1 Debian package got released too. There was some delay to it (see below why) but it is now out. Unless you run a mips or superH architecture, there is not really any exciting changes, but should make it compile for those two architectures and then at least get the mips buildd underway so the versions all line up.

Secondly, gjay had two minor bug fixes and was updated. If the analysis daemon kept playing music instead of looking at it or the vorbis files were not being recognised, then this new version will help there. The mpg123 command line patch will be rolled into the upstream too.

I'm also working on getting gjay to work with the Exaile player. If you want it to work with your player the easiest thing is to send me patches or code snippets that do the following:

• Detect your player is running
• Can give me the currently played song, preferably the filename of it. Exaile doesn't so I'll put it a kludge to guess it from title and artist
• Remotely send a playlist to your player and get the player to start

Last of all, my second son was born last week. It's back to those night feeds again and is probably why I've not written as much code as I normally would; but I wouldn't change that either.

My vacation is now over, and I think it is now time for me to summarise what I have done with regards to free software. First, I have sent my first real contributions to OpenStreetMap, in the area of my small town: http://osm.org/go/0DCaXeZ
I still have a lot of things to learn, but things are already better than they were!
I will also shortly write an article about how to use the GlobalSat DG-100 data logger on Debian GNU/Linux.
Working on OSM is real fun, as it can be done in family. I have also spent some time to test & improve the Debian packages for DSPAM (available from my repository). I have moved the SVN repository to a new GIT repository available at: http://git.debian.org/?p=pkg-dspam/dspam.git;a=summary. Matthijs M hlmann (matthijs) is currently reviewing the packages, and will upload them to the official archive if everything is ok. Matthijs also accepted to be my advocate for my New Maintainer application. Thanks a lot to him! I am now waiting to be assigned an Application Manager: http://qa.debian.org/developer.php?login=julien%40kirya.net As usual, I have also been quite active at reporting bugs on the Debian BTS. unfortunately, I haven t had the possibility to fix (RC) bugs.

I'm currently working on a Fortran program at work: a post-processing tool that takes climate data, in NetCDF format, and outputs in CMOR2 format (a NetCDF variant with climate conventions). So, it links against netcdf and cmor. Now in HPC and climate in particular, codes are typically linked statically: partially for robustness, but mostly for speed (more on which later). So, I'd like to link this statically, as I have tens of terabytes of data to process. Now, mostly I've been linking using pkg-config:

  gfortran -o nemo-rewriter nemo-rewriter.f90  pkg-config --libs --cflags nemo cmor 

pkg-config assembles the libraries. For dynamic libraries, the netcdf and cmor libraries are themselves linked to dependencies. But in the static case, all dependencies need to be on the link line, which is more complex. Never mind, it should be possible with:

  gfortran -static -o nemo-rewriter nemo-rewriter.f90  pkg-config --static --libs --cflags nemo cmor 

This should work by assembling all the required static libraries, via pkg-config dependencies. Unfortunately not every package has a .pc file, and so this fails: As of version 4.1 NetCDF allows a URL instead of a file to read, and hence depends on curl to retrieve the file. Curl has no pkg-config .pc file describing its libraries, and it fails. Never mind, lets assemble the static libraries by hand. Debian provides static versions of libraries in the -dev packages. Can I assemble a statically-linked program ? For this I need:

• NetCDF needs libnetcdff.a and libnetcdf.a directly.
• NetCDF needs HDF5: libnethdf5_hl.a and libhdf5.a for version 4 files.
• CMOR2 needs: libcmor2.a
• CMOR2 depends on libudunits2.a, to convert between physical units.

Now here it gets interesting. To handle secure communications and authentication, curl has some complex dependencies. It has two versions. Pick the gnutls one for example:

• libgss.a for Generic Security Services.
• libgss needs libidn.a for Internationalized Domain names
• libgss needs libshishi.a for Kerberos
• libshishi.a needs libgpg-error.a
• libshishi.a needs libgnutls.a
• libshishi.a needs libtasn1.a
• libshishi.a needs libgcrypt.a
• libshishi.a needs libresolv.a
• libcurl.a needs libssl.a
• libcurl.a needs libssh2.a
• libcurl.a needs libldap_r.a
• libdlap_r.a needs liblber.a
• libldap_r.a needs libsasl2.a. Which access databases, so ...
• libsasl2.a needs libmysqlclient.a
• libsasl2.a needs libpq.a
• libsasl22.a needs libdb-4.8.a
• libsasl.a needs libsqlite.a
• libssl.a needs libcrypto.a
• libcrypto.a needs libz.a
• libcurl.a needs libdl.a
• libcurl.a needs libcom_err.a
• libcurl.a needs libkeyutils.a
• libcurl.a needs libgssapi_krb5, which is in dynamic form only
• libcurl.a needs libkrb5crypto, which has no static library

I may have missed some out, having stopped because there is no static implementation of Kerberos on Debian. But still, the idea that a simple little fortran proggie will statically link in four database libraries is silly. It appears to be no longer possible to simply statically link a program in Debian, and definitely not via pkg-config, because so many dependencies do not yet have configuration files. Tags debian, static, cmor, pkg-config

The following post is the work of Pablo Duboue, the DebConf10 fundraising team leader.So you want to make DebConf, eh? You will need quite a bit of money then. That's when fundraising comes into play.There are traditionally two sources for funds for DebConf: government and businesses. A third one, micro-donors, has been mentioned, but haven't been explored yet. (Note, in this text I use the terms sponsors and donors interchangeably, as all sponsors are actually doing a donation to the Debian project earmarked for the use at a given DebConf.)Government funding has been key in a number of DebConfs. In DC10, though, by the time fundraising proper started, it was too late to secure it. Extrapolating from two DebConfs that had government funding, I would venture the following two pieces of advice: target a government level as local as possible (city, provincial level) and try to secure the funding at the time of the bid (which, by the way, can help you win the bid). A successful process will include government representatives aware of the bid status and the whole bidding process. This partnership can help unlock extra government resources such as housing or transportation. But to reiterate, if you haven't at least applied for government grants by the time the DebConf previous to yours is happening, then you have to plan to go along without government funding.Business funding for DebConf includes a significant amount of recurring donors. Keeping them happy is even more important than any money they will put forward for a given DebConf. Now, because of the high level of recurrence and the fact that government funding can cover easily one third of our costs, fundraising without government funding really means your projected funds are one third off. If that is the case, make sure to communicate that to the rest of debconf-team and even the larger Debian community (see below "asking for help").This entry has been truncated read the full entry.

Russel Garlick writes on behalf of the KohaCon10 Organising Committee:

KohaCon10 starts on October 25th in Wellington, New Zealand. We have an exciting line up of speakers on a range of topics related to Koha and [Free and] Open Source and Open Standards in libraries. See our programme for details. KohaCon is an opportunity for the entire Koha community, librarians and developers alike, to come together, meet each other, swap ideas and learn something new. The conference is split into 2 parts. The community conference will be held over 3 days 25-27th of October. This is not just a developer s conference. There will be presentations from librarians and developers alike. The second part of the conference is the Hackfest for Koha developers that will be held from 29th-31st of October. For more information see our website KohaCon10 is a free conference (that is right it will cost nothing for you to attend), but you still need to register to reserve your place. Registrations from the international Koha community have been very strong. Over half of all available spaces are already taken. If you have been holding off on the premise that you will have plenty of time to do this later, then please register now. Please do not rely on there being free spaces on the day. Registration is quick and easy via the website. We look forward to seeing you in Wellington!

Our co-op will be represented there. Will you?

Since I published How to make 110.28 EUR in one month with free software and Flattr , quite a few people joined the movement and I had less troubles finding projects to Flattr. That s great! Without further ado, here are my recommendations for september:

1. Smuxi is an IRC application that integrates from the start the need to be permanently connected. You can run the engine on a server and the graphical interface connects there. The engine has all the intelligence so it remembers what message you last saw in each channel (including highlights) and you won t miss anything. I recently switched to it and I like it. It s not perfect but it s improving quickly. The next version even supports Ubuntu s messaging indicator for better integration with the desktop.
2. Geshi is a PHP class used by many other software to provide enhanced views of code on web pages by coloring the various parts based on the syntax of the respective programming language. It s used by Wikipedia, many blog plugins and lots of other web applications.
3. Git-buildpackage is a set of tools developed by Guido G nther to make it easier to maintain Debian packages in Git repositories. I use it for several packages that I maintain.
4. Mixare is an augmented reality browser. Take your Android phone and it will incrust information in the (camera) view of the environment. Impressive video on the website.
5. Lightspark is is a modern flash player implementation targetting ActionScript 3.0 support (while Gnash only supports older versions). It s one of those few projects that are really needed to get rid of the non-free Adobe plugin that almost everybody installs because there s nothing else that work well enough.

This article is part of the Flattr FOSS project.

10 comments Support my work

(For Those Who Care About An Introduction: Christine Spang gave a talk during DebConf10 about Simple Defects (SD), and blogged about it later on.) Folks maintaining Debian packages are already able to partially-clone bugs.debian.org s bug database thanks to the local-debbugs tool. But what about upstream s bug tracker? Taking a (shamelessly self-centered) example: X.Org packages are hosted on FreeDesktop.org s bugzilla. Thanks to SD, it s possible to fetch bugs from there as well! Here s the obligatory picture: This means that you can browse/search them locally while being offline (or well-connected, but without having to use that !$\ %$^ bugzilla web interface). Many of the replica types support both reading and writing, meaning you can also queue some changes locally, and push them later. Currently, sd help sync says that read-write support is available for RT, Hiveminder, Trac, Google Code, and GitHub. There s also read-only support for redmine. Debbugs is being worked on, see Christine s blog post about her SD talk for more info. Given there was no support for bugzilla, I had a quick look and reported my findings. The main point being: \o/ Bugzilla s XMLRPC \o/ A little while later (I m not exactly fluent in Perl ), I came up with a tentatively-mergeable branch adding preliminary read-only support for bugzilla. There s still a lot of work, but I m trying to work on it on a regular basis, adding support for more properties, and fixing bugs (tests should be written some day, too).

The last few weeks I have been using LaTex a lot, to write the final report for my master's course. During that period I found a couple of LaTeX goodies that might be interesting to share: the memoir class, gnuplot integration, and some stuff to help preparing cover pages. The memoir class is a kind of all-in-one package. It is meant to be used in place of the book and report classes, but it can also replace simpler environments, such as article. It provides many ready-to-use templates, and loads of commands to easily customize them. It seems to be specially tailored for dead-tree publication, dedicating an entire chapter in the manual to page layout. It also aims at removing the usual cruft you have to add into any medium sized project, it is macro-compatible with: abstract, appendix, array, booktabs, ccaption, chngcntr, chngpage, dcolumn, delarray, enumerate, epigraph, framed, ifmtarg, ifpdf, index, makeidx, moreverb, needspace, newfile, nextpage, parskip, patchcmd, setspace, shortvrb, showidx, tabularx, titleref , titling, tocbibind, tocloft, verbatim, and verse. Furthermore, it provides functionality equivalent to the following packages: crop, fancyhdr, geometry, sidecap, subfigure, titlesec. In the end, you have a project that only needs a couple of packages, and a few customisation commands, which is nice and also eases distribution. Other nice thing I have found is that there is not one, but many ways of merging gnuplot graphs into your text while having LaTeX typeset all the labels. There is some LaTeX packages aimed at this, but I have more success using a special gnuplot terminal. Again, there is more than one option: eepic, epslatex, latex, lua, mp, pslatex, pstex, pstricks, texdraw, and tpic. While I could not try them all, I tried a few and the easiest and prettiest was epslatex. This terminal creates two files: a EPS file, as would have been created with the usual postscript terminal, except that it does not contain any text; and a LaTeX snippet meant to be included with \input foo . Finally, I could generate graphs that match the rest of the document! To end this post, a couple of things about creating the dreaded cover page. First, a suggestion: don't fret too much about finding a ready-made template, once you understood the basics of how to insert arbitrary spaces around text, it is quite easy to roll your own cover! Having said that, the memoir manual has some sample cover pages that can be useful to use as a base, and there is another really useful resource I have found: titlepages.pdf, a collection of 40 sample title pages with source code for you to choose. Tags: Planet Debian, Planet Lugfi, latex

Debian and Ubuntu have a set of official membership roles that can be granted to regular contributors. Those roles come with rights that enable the contributors to do their work and to participate in the project governance (elections and other official decision-making processes). It s also a way for the distributions to acknowledge the work done: most contributors are proud of the status they reached. The membership structure plays an important role in the development of a distribution: it defines the kind of contributors that are welcome in the project, it sets expectations of the project towards its contributors and defines their rights. In the end, this shapes the project s ability to recruit new contributors to keep the project alive and kicking. This article introduces the existing statuses in Debian and Ubuntu, and defines the sometimes confusing jargon associated with them. The Debian Case Debian only has two types of official members: Debian Developers (DD) and Debian Maintainers (DM). The rights of the developers are codified in the Debian Constitution while those of the maintainers have been defined in a general resolution of 2007. The Debian Maintainer status is still mostly documented in a wiki page. The integration of this new status in Debian s official processes has been slow to come largely because it was introduced at that time without enough negotiation with the involved parties. Nowadays, it is preferred that people get the DM status before applying for DD. DM is a very limited role: maintainers can only upload packages that already have their name on them (either in the Maintainer or Uploaders field) and a specific flag (DM-Upload-Allowed: yes) that only Debian Developers can add. They have no other rights and limited access to Debian s resources. Besides those official roles, there are also maintainers of packages that have no official status within Debian except that they are listed in the Maintainer field of the package. They are doing the maintenance work but all uploads are done by a Debian Developer after verification of the work done (this is called sponsorship and is the only way to start with official packaging work). Once the DD trusts the maintainer, the developer will typically ask the maintainer to apply for DM status in order to be relieved from the sponsorship work. In the end, that makes three different kind of package maintainers and a lot of confusion when you discuss membership issues in particular when the New Maintainer process is the path that you follow to become a Debian Developer. Don t be fooled by the names when reading Debian s documentation! The Ubuntu Case Ubuntu had, from the start, an official Ubuntu Member status that includes all contributors: developers of course, but also documentation writers, artists, translators, etc. This status notably grants the right to vote in elections of the Community Council, the right to participate on Planet Ubuntu, and the @ubuntu.com email alias. For developers, the situation is more complicated: the wiki page lists no less than five different statuses. Initially, developers were split between Ubuntu Core Developers and the MOTU (Masters Of The Universe). The latter were responsible of the universe/multiverse sections of the archive while the former also had upload rights for the main/restricted sections. But, inspired by the Debian Maintainers concept and facing concrete problems in terms of archive management, they changed their infrastructure to offer more fine-grained control on package uploads. Ubuntu can now grant upload permissions on a package-per-package basis, but it can also delegate the right to grant upload permissions with the same granularity. This lead to the new Per-Package Uploader status which is simply an Ubuntu Member with upload rights on a limited set of packages where they have a specific expertise. The more generic Ubuntu Developers status now encompasses members of various development teams that have been delegated the right to manage upload permissions on a (usually large) package set (the current teams are Ubuntu Desktop, Mythubuntu, Kubuntu, and Edubuntu). Those teams can define their own policy to add new members provided they follow the basic rules defined by the Developer Membership Board (see this wiki page). Ubuntu Contributing Developer is an intermediate status for someone who is not yet ready for one of the other developer statuses but who has still shown enough commitment to be an Ubuntu Member. All those statuses can be obtained in a similar way: you prepare a wiki page listing your past contributions, you collect testimonials from existing members that you have worked with, you add yourself in the agenda of the next meeting of the board (or council) that grants the status that you seek, and you attend the meeting. The members of the board will decide whether you are ready for the status (or not) based on what you provided in the wiki, based on your answers during the meeting (and on a mailing-list for developers), and based on what others have to say about you. The most important boards are usually elected by the community while others are commonly appointed by the community council. Those governance bodies include Canonical employees but not as many as one would expect: two out of eight in the Developer Membership Board, two out of eight in the Community Council, but all six members of the Technical Board. The last figure, while not intended, is not surprising given the high expectations set on potential members of the technical board. Mark, as the founder, is the only person to have a permanent seat on both the Community Council and the Technical Board. Comparison of the Statuses Between Debian and Ubuntu The following table summarizes the rights given to each developer role in the two projects (Put the mouse over the abbreviations to know what they are referring to).

Rights	Debian		Ubuntu
	DM	DD	UM	PPU/UD	MOTU	UCD
Package maintenance via sponsorship	Y	N/A	Y	Y	Y	N/A
Official email alias	-	Y	Y	Y	Y	Y
Participate in votes for members	-	Y	Y	Y	Y	Y
Participate in votes for developers	-	Y	-	Y	Y	Y
Upload rights restricted to pre-approved packages	Y	-	-	Y	-	-
Upload rights restricted to a section of the archive	-	-	-	-	Y	-
Unlimited upload rights	-	Y	-	-	-	Y
Number of contributors (as of 2010-07-27)	117	904	462	27	85	63

Please note that the number of contributors are not 100% accurate for Ubuntu. A contributor can have multiple statuses (direct membership to a launchpad group) granted over time (while gaining experience). The problem has been mostly avoided by calculating differences between number of members of the various groups but it s not perfect and it can t be: some MOTU are also PPU for packages in main and it s legitimate (but I only counted them as MOTU and not as PPU). Another limitation is that members of some administrative teams are included indirectly in many teams and thus appear in the count while they should not. Anyway, this simple table makes it obvious that Ubuntu s structure offers a broader choice of statuses. They acknowledge the work of all contributors from the start while still giving the most critical rights only to those who have proven that they deserve them. Despite this difference, Debian still has a significant advantage in terms of number of developers. That number does not tell the whole story though: the Ubuntu contributors include many Canonical employees (e.g. 36 out of 63 core developers have a @canonical.com email registered on their launchpad account) that are likely to spend more time working on the distribution than the average Debian member. But even if comparing person-hours would be a challenging thought experiment, in practice it s of not much interest if both projects continue to cooperate and if more and more of the contributions flow in both directions. Debian is aware of the shortcomings of its structure. Changes to better accommodate non-packagers have been discussed several times already. The last efforts in that direction were unfortunately perceived as a solution ready to go rather than a proposal to be discussed, and the project got quickly buried by a general resolution (GR). Even if that resolution invited for further discussion and a new proposal, the truth is that when someone s initiative is corrected by way of GR, it usually kills any motivation to go forward. Possible Evolution? On the Ubuntu side, the infrastructural changes were completed recently and they don t expect any further change in the near future. They do plan, however, to expand usage of those new features so that more teams benefit from the possibility to control upload rights on packages that are relevant to them, and so that more individuals developers apply to become Per-Package Uploaders on packages that they know very well. On the Debian side, a recent discussion on the debian-project list brought back the topic of the bad terminology and it was agreed that the New Maintainer process should be renamed into something else ( New Developer process has been suggested). But Christoph Berg Debian Account Manager and hence heavily involved in the New Maintainer Team suggested that Debian would be better off implementing the long-awaited membership changes before trying to update all the documentation. It would certainly imply some more vocabulary updates. Later in the discussion, he confirmed that membership reform is on the top of the TODO list of the new maintainer team (just after the rewrite of the nm.debian.org website). What can be expected from this reform? The following answers are my own guesses based on my experience of Debian, but the project hasn t decided anything yet.

• First of all: a new status for contributors that are not packagers. The tricky part will be defining the process to follow and the rights granted.
• Changes to the technical implementation of the DM status. The current implementation does not allow to give upload rights to a single DM if two are listed in the Uploaders field of a package (and both might not have the same experience for that package). Furthermore, it suffers from annoying restrictions like the inability to upload new binary packages.
• A change of the Debian constitution to integrate those new statuses is almost unavoidable.
• Other more invasive changes have been proposed like replacing the NM process by a simple designation by other DD, but it s unlikely to happen. The NM process can already be greatly simplified by the application manager if the applicant can show good testimonials from other developers and if he has a track record of real contributions (e.g. as witnessed by changelog entries in Debian packages).

Almost two years have elapsed since the previous efforts in that direction, the new maintainer team has recruited new members and is in a general better shape. Hopefully, the next episode of this saga will have a better outcome.

This article was first published in Linux Weekly News. In a comment, Mark Shuttleworth tried to explain how the Ubuntu community is being setup.

10 comments Support my work

The DebConf registration team has the hard job of coordinating accommodation, food, and payments for hundreds of people.There are two major dates here: the sponsored registration deadline, and the reconfirmation deadline. There is also the date of opening registration, but that's usually more a matter of "how soon can we?" than "when should we?"The sponsored registration deadline (April 15 for DC10) is the deadline for requesting sponsored accommodation, food, or travel. This allows us to figure out how much we'll have to pay for attendees - attendees who register after this point should pay enough to cover their own costs.The reconfirmation deadline (June 10th for DC10) is used as a ping for attendees, particularly the sponsored ones. Attendees must reconfirm they will attend, so that we don't waste money reserving more food/rooms than we will have people using them. This deadline usually depends on the date we must give firm numbers to our accommodations.Historically, registration hasn't ever closed, anyone can come, just if you register too late you won't necessarily get any free stuff (and definitely no free food/accommodation). The deadline for food and accommodation themselves has been based on the logistics of the conference itself - DC9, when we ran out of physical rooms for accommodation, and DC10, when we had to give firm housing numbers to Columbia. Food has tended to be easier as caterers could take updates on shorter notice, usually the day before.DebConf8 began a system where attendees could pay a registration fee if they desired. This was encouraged, for example, for attendees whom could get this fee reimbursed by their company, but was not required nor encouraged of sponsored attendees. DC8-9 had two classes of fees: Professional, covering the actual costs, and Corporate, covering about double of professional and for companies or registrants whom could provide extra financial support for DebConf. DebConf10 added an extra layer of "pay per day" on top of this.It's up to the registration team to pin down attendee numbers as early as possible, as this is usually the biggest cost. DebConf provides food and accommodations to a huge number of people, but the numbers are constantly changing. It's hard to know how many people will drop out at the last minute. It's hard to know how many people will actually pay a corporate or professional fee, as opposed to mistakingly selecting it when they'd rather something else. Our registration system can't support all the little intricacies of how much an attendee will cost us, or pay us. Without knowing how many people will attend, how can we accurately get the right number of meals and right number of rooms? Sponsored people may always choose to not eat some meals with us, possibly saving us money there. How many people with travel sponsorship will actually request their full reimbursement?The registration system, Pentabarf, is used to track attendance numbers, but it isn't completely reliable. Not everyone in it will attend all days, and most people who aren't staying with us don't have accurate dates of attendance. Thus, it can only be used as estimates, not for all decisions.
The difficulties above are challenges for the registration and budgeting teams, but are not too bad - they just require thought, time, and answering many questions from attendees.Tags: debconf

After a long time, I decided to write again. I decided to start with software I wrote for my own usage that could be usefull for other people. I'll start with python-dvdvideo, a DVD-Video reader written in Python 3, and the reference tool dvd-video-backup-image, a generic DVD-Video dumper. Lets see, if this blog will see more postings in the future.

It looks a lot as if Facebook is closing up on third party developers. Seems like they've cherry picked the features they want to copy, and now are no longer interested in third parties anymore:

• They removed the "publisher" feature, which allowed applications to offer custom story forms. They said it wasn't used a lot - the reason was simple, they placed third party publishers in a tiny dropdown in the "attach" section, second class to their link sharing and even crappy 'gift' application. Of course: otherwise nobody would buy their birthday "gifts".
• They're removing the extra info boxes you could put into your profile. This is a key feature for two of my applications, which allowed people to publish a calendar feed in their profile.
• Places is aiming at killing FourSquare and similar on the long run.

Expect once-famous Facebook applications such as SuperPoke to disappear on the long run - their integration hooks have been pulled. The API gets more and more limited for the cash cow use case: games. And of course marketing. Oh, I forgot one type of "applications". The one that basically embeds you regular web page within a Facebook tab. Wow. That's technology!If it was really about the users, Facebook would seriously fight Facebook spam, worms and fraud. They still don't have that under control (and none of them used either of the removed features!). They do even less to prevent users from clicking on such bad links such as the Free iPad scam, which has been around for at least a week now. Or the famous "dislike button" scam. They could not spell much more clearly that they do not care about their users - they just want your data to sell more directed advertising to you. Forget about privacy, too!Now that all my three Facebook applications are pretty much dead, this comes with a good side for me: I've been considering to disable/suspend/kill my Facebook account for quite some time. But I wasn't sure if that would also kill my applications for their users. Now that they're dead anyway, there is not much keeping me anymore.

My hard drive does not currently make any additional sense.