At the end of September, things were quite exciting as we had our first project meeting. At that meeting those in the room saw a demonstration of the Moonshot GSS EAP mechanism and we discussed a number of open issues and began to plan for our test infrastructure. We ve made significant progress on the specification front and on explaining Moonshot to important communities since then. However there has been little public progress on the implementation front. Unfortunately, getting the necessary legal clearance and agreements to release code often takes longer than anyone would like; that is what is happening here. We re all eagerly awaiting final approval from the lawyers and JANET(UK) management. However, things have been moving behind the scenes. Throughout much of October, Luke Howard and Linus Nordberg were working on their respective parts of the code. I ve also been working on putting together the test and build infrastructure. As we discussed at the meeting, we re going to use Debian and Ubuntu as the basis for our testing. For example, we hope to release virtual machine images for these platforms for the major Moonshot components. Thus the primary build environment for our testing and virtualization will be for Debian. I ve been putting together that here. Right now, that branch will pull together packages of the SAML infrastructure that we need. I ve also been looking into virtualized test frameworks and believe I ve found one that meets our needs. I ve also put together some primitive build infrastructure that is independent of packaging available here. I ve set up a buildbot that builds both environments. So, as the code becomes available we ll be in a good position to start making it available.

Paul Wayper has written a blog post criticising the main chains of grocery stores in Australia [1]. That is the cartel of Coles and Woolworths (which was formerly known as Safeway in some parts of Australia), and the German newcomer Aldi.While he does make some good points I think he was a little harsh on Aldi. As a portion of the stock on shelves (by quantity stocked, shelf space, number of products, or almost any other metric) it seems that Aldi easily beats every other decent sized grocery chain for the portion of the stock being organic food. Aldi also has a very high portion of their stock being fair-trade also probably beating the other chains handily. Having organic fair-trade food sold in bulk at low prices is a really good thing!Paul criticises the lack of range on offer in the major chains. Aldi has the smallest range as their entire business model is based around offering a small range of products in a relatively small store at very low prices. That dramatically decreases the amount of stock that they have and therefore decreases the amount that is spoiled and the expense of storing and tracking it all. I wonder if the success of Aldi can drive other stores to getting a better range of products. Why buy a small range of products from Coles or Woolworths when you can get a similar small range from Aldi at low prices that the business models of Coles and Woolworths can t match? This may even drive consumers to buying core products (bread, milk, pasta, etc) and some basic luxury products (cooking wine, basic chocolate, cheap soda, etc) from Aldi and then buying other products from specialist stores. This might result in more delicatessens!Aldi also has a weekly selection of special items. For example selling snow boots etc just before winter starts. In most parts of Australia we don t see snow much and a cheap pair of snow boots is a good investment for the one day a year that most people spend at the snow. These specials provide real benefits to consumers although not to the Australian economy.Paul complains about own brand products. But for some products such as sugar you will get the same product from home brand as from other brands (there aren t really many options for sugar) and it ll probably end up being Australian just due to economics.When you buy big-name products you are mostly paying for marketing. For some things there are real taste differences (such as Coke for me), but for lots of things there are no difference, so I m happy to buy own brand . Generally not paying for marketing is a good thing.I totally agree with Paul s point about buying Australian to help the local economy and also to avoid the transmission of diseases. Any treaty which forces us to accept the import of products from countries that have diseases that can be transmitted to Australian crops should be voided. It seems that the WTO is specifically designed to benefit corporations at the expense of member countries and their citizens.Paul gave a positive review to IGA, I wasn t impressed with my local IGA the last time I visited it. But I ll give it another go.

• [1] http://www.mabula.net/tbfw/blosxom.cgi/2010/11/29#2010-11-29-scarcity-in-abundance

AltOS 0.8 New Software and Firmware for Altus Metrum Devices Bdale and I are pleased to announce the release of AltOS version 0.8. AltOS is the core of the software for all of the Altus Metrum products. It consists of cc1111-based microcontroller firmware and Java-based ground station software. AltosUI New Features in the AltusMetrum Ground Station AltOS version 0.8 contains significant upgrades to the ground station software, AltosUI:

• Post-flight graphing tool. This lets you explore the behaviour of your rocket after flight with a scroll-able and zoom-able chart showing the altitude, speed and acceleration of the airframe along with events recorded by the flight computer. You can export graphs to PNG files, or print them directly.

• Real-time moving map which overlays the in-progress flight on satellite imagery fetched from Google Maps. This lets you see in pictures where your rocket has landed, allowing you to plan recovery activities more accurately.

• Wireless recovery system testing. Prep your rocket for flight and test fire the deployment charges to make sure things work as expected. All without threading wires through holes in your airframe.
• Optimized flight status displays. Each flight state now has it s own custom tab in the flight monitoring window so you can focus on the most important details. Pre-flight, the system shows a set of red/green status indicators for battery voltage, apogee/main igniter continutity and GPS reception. Wait until they re all green and your rocket is ready for flight. There are also tabs for ascent, descent and landing along with the original tabular view of the data.
• Monitor multiple flights simultaneously. If you have more than one TeleDongle, you can monitor a flight with each one on the same computer.
• Automatic flight monitoring at startup. Plug TeleDongle into the machine before starting AltosUI and it will automatically connect to it and prepare to monitor a flight.
• Exports Google Earth flight tracks. Using the Keyhole Markup Language (.kml) file format, this provides a 3D view of your rocket flight through the Google Earth program.

Continuing Features AltOS version 0.8 continues to provide the following features:

• Receive and log telemetry from a connected TeleDongle device. All data received is saved to log files named with the current date and the connected rocket serial and flight numbers. There is no mode in which telemetry data will not be saved.
• Download logged data from TeleMetrum devices, either through a direct USB connection or over the air through a TeleDongle device.
• Configure a TeleMetrum device, setting the radio channel, callsign, apogee delay and main deploy height. This can be done through either a USB connection or over a radio link via a TeleDongle device.
• Replay a flight in real-time. This takes a saved telemetry log or eeprom download and replays it through the user interface so you can relive your favorite rocket flights.
• Reprogram Altus Metrum devices. Using an Altus Metrum device connected via USB, another Altus Metrum device can be reprogrammed using the supplied programming cable between the two devices.
• Export Flight data to a comma-separated-values file. This takes either telemetry or on-board flight data and generates data suitable for use in external applications. All data is exported using standard units so that no device-specific knowledge is needed to handle the data.
• Speak to you during the flight. Instead of spending the flight hunched over your laptop looking at the screen, enjoy the view while the computer tells you what s going on up there. During ascent, you hear the current flight state and altitude information. During descent, you get azimuth, elevation and range information to try and help you find your rocket in the air. Once on the ground, the direction and distance are reported.

AltOS Firmware Update AltOS version 0.8 contains a minor firmware update for TeleMetrum to resolve an issue with main deployment. A mis-feature in the igniter firing code would delay main deployment by 2 seconds in some cases. Thanks to our contributors! We had a lot of help with this release:

• Anthony Towns wrote both the new data graphing interface and the moving map display.
• Bob Finch helped clean up our documentation, and provided flight testing for the firmware updates.

Future Plans A number of features are implemented or in process in the sources available in our publicly visible repository that are not part of the current stable release.

• A Kalman-filter based approach to apogee detection using more than just the baro sensor, so that we can safely control apogee ejection on flights to altitudes beyond the range of our baro sensor alone. Unlike the other items on the list, this will be a significant change to the in-rocket TeleMetrum firmware. It may therefore be a while before this becomes part of a stable firmware release.
• Motor characterization. Because TeleMetrum contains a high-resolution, high-frequency accelerometer, it is possible to take the data from that and compute an accurate thrust curve for the motor.
• Comprehensive PDF and/or HTML -based flight report. Construct a complete report of the flight suitable for publication on the web that includes graphs of the flight and details about motor performance etc.
• Publish flight data to the Altus Metrum web site. This will allow you to share your flight data with others, and let you download flights published by others.

There are any number of additions that could be made to this list; feel free to send along ideas that you ve got. Of course, all of this software is licensed under the GNU General Public License, so you can get the source and hack on it in the comfort of your own home.

Ever wondered how to easily convert e.g. 10 seamiles into kilometres? Use units:

$ units 
2411 units, 71 prefixes, 33 nonlinear units
You have: 10 seamiles
You want: km
        * 18.288
        / 0.054680665
You have: ^C
$

Of course this is interactive. There s also a non-interactive mode:

$ units '10 seamiles' 'km'
        * 18.288
        / 0.054680665
$ 

The line with the asterisk means that 10 seamiles are 18.288 kilometres or 1 kilometre is the 0.054680665th part of 10 seamiles. This quite non-intuitive output is caused by the fact that unit is designed to be used with units only:

$ units seamiles km
        * 1.8288
        / 0.54680665
$

Now this makes more sense: You have to multiply seamiles with 1.8288 to get kilometres and you have to multiply (not divide) kilometres with 0.54something to get seamiles. But this output is still a little bit cumbersome, and annoying if you want to use it in shell scripts. But for luck, units knows some nice options, especially -v ( verbose ) and -t ( terse ):

$ units -v '10 seamiles' km
        10 seamiles = 18.288 km
        10 seamiles = (1 / 0.054680665) km
$ units -t '10 seamiles' km
18.288

Now that s way easier to read and script! You can also script more complex things like

$ units -v '100 attoparsec/microfortnight' m/s
        100 attoparsec/microfortnight = 2.5509901 m/s
        100 attoparsec/microfortnight = (1 / 0.39200466) m/s

Unfortunately not all common units are unambiguous for units:

units -v '100 km/h' m/s
conformability error
        100 km/h = 1.5091905e+38 s / kg m
        m/s = 1 m / s

Well, h seems not to be units unit for hours , so lets tell it explicity that we want km per hour:

units -v '100 km/hour' m/s
        100 km/hour = 27.777778 m/s
        100 km/hour = (1 / 0.036) m/s

Looks more like what I expected. units behaves though a little bit strange when I try to convert litres per 100 kilometres into miles per gallon:

$ units -t '6L/100km' 'mpg'
conformability error
6e-08 m^2
425143.71 / m^2
$

Interestingly changing the verbosity helps already in this case:

$ units -v '6L/100km' 'mpg'
        reciprocal conversion
        1 / (6L/100km) = 39.202431 mpg
        1 / (6L/100km) = (1 / 0.025508622) mpg

Greetings from the Debian booth at LinuxDay.at and thanks to Y_Plentyn, rhalina and bzed for example ideas and the suggestion to write a blog posting about units. :-)

As phpMyAdmin is heading closer to 3.4 release, we would like to invite translators to be able to bring this version localized as much as possible. We don't expect any message changes to happen in 3.4 anymore, so it's best time to start translating right now. You can find more information about translating on our website, but you can as well start translating right now at translation server. There are so many languages, but majority of languages are really in bad shape right now. This is your chance to contribute to the free software!

Filed under: Phpmyadmin 0 comments Flattr this!

Okay, so it's been five months since the last update. Sorry about that. I guess I could force myself to more regular updates. What have you missed? Only the branching of what will eventually (soon?) become FusionForge 5.1. Quoting from the default home page, and in no particular order:

• New Funky theme.
• New UI and features for the document manager (download as zip, locking, referencing documents by URL).
• New progress bar displaying completion state of trackers using a custom status field.
• Improved sorting in trackers.
• More flexible and more powerful role-based access control system.
• New unobtrusive tooltip system based on JQuery and Tipsy to replace old help window.
• New plugins: Blocks, to add free HTML blocks on top of each tool of the project; Gravatar, to display user faces; OSLC, implementing the OSLC-CM API for tracker interoperability with external tools.
• scmgit plugin: Personal Git repositories for project members.
• Simplified configuration system, using standard *.ini files.
• Reorganised, modular Apache configuration.
• RPM packaging.

We've also added many tests to our testsuite, and we'd welcome more. If you're interested in the new features coming to FusionForge, now would be a good time to try them and report the bugs you find. And if you don't want to upgrade your production server just yet, you can use a VM image that builds and installs everything automatically inside a VirtualBox (or another virtualization system). See details of my VM image for easy testing mailing-list post. That VM covers trunk rather than the 5.1 branch, but since the branching is still rather recent there isn't much difference yet. Another tidbit of information: since I recently realized the IPv4 horizon is less than a hundred days away, I've been engaging in a flurry of migrations from IPv4-only to dual-stack, one result of which is that fusionforge.org is now accessible via IPv6. Yay for less NAT and reverse-proxying. And I think that's it for now. There's one obvious piece of news I'm looking forward to announce, but it has to actually happen first

For the Freaky Wall I have installed a Debian GNU/kFreeBSD system using the installer with ZFS support announced on: http://robertmh.wordpress.com/2010/09/06/debian-installer-with-zfs/ I used the mini.iso found on: http://people.debian.org/~rmh/zfs/kfreebsd-amd64/monolithic/mini.iso the 12th of October of 2010; as I had some problems and reported them to Robert is possible that the current image solves part of them. Installation plan I did a standard installation on a machine with two hard disks, but only used the first one from the installer. The plan was to use ZFS with RAID-1, but current versions of grub do not support booting from a ZFS + RAID file system, so I had to use the same technique used for Linux for a long time; three partitions: a swap partition, a small /boot partition and a big partition for /; / and /boot were formated to use ZFS. First reboot After the installation the system failed to boot because of a bug when building the /boot/grub/grub.cfg (some routes were missing a //@ prefix); to be able to boot Iwe edited the config on the grub prompt and later fixed the file:

    --- grub.cfg.orig       2010-10-13 16:40:39.000000000 +0200
    +++ grub.cfg    2010-10-13 18:38:47.535436766 +0200
    @@ -64,7 +64,7 @@
            set root='(hd0,1)'
            search --no-floppy --fs-uuid --set a371979bb836d1fe
            echo                    'Loading kernel of FreeBSD 8.1-1-amd64 ...'
    -       kfreebsd                /kfreebsd-8.1-1-amd64.gz
    +       kfreebsd                //@/kfreebsd-8.1-1-amd64.gz
            insmod part_msdos
            insmod zfs
            set root='(hd0,3)'
    @@ -75,7 +75,7 @@
            insmod zfs
            set root='(hd0,1)'
            search --no-floppy --fs-uuid --set a371979bb836d1fe
    -       kfreebsd_module         /zfs/zpool.cache type=/boot/zfs/zpool.cache
    +       kfreebsd_module         //@/zfs/zpool.cache type=/boot/zfs/zpool.cache
            set kFreeBSD.vfs.root.mountfrom=zfs:dkfbf1-ad4s3
            set kFreeBSD.vfs.root.mountfrom.options=rw
      

I haven't tested the installer since that day, but I believe that the current ZFS installer was fixed by Robert to deal with that problem. Once the system was booted I had to fix a couple of things:

• The keyboard configuration was wrong, but it was easy to fix the Debian Way:

  dpkg-reconfigure kbdcontrol
  

• The /boot partition was mounted on /target/boot, as that was what was recorded on the ZFS file system; to fix it I executed the following commands:

   # zfs set mountpoint=/     dkfbf1-ad4s3
   # zfs set mountpoint=/boot dkfbf1-ad4s1
  

  Where dkfbf1-ad4s3 is the root file system and dkfbf1-ad4s1 is the original /boot. I reported that to Robert also and I believe it is fixed on the ZFS installer now.

Adjusting ZFS to do RAID-1 On the second disk I created the same partitions as the ones on the first disk using parted; the final result was:

    # parted -l     
    Model: ST3250620NS/3BKS (ide)
    Disk /dev/ad6: 250GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Number  Start   End     Size    Type     File system  Flags
     1      1049kB  256MB   255MB   primary
     2      256MB   4256MB  4000MB  primary
     3      4256MB  250GB   246GB   primary
    Model: ST3250620NS/3BKS (ide)
    Disk /dev/ad4: 250GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Number  Start   End     Size    Type     File system     Flags
      1      1049kB  256MB   255MB   primary
      2      256MB   4256MB  4000MB  primary  linux-swap(v1)
      3      4256MB  250GB   246GB   primary

To use the second partition of both disks as swap I added the following to /etc/fstab:

    /dev/ad4s2      none            swap    sw              0       0
    /dev/ad6s2      none            swap    sw              0       0

To configure the mirroring for the root file system I did the following:

    zpool attach dkfbf1-ad4s3 ad4s3 ad6s3

As the /boot can't work as a replica I adjusted it to make two copies of every file:

    zfs   set copies=2     dkfbf1-ad4s1

Leaving the second disk copy alone, although my plan is to configure it to hold a copy of the /boot partition synchronized with rsync each night. After all those changes the system didn't boot, as the grub-pc generates a buggy /boot/grub/grub.cfg; the problem is on the /etc/grub.d/10_kfreebsd section:

    ### BEGIN /etc/grub.d/10_kfreebsd ###
    menuentry 'Debian GNU/kFreeBSD, with kFreeBSD 8.1-1-amd64' --class debian \
      --class gnu-kfreebsd --class gnu --class os  
          insmod part_msdos
          insmod zfs
          set root='(hd0,1)'
          search --no-floppy --fs-uuid --set a371979bb836d1fe
          echo 'Loading kernel of FreeBSD 8.1-1-amd64 ...'
          kfreebsd /kfreebsd-8.1-1-amd64.gz
          set kFreeBSD.vfs.root.mountfrom=unknown:/dev/ad4s3
          set kFreeBSD.vfs.root.mountfrom.options=rw
     
    ### END /etc/grub.d/10_kfreebsd ###

To fix it there has to be a copy of the modules for ZFS on the boot partition (in my case I moved the /lib/modules directory to /boot and created a link on the root partition to the new directory):

    cd /boot
    mkdir lib
    mv /lib/modules lib
    cd /lib
    ln -s ../boot/lib/modules

And instead of fixing the /etc/grub.d/10_kfreebsd code I wrote a new script (/etc/grub.d/09_zfs_kfreebsd) that creates the right config for my current configuration on the grub.cfg file:

    #!/bin/sh
    prefix=/usr
    exec_prefix=$ prefix 
    bindir=$ exec_prefix /bin
    libdir=$ exec_prefix /lib
    . $ libdir /grub/grub-mkconfig_lib
    prepare_boot_cache="$(prepare_grub_to_access_device $ GRUB_DEVICE_BOOT    sed -e "s/^/\t/")"
    kfreebsd_versions="$(ls /lib/modules/)"
    zfs_root_device="$(zfs list   awk '/\/$/   print $1  '   head -1)"
    for kversion in $kfreebsd_versions; do
      cat << EOF
    # Entry when using ZFS (we have issues with /etc/grub.d/10_kfreebsd)
    menuentry 'Debian GNU/kFreeBSD, with kFreeBSD $kversion and ZFS' --class debian --class gnu-kfreebsd --class gnu --class os  
    $ prepare_boot_cache 
          echo                    'Loading kernel of FreeBSD $kversion ...'
          kfreebsd                //@/kfreebsd-$kversion.gz
          kfreebsd_module_elf     //@/lib/modules/$kversion/opensolaris.ko
          kfreebsd_module_elf     //@/lib/modules/$kversion/zfs.ko
          kfreebsd_module         //@/zfs/zpool.cache type=/boot/zfs/zpool.cache
          set kFreeBSD.vfs.root.mountfrom=zfs:$zfs_root_device
          set kFreeBSD.vfs.root.mountfrom.options=rw
     
    EOF
    done

I solved the problem this way to have a working solution that does not break with squeeze upgrades, assuming that a future grub-pc package will deal well with my config and I'll be able to remove this script, but I guess I'll have to install it from backports. The entry generated by the script when called from update-grub will be similar to:

    ### BEGIN /etc/grub.d/09_zfs-kfreebsd ###
    # Entry when using ZFS (we have issues with /etc/grub.d/10_kfreebsd)
    menuentry 'Debian GNU/kFreeBSD, with kFreeBSD 8.1-1-amd64 @ ITI' --class debian --class gnu-kfreebsd --class gnu --class os  
          insmod part_msdos
          insmod zfs
          set root='(hd0,1)'
          search --no-floppy --fs-uuid --set a371979bb836d1fe
          echo                    'Loading kernel of FreeBSD 8.1-1-amd64 ...'
          kfreebsd                //@/kfreebsd-8.1-1-amd64.gz
          kfreebsd_module_elf     //@/lib/modules/8.1-1-amd64/opensolaris.ko
          kfreebsd_module_elf     //@/lib/modules/8.1-1-amd64/zfs.ko
          kfreebsd_module         //@/zfs/zpool.cache type=/boot/zfs/zpool.cache
          set kFreeBSD.vfs.root.mountfrom=zfs:dkfbf1-ad4s3
          set kFreeBSD.vfs.root.mountfrom.options=rw
     
    ### END /etc/grub.d/10_iti-kfreebsd ###

And after rebooting the machine with this new configuration the system boots OK. On my next post I'll continue explaining how to compile a kernel that supports the use of the OpenBSD Packet Filter and related technologies (CARP, pflog, etc.).

This post and the next to come are about a project I'm doing at work that I've called The Freaky Wall. The project has its origins on the idea that using multiples technologies is better for security; almost all the servers I use are running Debian GNU/Linux and use iptables locally, so when I decided that we had to build new firewalls at work I thought it was a good idea to look at different technologies, that is, a different kernel and firewalling tools. As I wanted to avoid iptables and the Linux kernel my first idea was to go after the free BSD systems (FreeBSD, OpenBSD or NetBSD), and soon realized that pf (the OpenBSD Packet Filter) was the way to go; it has a clean syntax and includes advanced features like CARP and pfsync that allow me to build redundant firewalls. Before going after the standard systems I looked at pfSense a firewall appliance built on top of FreeBSD that uses a PHP interface to do everything. At first it seemed that it was going to be a good option, but soon I felt that I wasn't in control of what the system was doing and I had to change the PHP code to do trivial things (I wanted to configure IP aliases on a CARP interface and it was not possible with the web interface, while it is trivial to do using the standard system configuration files), so I left the idea of using it. The second option was to use OpenBSD directly, as it is the system were pf has been developed. Soon I saw that I was going to be able to do what I wanted with the system, but I missed the Debian's way of installing and upgrading the system and the list of packages available. For different reasons the firewall project was left in a limbo for a little while and when I went back to it I already had to upgrade my test systems to a new OpenBSD release; after reading a little bit about how to upgrade and not liking the idea of doing it I remembered that jordi suggested that if we only want the kernel and the firewall tools the Debian GNU/kFreeBSD port could be an option instead of OpenBSD or FreeBSD. Before trying to install the Debian GNU/kFreeBSD system I saw Robert's post about a Debian installer with ZFS support and I decided to start with it, as the use of ZFS will allow us to use software RAID-1 and snapshots, something we have on almost all our Linux servers (we use software RAID for redundancy and LVM snapshots to be able to do our backups at any time of the day with consistent data, but that is for another post). On my next post I'll explain how I did the initial installation with ZFS, and after that I'll explain the changes I did to the kernel and some of the packages to be able to build a firewall as described on the Firewalling with PF document (that is, I needed pfctl, pflogd, a tcpdump with pflog support, the pf's ftp-proxy, etc.) and on the last document I'll explain how I've configured the firewalls The Debian Way .

Can I count myself in a majority of people that feel that, while the USB standard is a great advance regarding the way we used to connect our stuff to the computer some years ago (when we used a different cable for every friggin' device we had, and we had to care about having only one parallel and two seral ports in regular configurations, and don't even get me started on port settings speed, parity, etc.), every time we hold a USB cable in our hand we feel one of the designer teams decided to play a prank on humanity by making the connector's orientation basically unguessable? By the frustration this square thingy inflinges on us (being any other decent cable used on a computer either have a visually recognizable shape or just round and thus omnidirectional) I think that the rate by which I get the connector with the correct side up (or front, or right, or whatever) is way lower than 50%. Anyway, I found that the standard does provide for a (IMO quite dubious but still better than guesswork at the port) way to distinguish which way up - Quoting from Wikipedia's entry on USB connectors:

( )The side of the connector on a USB cable or other product with the "USB Icon" (trident logo) should be "visible" to the user during the mating process.( ) Officially, the USB 2.0 specification states that the required USB Icon is to be "embossed" ("engraved" on the accompanying diagram) on the "topside" of the USB plug, which "provides easy user recognition and facilitates alignment during the mating process."

There are several caveats which make this less than ideal, some of which you already thought of, some others are in the referred article and I won't reproduce them. I am just amazed that... USB has been around for almost 15 years (and has been the most popular connection type for ~10 years). And I have never seen anybody apparently knowing this rule. This was a public service announcement in the hopes you will be happier people armed with its knowledge.

Vettel made it F1 World Champion 2010.
Filed under: Uncategorized

In my professional role as Ubuntu CTO, I take on a number of different perspectives, which sometimes compete for my attention, including:

• Inward supporting the people in my department, alignment with other departments in Canonical and reporting upward
• Outward connecting with customers, partners and the free software community, including Debian
• Forward considering the future of the Ubuntu platform and products, based on the needs of their users, our customers and business stakeholders within Canonical
• Outside-in taking off my Canonical hat and putting on an Ubuntu hat, and looking at what we re doing from an outside perspective

My recent work, as Canonical has gone through a period of organizational growth and change, has prioritized the inward perspective. I took on a six-month project which was inwardly focused, temporarily handing off many of my day-to-day responsibilities (well done, Robbie!). I ve grappled with an assortment of growing pains as many new people joined Canonical over the past year. With that work behind me, it s time to rebalance myself and focus more outside of Canonical again. It s good to be back! In my outward facing capacity, I ll shortly be attending Web 2.0 Summit in San Francisco. I attend several free software conferences each year, but this is a different crowd. I hope to renew some old ties, form some new ones, and generally derive inspiration from the people and organizations represented there. Being in the San Francisco Bay area will also give me an opportunity to meet with some of Canonical s partners there, as well as friends and acquaintances from the free software community. With my head down, working hard to make things happen, it s easy to lose perspective on how that work fits into the outside world. Spending more time with people outside of Canonical and Ubuntu is an important way of balancing that effect. Looking forward, I ll be thinking about the longer term direction for the Ubuntu platform. The platform is the layer of Ubuntu which makes everything else possible: it s how we weave together products like Desktop Edition and Server Edition, and it s what developers target when they write applications. Behind the user interfaces and applications, there is a rich platform of tools and services which link it all together. It s in this aspect of Ubuntu that I ll be investing my time in research, experimentation and imagination. This includes considering how we package and distribute software, how we adapt to technological shifts, and highlighting opportunities to cooperate with other open source projects. My primary outside-in role is as chair of the Ubuntu Technical Board. In this capacity, I m accountable to the Ubuntu project, the interests of its members, and the people who use the software we provide. Originally, the TB was closely involved with a range of front-line technical decisions in Ubuntu, but today, there are strong, autonomous teams in place for the most active parts of the project, so we only get involved when there is a problem, or if a technical question comes up which doesn t fit the charter of an established team. It s something of a catch-all. I d like to re-establish the TB in a more central role in Ubuntu, looking after concerns which affect the project as a whole, such as transparency and development processes. I m also re-joining Debian as a non-uploading contributor, to work on stimulating and coordinating cooperation between Debian and Ubuntu. I m looking forward to working more with Zack on joint projects in this area. This change will help me to support Canonical and Ubuntu more effectively as they continue to grow and change. I look forward to exercising some mental muscles I haven t used very much lately, and facing some new challenges as well.

Someone on Hacker News asks about the future of programming languages, and I realized that it's a pretty interesting topic. Here's my take on it: Programming languages are about people, and the compromises they may or may not make in order to communicate their intentions to computers. The technology in question may change, but there will likely always be people involved, and as long as that's true, looking at the problem through the lense of economics provides useful results. Therefore, looking at programming languages pretty much requires that you look at the economics of them:

• Network effects are important, but obviously don't squeeze out smaller languages entirely. I.e. it's good to be able to get libraries, find people to hire, books to read, friends to chat with about problems, etc... This means that future languages will have to seek out a critical mass of users.
• Since the times of Fortran, languages have been getting more efficient in terms of programmer time, because programmer time stays the same or gets more expensive, while computer time keeps getting cheaper, and language implementors keep getting better at making things fast under the hood. This means that languages of the future are likely to let you do more with less, all else being the same.
• Learning a programming language will continue to be a fairly involved process for most people, meaning that for many, it's more efficient to learn a few languages well and use them for as much as possible, rather than to try and find the 'perfect' language for each problem. This rewards general-purpose languages, compared to niche languages. This also rewards flexible languages that can be used in many different environments.
• Computers are only going to become more and more important, meaning that more people are going to want to develop the skills to work with them. Not all of them will be experts, and many may not know much, but have a vision for how a program could solve a particular problem in a field they are knowledgeable about. This means a low barrier to entry is important, because those novices will pick easier languages, become progressively more competent in them, and thus (network effects) contribute to their diffusion. Case in point: PHP. Future languages should be capable of solving hard problems, but should also make it easy to get started solving simple problems too.
• Working on languages is lots of fun, so even though there aren't necessarily monetary rewards, people will continue to play with and explore new ideas. I don't see there being a limited supply of new languages in the future.
• The costs of switching from established languages will continue to be high, so future languages will still need to interact with "legacy" languages like Cobol, Fortran, C, Java, and so on, that are so widely established in industry that systems written in them will be with us for a long time yet.

I wrote this in a hurry. Have I missed anything? How do these factors apply to languages you see emerging?

Back when Linux kernel 2.6 was released, one of the immediate benefits that I noticed was how beautifully responsive my desktop had suddenly become. As it turned out, Linus figured out that he would create a buzz to accompany kernel 2.6's release by having a default clock rate of 1000Hz. However, the excitement was short-lived and kernel defaults were soon brought back down to a more conservative rate of 250Hz, allegedly because some peripheral chips could not handle such a high frequency bump without producing an increased amount of processing errors. Adding insult to injury, Ubuntu defaults eventually dropped another responsiveness enhancement, kernel preemption, allegedly because the goal was to have a one-size-fits-all kernel shipping with Ubuntu and preemption was detrimental to the kernel performance when the host is used as a server, which was a problem because Ubuntu had somehow decided to shift its focus from the desktop market towards the more lucrative server market. While there is nothing wrong with universally-safe kernel defaults or with a corporate decision to shift a distribution's focus towards more lucrative markets, it nonetheless left me pondering what would be the best way to get a 1000Hz kernel with preemption on my desktops, without constantly having to crank out my own kernel packages. As such, I was wondering if there would be enough traction from desktop users to justify producing such a kernel and to make it available as a post-install option from the Ubuntu repository? PS: I'm already aware that a specialized Ubuntu kernel is available to cater for the needs of audio production, but my understanding is that it has power management disabled, because it can interfere with the real-time operation of MIDI devices, while normal desktop users would definitely want to have power management enabled.

If you want to see a really big Google Calendar, check out this calendar. Essentially, it is all the events from over 100 other calendars (all containing swing dancing events) merged into one, resulting in often more than 50 events every day, occasionally over 20 at the same time). The data is approximately the same that can be seen on this map; in fact I'm trying to move the data keeping from a custom database (which has superior filtering capabilities such as synonyms in text search) to just having Google keep the data for me (meaning it will eventually not require me to have an own web server for this). I'm doing some filtering such as removing events that don't have parseable geo information (I'm adding GeoRSS:where elements, in fact), some text to html conversion of the entry contents and filtering out inappropriate events via some regular expressions (e.g. only including "Swing" not "Ballroom" events from a calendar) There are still a couple of things to resolve. I'm not sure that recurrence exceptions are propagated properly to the combined calendar; the updates could work more incrementally, and I have to resolve some timezone issues due to the global nature of the calendar: I need to make sure the timezone information is preserved, since usually it makes more sense to give the events in local (to the event) time. P.S. sorry, the code is too specialized and needs cleanup, I won't share it yet.

On the drive back home from BelHac I thought about the configuration problem in Haskell: The issue is finding a convenient way to work with values that are initialized once and used in many places all over the code. Assume you have a large module of pure code that, using many custom functions and combinators, parses some data structure. Later you noticed that somewhere far down in the parser, you need to react differently depending on some user preferences say, his preferred language. The usual solution is to add a new parameter to that function and, in consequence, to each and every function that calls or might call directly or indirectly this function. This is often very inconvenient. Other solutions include:

• Using mutable references and some hacking with unsafePerformIO, which always gives the programmer a bad conscience.
• Using a Reader monad, requiring a rewrite of the whole program in monadic style.
• Using implicit parameters which is ok if you did not write type signatures, but if you did, you still have to modify them a lot.
• Some advanced type hackery.

The solution I thought of and implemented uses Template Haskell, the Haskell library to modify code at compile time, to turn the style you prefer to write in (pure code that uses configuration values as if they were global constants) into the style that is semantically correct (pure code with configuration values as an additional parameter). I uploaded the resulting code as seal-module to hackage and added plenty of comments and examples to the SealModule module ( are comments according to ohcount). I refrain from copying that into this blog post, so if you are curious, please continue reading there.

Last week I was on my first Ubuntu Developer Summit (UDS-N), I wanted to attended this summit for couple of reasons: 1. Developer, I m part of Debian Java, working on Eclipse IDE, and as you re developing Debian, you re automatically developing Ubuntu, no? 2. DebConf11 main organizer, I really wanted to see how it s all done on corporate level, for instance we re going with hotels instead of student dorms and so on. 3. Migration to Linux infrastructure, make contacts and have some talks on this topic, because after DebConf11 in Bosnia a lot of institution will most probably open doors to open solutions such as Linux. I went to this summit without any expectations or plans, just said I ll play it by ear. This years UDS was held in Orlando, Florida before Orlando I was planning to stay in NYC for couple of days, therefor my whole plan was to get to NYC, then Orlando then head back home I NYC I spent couple of days here with my cousins and friends, as always NYC never fails to surprise me, and I ll always stop by even if it s only for couple of days I wanted to meet up with couple of Debian people as well, but unfortunately my schedule was too tight so Caribe Royale, Orlando Please note that this whole trip was crazy (in a good way) starting from Sarajevo; on my flight from NYC to Orlando I see a guy that s sitting in front of me is wearing Canonical Landscape tshirt, I approach him and it turns he was Jamu Kakar working on Landscape. Since my luggage was late we had conversations from Debian over Ubuntu to Java to everything pretty much. While we were still at the airport, we bump into two more people going to UDS. As we got to hotel, same moment we walked out of cab there he was Jorge Castro, was he there just to smoke a cigarette or whatever doesn t even matter, what matters is that even before walking into hotel building I knew where reception, my tower and bar was, everything I needed to know at that moment Hotel where everything was held was Caribe Royale and even though I looked at their website I thought there was no way it s all going to be that nice, to my delight it was exact replica. Place is All-Suite Hotel and Convention Center , which meant that both accomodation, venue and everything else was in one place which is 15 minutes later after I checked in, I was at a bar talking to Jim Baker about Java, Jython and Python, exact things I wanted to talk about at that given moment. What I said in these last couple of paragraphs sounds pretty cool eh? Well to me this whole UDS was just like this, everybody was so accessible and open for discussion, from Jono Bacon walking around offering candy to UDS reception staff Michelle, Marianna and others who were always there whether you had a complaint, problem or a praise. I really should stop mentioning names, because I ll forget somebody, and since it s just not fair because I had a feeling everybody was just there for you, whatever it was you needed. Ubuntu Developer Summit (UDS-N) Ubuntu, Debian, DebConf, differences? Even though I said I ll play it by ear, from very beginning I thought I d be complete outsider, being part of Debian and all. Quite contrary, I was everything but outsider, and even more there was plenty of Debian people there, including current DPL Stefano Zacchiroli, Colin Watson, Riku Voipio just to name few. There was also a Debian Healt Check plenary. But overall, there are differences between DebConf and UDS. One of the things is that you can attend UDS remotely I m not sure how much is this doable on DebConf, but then again vast differences are that UDS is filled with BOF s (as we would call it in Debian) meetings of different teams, I attended a lot of these, and most of them end with some kind of conclusion. While each day there was (only?) an hour of lectures on various topics, which on DebConf we would characterize maybe even as speed talks . However, everything was high paced, and it all ended up on You think that s a problem? How do you think we could solve it? . I had a feeling even if I didn t say something out loud that someone would hear me and approach me to discuss it. Everything was straight forward, all in goal to resolve certain issue or a problem, which I absolutely loved. Fun It wasn t all work, even though Orlando doesn t seem to be my kind of city from what I saw, it s way too magical for my taste There was a lot of places where you could go to enjoy yourself. Good people at Canonical even offered me free ticket to Disney World, there was a lot of destinations to visit and transpiration was also organized by Canonical. Unfortunately I missed the notorious UDS party on Friday because I had to head home early, but possibly I ll make it next year Even though, hotel was that great and to be honest you didn t have to leave to look for party. My favorite was Universal City Walk, one night I decided to go to see Blue Man Group, best part of it was that I got pulled up on stage by one of the members and was part of one of the acts! This is definitely something everyone should see during his life, one moment you re laughing like a lunatic, while other moment you re just having goose bumps from the performance itself. Amazing stuff. After all I wrote here along with the pictures, is there even anything else I need to say? I believe that even if I didn t play it by ear, this whole trip would exceed all my expectations in every possible sense. Because that s really what happened, those three reasons I was going to UDS for? Far more happened than that. All I can say in the end is that I ll move closer to having a part in Ubuntu development, and yea, I ll try to be regular on UDS from now on See you in Budapest? If all these photos weren t enough, you can find official group photo as well as personal set of photos can be found on: UDS-N photos by 2010 Sean Sosik-Hamor

It is widely considered good style to include

set -e

near the beginning of a shell script so that it aborts when there is an uncaught error. The Debian policy also recommends this.

Unfortunately, this doesn't work in pipelines. So if you have something like

some_command   sed '...'

a failure of some_command won't be recognized.

By default, the return status of a pipeline is the return status of the last command. So that would be the sed command above, which is usually not the failure candidate you're worried about. Also, the definition of set -e is to exit immediately if the return status of the last pipeline is nonzero, so the exit status of some_command isn't considered there.

Fortunately, there is a straightforward solution, which might not be very well known. Use

set -o pipefail

With pipefail, the return status of a pipeline is "the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully". So if some_command fails, the whole pipeline fails, and set -e kicks in. So you need to use set -o pipefail and set -e together to get this effect.

This only works in bash, so if you're trying to write scripts that conform to POSIX or some other standard, you can't use it. (There are usually other ways to discover failures in pipelines in other shells, but none are as simple as this one, it appears.) But if you are writing bash anyway, you should definitely use it. And if you're not using bash but use a lot of pipelines in your scripts, you should perhaps consider using bash.

(Hmm, it looks like there could be a number of latent bugs in the existing Debian package maintainer scripts, because this issue appears to be widely ignored.)

During my recent trip to Massachusetts for CalConnect XIX I passed back via New York, surfing a few nights on the couch at the Washington Cube Garden¹. This was just enough time for my new Davis VantagePro2 to arrive by UPS ground (phew!) causing me much consternation, as the box was about twice as wide as I expected it to be. On opening it I discovered that the reason for the size was the physical dimensions of the rain gauge, leading me to realise that I had actually bought exactly what I wanted: a high quality weather station. Also leading me to wonder how the hell I was going to get it halfway around the world with me the next day. Unfortunately that extra wide carrying case was in no way going to fit inside my suitcase. Fortunately it came with a handle. And those wonderful people at Air New Zealand gave me a Koru Gold upgrade for a 50th birthday present, so it was time to put it to the test... Travelling on the subway out to JFK was it's own little adventure, but I made it out to terminal 5 and wandered around a bit before concluding I needed to be in terminal 7. Once I was in the right place I found the fancy people's check in counter and the nice lady there was only too happy for me to carry it to New Zealand, though she did aver that perhaps the security scanners would be dubious. Not a hitch there though: I guess they're used to letting everything through that's not specifically denied, so weather stations are fine because nobody would be silly enough to even think of doing that... Being Koru Gold really helped at this point though, because it enabled me to queue jump use the priority boarding line when getting onto the plane, and every airline in the US seems to have gone all-out for this "checked baggage costs more" approach, so the planes all fly with empty baggage holds and totally overstuffed overhead lockers. If this progresses I shouldn't wonder that the planes will get so top-heavy they roll over, but through the miracles of priority boarding I was able to commandeer all the locker space I needed before anyone else had even made it past row 40. I confess that I was a little worried that it wouldn't be quite such plain sailing getting through LAX. I mean we all know what reputation this delightful little exit port has. In the event it was totally anitclimactic. If it has a handle on it, and it's about the same size and shape as a largish carryon then it's fine. There have been no recorded stabbings with wind vanes anyway - at least on aeroplanes - so clearly they are perfectly safe. Finally back in my country of birth, thinking I was home free, I breathed a sigh of relief transferred my bags and nipped off to the Koru lounge for a much needed shower and change of clothes after I snuck the station through another scanner. By now the handle had broken, so it was looking decidedly more 'box' like and less 'carryon' like, and when I eventually rolled up at the gate with my boarding pass the attendant made a valiant attempt to relieve me of it, but she was too slow and I danced around her and swanned up the gangway to take my seat. I was slow, this time, having missed the first boarding call, and all overhead lockers were crammed full. There was no seat in front of me to put the weather station under. Fortunately a nearby cupboard proved fit for purpose and it was secreted there and we were underway. At this point the guy two seats over said "didn't I see that weather station in New York?". Of course being bright yellow does make it a little conspicuous I guess, and I'm not renowned for my own inconspicuousness myself. So that's how I got my new weather station home, and you can see right away that doing things the easy way would never even occur to me. Possibly this is why I run Linux: there just isn't enough challenge when you run a different operating system. On Windows someone has always written a program for your new weather station, and a Mac user wouldn't be seen dead with it because it's black and beige and comes in a bright yellow box! So now I had to find some code to write. I looked around and very rapidly discovered that some software called vproweather was available, which would happily talk to my weather station and download the data from it, write web pages, stuff it into a MySQL database and so forth. All, obviously, far too easy, so I thought aha! I could convert that to PostgreSQL, and then I would be happy and relax with the thought of a job well done. So I downloaded the code in question an impressive several hundred kb of C code, cranked open a text editor and started to crawl through it and see where I could add the magic pixie dust to make it work with PostgreSQL. But it was not to be. My eyes glazed over or scales fell from them or something, and I realised that this particular edifice was beyond my help. Back to square one. I downloaded the Vantage Serial Protocol docs from the Davis Instruments website did some quick googling and decided I should write a module to talk this protocol. And I should write it in Perl (it was a Thursday, and Perl always seems more sensible on a Thursday, and not because of Douglas Adams, either). I beavered away into the night. Rain threatened, so I raced outside and installed the new weather station on the roof in the hopes that it's magnificent rain collector would collect. I raced back inside and was able to count the first (and so far, only) tip of the bucket. And so I now release upon the world Davis::VantagePro my first perl module. Well, the first one I care to share with the world, anyway. I have stressed it with very little testing, lumbered it with no planning or design whatsoever and so I feel it only fair that I should cast it upon the world with very little thought for it's survival or existential goals.

¹ I seem to recall one night we came up with some fantastic names for Micah & Biella's Washington Square apartment, but I couldn't remember any of the good ones and had to slap a new one on there. I should also mention my stay at the Acetarium in Boston, which was a fabulous few days, but it would pad this blog post needlessly, and I've already done that.

So, I m (almost) back from the first edition of MiniDebConf Paris, which was a success thanks to the great organizing skills in panic mode of Carl Chenet and Mehdi Dogguy. I think that everybody is already looking forward to the next edition. I met lots of people, and it was great to be able to finally put faces on names from the french free software community. I gave two talk there, the first one on Debian and Ubuntu (slides), the second one on Debian Quality Assurance (slides), which both went well (I think). I also hacked a bit:

• I added tags and claimed by to UDD Bugs Search.
• I worked on my piuparts replacement (which really needs a sexy name ; I m open to suggestions), processing the results, making some improvements to the script, and filing 16 RC bugs.

If you want to help me, here are some easy tasks that need takers (it s unlikely that I ll be able to tackle them anytime soon):

• One missing thing in UDD Bugs Search is information about the deferred/delayed queue. Unfortunately, the ftp-masters status information is not available in a machine-parseable format.
  • Improve dak code to also export the data in a machine-parseable format (JSON, YAML). I was told the changes are to be done in dak/show_deferred.py.
  • Write the corresponding UDD importer (actually, that s a 30 mins job for me, so I could easily find time to do it if the data was available).
  Update: done!
• Another missing thing is comments (as on bts.turzimmer.net). I don t really have good ideas about how to implement them, so if you want to tackle that with your own idea, go ahead. (It s probably easy to setup a copy of bugs.cgi on alioth if you are not a DD). The CGI is in Ruby.
• Help me process the logs of my piuparts replacement, and file the bugs. I don t have any good scripts to automate bug filings yet. You need a good understanding of packages dependencies and maintainers scripts, but it s a good way to learn, too. (I would need to regenerate the full results first, so talk to me if you are interested)

One of the other things I d like to do is discuss ways to improve the synchronization of UDD (currently, it s synchronized at arbitrary times, while we could sync Sources+Packages just after dinstall, for example).