The cloth is woven, so the tailors are busy collecting measurements. If you've ordered a kilt or whatever, and have not yet sent in your dimensions & money you need to do that within about 24 hours if you're hoping to get something in time for DebConf.We seem to have got about 20 yards more than we ordered (weaving's more of an art than a science) so if you've only just realised that you want a kilt, skirt, whatever, in glorious Debian Tartan, I suggest you get in touch promptly, as even if you don't want it in time for DebConf, I imagine the rest will get snapped up as soon as people see what they've been missing in New York.I'm getting some Ties made -- they'll cost about 25.00 GBP each. If you fancy one, tell me, and add yourself to this table on the wiki. There's more background on that page.

The latest kernel upload (2.6.32-16) brings goodies to SystemTap in Debian. This version has added support for kprobes, on which systemtap has a major dependency, for many of its features.Most of the systemtap instrumentation should work now and all of this will be part of the Squeeze release. Instrumenting the kernel modules still needs some work (DBTS: #555549) but can be done.

Bug-Squashing-Party in Munich We are organizing another BSP in Munich on the weekend of 17th/18th July. Like the last BSP, it will take place in the LiMux office in the city center. See the coordination page for further information and directions and in order to sign up for it. Food and drinks for the event are partly sponsored by the City of Munich and this amount (100 ) has been again matched by credativ GmbH. Contributions by non-DDs are welcome as always; the BSP will be attended by several Debian Developers who will be able to upload fixes. Attending BSPs is a great way to get involved in the Debian community! We cannot guarantee crash space to people from outside Munich at this point, so if you want to attend please contact the Munich list (area-muc@teams.debian.net) beforehand or arrange your own accomodation. We probably start the BSP at some point after 6PM on Friday evening already, but the main action will be on Saturday and Sunday. As usual, people should bring their notebooks and possibly an ethernet cable. Wireless will be present as well, but a certain bandwidth cannot be guaranteed. Bug-Squashing Introduction I also plan to give an introduction to bug-squashing at this month's Garching Debian Stammtisch, which will take place next Wednesday, July 14th in the campus-cneipe (or its beergarden) as usual (the monthly Stammtisch is always on the second Wednesday of the month). If you study/work in Garching and would like to attend the BSP, come to the Stammtisch for a tutorial (and beer)! Debian Packaging Tutorial Some time ago, I gave a tutorial on Debian packaging in the TechTalk series of the Open-Source- School. The audience seemed to be pleased with the talk, and as more people requested to attend than there were seats in the room, we decided to reprise the event on September 7th. Attendence is free, but you have to register (see the event page). There are still some places left at this point.

Currently I own and use an iPhone 3G. I bought it almost two years ago, when the local phone provider LMT started offering the iPhone legally. I had a pretty good experience with it most of the time, but now it is showing its age:

• The two year warranty will run out in September
• The iOS 4 update left out most of the new features my model cann t have multitasking or backgrounds or any of that new cool stuff that is exclusive to the new iPhone 4
• The hardware itself is starting to wear the speaker stopped working a few weeks ago (so I have to use the loudspeaker function or headphones) and also approximately around that time the phone stated to randomly lock up approximately one a week or two it freezes and after a few minutes reboots and demads to be connected to iTunes and restored from backup
• Also the battery does not last as long as it used to

I will try to send my iPhone in for repair hoping to extend its usefulness, but frankly it looks that I might have to get a new phone at or before Debconf. So I am considering my options for a new phone and so far I have 3 main options each with some sub-options. The main options are: iPhone 4, MeeGo and Android. Each choice has its benefits and drawbacks and each also has several sub-options related either to specific phone models or to purchase methods. What I need is a phone with: calling function, contact information synchronised to Google, support for Google Mail/Contacts/Calendar simultaneously with one Exchange account also providing mail/contacts/calendars, music, audiobooks saving my place in the audio book and in the list of books, fully featured Twitter client with Twitpic and location support, encrypted password storage application (preferably open source), GPS with maps, ability to download maps for offline use, Geocaching support with offline cache info and logging, Skype client, ability to write in Latvian, English and Russian in all applications, reliable backup and restore of phones data including settings and data of installed apps. Optional features: last.fm streaming and reporting support, ebook applications with purchasable ebooks, high quality video and photo recording (for a phone), ability for me to write applications on the phone, ssh client, wireless data upload to the phone, ability to change the phones battery without loosing data in the phone (on the go), compass, physical keyboard, confidence in getting software upgrades and developer interest for the next 2 years. I ll try to describe all of the options here to organise my thoughts and maybe also help someone else make the choice. Option 1: iPhone 4 To get the iPhone 4 I have two options: LMT and unlocked. The official arrival of iPhone 4 to Latvia is expected in September and it will likely be sold by LMT for the same prices as iPhone 3GS is now. If I choose this scenario, I will need to survive with my semi-broken iPhone for 2-3 more months. Total price over 2 years (LVL): 131 (purchase price) + 24 * 8 (surcharge) = 323 LVL. Note: the surcharge is the extra cost of the iPhone call plan compared to a comparable regular plan (Vien dais 7 + Internets telefon 5). If I go and purchase an unlocked iPhone 4 in a country where such units are sold (UK is the most easily accessible choice, because a few of my workmates travel weekly to and from UK) then the total price over 2 years would be the purchase price: 499 GBP + 65 GBP (2 year warranty extension) = 505 LVL Now lets go over the general benefits and downsides of the iPhone 4, first the positives:

• It is a simple to use device that I am used to over the two years
• The screen resolution is higher than any other option
• It is smaller and lighter than Nokia N900 and the bigger Android devices
• There is a huge number of free and paid apps as well as accessories for the iPhone docks, headphones, chargers,
• I know that it can do every thing that I need it to do (with apps) and most of the optional features too, except (see drawbacks)

Drawbacks:

• The first thing that I hate about the iPhone is its umbilical cord like link to iTunes. iTunes is the sole purpose I still have a real Windows installation on my main computer (in a Sun VirtualBox, thankfully). iTunes is slow, clumsy and generally unusable piece of dog excrements. It can only do one thing at a time, mostly: if you are adding a folder to your library, you can t switch to another tab and see how the backup of your iPhone is going. The actions do happen in the background, but more often than not you cann t switch away from some actions. If you try to do multiple things in parallel, instead of resolving any process conflicts and queueing the actions iTunes will simply silently fails to do something, often leaving remains of the half-finished actions. It also does not bother to check if the backup or restore of the phone is complete before using it in this way I have many time gotten into a situation where I connect the iPhone to the iTunes, it starts backing it up, then crashes and next time happily offers this half-done backup copy as a valid restore point. Or it crashes during restore and only writes some files and settings to the phone other settings stay at their default values. It is also very cryptic, for example if you have a lot of music it can take hours to import it into iTunes (it spent 2 hours Analysing gapless playback information of 5000 songs) and then you will be puzzled about how to put some of that on the iPhone. If you just tick the checkbox to sync your music you ll get an error stating that there is not enough free space, which is not very helpful. The magic combination was to check Manually manage music and then go to music tab and choose Autofill option. At that point iTunes will spend half an hour choosing which songs to put on the phone, before it even starts copying. And god forbid that you would use that AppStore application on the iPhone to actually install apps sooner or later you will run into a backup bug where creating or restoring a backup of your iPhone might take 2-3 hours instead of more normal 5-15 minutes. Only a factory reset of the phone followed by selective reinstallation of applications (loosing all their settings in the process) can work around this bug currently.
• The closed nature of the iPhone means that I cann t install unapproved software unless I jailbreak the phone and even then it is rather problematic to develop for the phone unless you have a Mac and shell out 99$ a year for participation in the developers program
• The iPhone is a consumption device and not a productivity device. I would like to have a device on which I could be creative as a software developer as well

Option 2: Maemo/MeeGo Some of my colleagues at work (in the Riga office of Accenture) have N900 phones and I have been exposed to people with raw enthusiasm towards the Maemo platform ever since the Debconf in Helsinki, where we saw the first N700 devices in the hands of some lucky Debian/Nokia people using it as an Internet tablet. N900 has been a strong leap forward for the platform, before its head was teleported sideways by the whole MeeGo merger/debackle. Again I have two options here: Get a N900 either here in Latvia for 325 LVL (that would be the total cost over 2 years) or get it from USA during Debconf for 399 USD (+129 USD for 2 year warranty) = 317 LVL. And additional option would be to wait until October/November and then buy the new Nokia MeeGo device, rumoured to be N9-00 and likely to be around the same price as N900 was when it was introduced (around 500 LVL). Benefits:

• A fully open and rooted Debian Linux based device with apt-get, X, pusleaudio, d-bus and Qt as core technologies
• I can install software from anyone and can also write my own software either in Qt/C++ or even in Python
• There is (or was) a significant hacker community that develops applications for N900
• As far as I know most of the features that I need do work on N900, but I am unsure about:
  1. audiobook support how easy it is to put a MP3 on the device and tell it is an audiobbok so that the device would remember position when playing the file and save it even if the player is stopped and the device is rebooted or backed up and restored from backup?
  2. can I have Google Mail, Contacts and Calendar and at the same time also have an Exchange account with mail and contacts and calendar active?
  3. how is the Geocaching.com support is there an app already that makes it easy to geocache while offline?
  4. Multilingual keyboards I ve seen people having to reimplement the on-screen keyboard in their programs, and naturally they do not bother to add support for all language. So support for multilingual keyboard input is an open one.

Drawbacks:

• All the uncertainties above while I could write all the above software, it would be better if I would not have to, so that I could focus on something more productive
• Also the whole position of Nokia on the MeeGo support on the N900 is kinda backwards. I am now used to the Apple way that if I get a device, then it will get software updates for at least next two years and will get the new software features developed in the next two years at least. If Nokia would have said All phone-oriented MeeGo releases will have a version for N900 until at least the end of 2012. Then this drawback would not have been there, but currently it is a mayor sticking stone for me. I don t want to buy a device that will be on a dead-end software platform that will die before the end of this year.
• N900 is the largest and heaviest of all the options

Option 3 Android After looking trough the options for Android phones, I ve narrowed the selection to the ones that are either available from local carriers or can be easily gotten unlocked also the phones need to have announced plans to have at least Android 2.2 version. Currently the choice is limited to HTC Desire (from carrier or unlocked) and Samsung Galaxy S (unlocked). The prices over 2 years break down as follows: HTC Desire (locked to LMT): 69 + 9*24 = 285 LVL
HTC Desire (locked to Bite): 199 + 2*24 = 247 LVL (Note: only 500Mb of data per month available)
HTC Desire (locked to Tele2): 179 + 6*24 = 323 LVL
Note: all the above options would also require me to pay a 45 LVL early termination fee on my current iPhone contract if I choose to do this before September.
HTC Desire (unlocked): 311 LVL
Samsung Galaxy S (unlocked): 350 LVL (import from Germany) Benefits:

• As far as I could find, the features I need are out there in one way or another there usually is an app for that!
• Google looks poised to continue development at a rapid pace
• There is an active hacker community and also an active commercial software community that offers cool applications for a few bucks
• As far as I understand the phone backups onto itself a backup of the phone is created onto a SD card in the phone and you can then copy that off the phone for permanent backup, it is a nice concept
• There are literally thousands of sources for Android applications: from the official Market to Google Code to individual web pages and forums
• The hardware looks solid, powerful and generic enough to last for a couple of years
• It is likely that knowing how to program Android app will be useful for me at work
• It is cheap and simple to start writing Android apps on Linux

Drawbacks:

• Typical usability of Android apps is pretty low compared to iPhone
• The Android environment is very unique and is not like anything else it s not really much of a Linux system from an apps perspective
• I would be relying on HTC to provide OS updates in a timely manner it is quite likely that after a year or so the updates to new version will come slowly or even stop altogether and I will have to use hacked ROMs
• I have a dislike for Java. I wish one could write fully featured Android apps in Python

Please correct me if I am wrong with something!!! And also it would be cool if you expressed your own opinions in the comments. Currently I am very undecided about what I am going to do, but after completing this entry I am leaning towards an unlocked HTC Desire. Update:
Olivier Cr te shares a N900 vs. Nexus One experience Edit 01-07-2010: Handed my iPhone in for a warranty repair, got a dumbphone Nokia as a loaner during repairs. Very surprised about how long a battery can last on a low powered and cheap phone. In the mean time from all the comments here and elsewhere I am starting to see that I am too annoyed with the iPhone platform (mostly iTunes) to stay there and that I am also not convinced in the direction MeeGo is going (a lot of community developers are annoyed and are jumping ship), so N900 is also out. I will need to see 2-3 MeeGo smartphones and also see how Nokia will treat their smartphone users and developers on this platform, in the long term, before I ll be ready to trust them with my money. Therefore, my choice is becoming pretty clear look for the best Android phone (after my iPhone dies). We have this thing at Accenture, where we can get company phones at a discount and HTC Desire is on that list. If I can get that, it would slash 100 LVL (almost a third of the price!) off it, but to get there I either need to get a promotion (rather rare after just one year with the company) or prove the necessity of the phone for the needs of my project (kinda hard currently). Well, the situation will be a bit clearer next week when I get my iPhone back. (Apparently the Latvia s largest holiday Midsummer Festivities or J i/L go with tons of traditional outdoor activities, caused a lot of phones to become broken and there is a backlog of warranty service ) So at this point my plan is: get my iPhone repaired and get a promotion or a project where I can justify a company-paid HTC Desire and then hope that Nokia/Intel really get their stuff together and make MeeGo a great smartphone platform over the next year or two. Update: Got my iPhone 3G back from the warranty . It is in quotes because they just gave me a new iPhone 3G. The battery lasts for 3 days of minimal use and it has not crashed yet. The recovery process was a pain again caused by iTunes: iTunes refused to restore my backup onto the new phone, because my backup was made on a iOS 4.0, while the new phone I got from the warranty has 3.1.3 on it. So I had to set this phone as a new phone, initialise it, make a backup, upgrade to 4.0, restore from the (empty) backup, then reset the phone to factory settings again and only then I could start restoring data from my original (pre-warranty) backup and start copying my data on to the device. All in all it took nearly 3 hours if iTunes doing something and almost a dozen reboots of the phone. Almost half of that time was spent doing completely unnecessary steps to work around the fact that iTunes is braindead. I ll continue using my iPhone for now, keeping my eyes on the newest Android phones and also waiting for the long-promised first MeeGo phone from Nokia.

check-deps.sh
Phil Hands and I devised a little script which is currently packaged as part of multistrap to check the dependencies of a .deb package before installing it - called check-deps.sh and can be found at /usr/share/multistrap/check-deps.sh. I just thought I'd publicise it a bit because it can help with one of my common test cases - when a binary package changes dependencies but you haven't uploaded it yet.

check-deps.sh takes a .deb to the command line -f --file option and outputs the dependencies of that .deb and whether those dependencies (at the requested versions) exist in your apt-cache policy. Optionally, you can use check-deps.sh to also then install those dependencies (not the dependencies that would come from the current version in the repository) and the package itself.

I've found this to be a much more friendly way of satisfying runtime dependencies of a package. Where things like pbuilder would just force install the .deb and then get apt-get -f install to try and fix things, check-deps.sh will fail if the required dependencies are not available in the first place.

The benefits are two-fold. First you get to see the full list and you can easily see if your work to drop a dependency has just resulted in another package bringing that dependency back in etc. Secondly, you can test whether your .deb is installable without breaking your system and then having to remove it - e.g. against backports or a Debian derivative test chroot. It should be possible to extend the script to read Build-Depends but that means reading the input from the source package, not the built binary. Might be just as easy to have a second script for Build-Depends.

(Comments are still disabled due to spammers and time-wasters, but if you would like to see check-deps.sh more widely used or extended it isn't that hard to find my email address at debian.org.)

xapt
Another new perl script, this time using the multistrap and apt-grip core to replace the apt-cross breakage to install cross-dependencies of a Debian source package. xapt has very few dependencies (dpkg-dev, dpkg-cross, apt, perl) and very simple logic. It doesn't try to second-guess which packages to use as dependencies of cross packages, it just gets them all. As such, it still isn't the tool we need for optimal cross-dependency resolution, but then neither is apt-cross - come to that, Multiarch or sysroot aren't going to provide the right tool any time soon either. So xapt at least gets over the worst of the design problems in apt-cross (by not using the apt perl bindings mainly) and seems to just work within clean chroots which is where apt-cross has the most problems.

xapt will be a new binary package, so it will need to go through NEW but I'm looking at using it as part of pdebuild-cross because it is faster than apt-cross and seems to work much more efficiently than apt-cross inside a clean chroot. xapt does not care about multiarch or sysroot, it just gets the packages and passes them to dpkg-cross. Normally, xapt cleans up after itself but you can choose to leave the built packages in /var/lib/xapt/output if that is useful.

Last weekend (2010-07-02 2010-07-04) nine people met at the FAI developer workshop at Linuxhotel in Essen/Germany. If you can t remember: FAI is a non-interactive system to install, customize and manage Linux systems and software configurations on computers as well as virtual machines and chroot environments, from small networks to large-scale infrastructures and clusters. The participants of the FAI meeting: second row from left to right: Michael Goetze, Michael Prokop, Andreas Schuldei
first row from left to right: Sebastian Hetze, Manuel Hachtkemper, Thomas Lange, Mattias Jansson
missing on the picture: Thomas Neumann (left on sunday midday) and Stephan Hermann (only part-time) Friday afternoon started with getting to know each other, continuing with discussions all around FAI. On saturday we started to hack on FAI. Between the hack sessions and discussions the attending people presented their FAI usage and approaches. Some notes from the presentations:

---

FAI Manager webfrontend / Stephan Hermann Stephan \sh Hermann presented his FAI web frontend which should be released under the GPL license in those days. The frontend uses qooxdoo whereas the backend is based on django, rpc4django and python-tftpy. A demo video is available at blip.tv. Currently Stephan is searching for a nice name for his FAI management tool please send suggestions either to him or to the linux-fai-devel mailinglist. Grml / Michael Prokop Grml is a Debian based Linux live system specially made for system administrators. Grml uses grml-live for building the ISOs, whereas grml-live itself uses FAI s dirinstall feature to build the live system. This provides the Grml team with a nice way to autobuild 18 ISOs per day, known as daily.grml.org. Mika also presented Grml s netscript bootoption and the ethdevice bootoption of live-initramfs which is useful for booting Grml/FAI via PXE. Host Europe / Michael Goetze Host Europe uses FAI for installing Debian and Ubuntu (32+64 bit) in the support center. They have ~20 FAI classes and use a Debian lenny NFSROOT as base for all deployed systems. Their main problems with FAI aren t related to FAI itself, but instead e.g. broadcom NICs with lack of support for it in Lenny s kernel. They are not using softupdate (yet) and currently use Kickstart for deploying CentOS but are working on deploying CentOS with FAI as well. LIS AG / Sebastian Hetze Linux Information Systems AG (LIS AG) are using FAI 3.2.17 and provide a luma and PyQt based GUI to their customers. They use DHCP, LDAP and DDNS for inventory, configuration and deployment. Mathematical Institute of the University of Bonn / Manuel Hachtkemper The Mathematical Institute of the University of Bonn uses FAI 3.1.8 and 3.3.5 for managing ~150 systems. They are automatically running softupdates every day, reporting how many hosts actually did run the softupdate and how many didn t run. The involved failogwatch tool supports two regex files, one for excluding specific hosts and the other one for grepping for known problems in the logs. Spotify / Andreas Schuldei + Mattias Jansson Spotify is a peer-to-peer music streaming service and the operating people at Spotify use FAI for deploying the systems. Currently they are using FAI 3.3.3 to deploy ~400 bare metal machines and ~150 virtualised machines. They have their class names in DNS using the txt/Text record entry. They are using a self written prepend_class script to manage dependencies between classes. University K ln / Thomas Lange Thomas uses FAI s trunk version (of course ), managing ~25 machines with less than 20 FAI classes. He s not using softupdates as Lenny s aptitude ignores the hold status of packages (this bug should be fixed for Squeeze). $COMPANY One of the big telecommunication providers in Germany uses FAI 3.3.3 for installing their bare-metal and virtual servers, providing Debian, Ubuntu and SLES. They are using Debian NFSROOT as a base for all systems as well and their main problems with FAI wasn t FAI itself but how to manage installation of virtual machines.

---

On Saturday evening we had a nice barbecue which included beer and K lsch *d&r*. On Sunday we continued with discussions and development. Our work-log of the weekend:

• identified important packages for the Debian/squeeze release
• discussed features that should be available (important packages and bugreports, missing features, )
• fixed several bugs (wrong exit codes, error handling, variable handling, )
• discussed FAI packaging for Ubuntu (packages for lucid are available at ppa at launchpad)
• implemented support for grub2 in fai-cd (will be merged soon)
• implemented initial support for retrieving sources with FAI (available in svn s trunk)

Important decisions made:

• next major release will have version number 4.x
• we want to continue to provide a stable version 3.x (no new features, just bugfixes and maintenance) side-by-side with version 4.x
• deprecated setup-harddisk will be dropped, setup-storage is well established, works fine and is properly maintained by FAI developer Michael Tautschnig (who sadly couldn t attend the FAI developer meeting)
• a FAQ section will be created on the FAI homepage

We noticed that many FAI users implement their own way how to handle dependency management between classes, we will re-consider how we could provide such a mechanism through FAI s core. We also noted that it s important that any self-written scripts used within FAI are fully idempotent and users should be aware of this. Last but not least many thanks to the sponsors of the FAI developer workshop 07/2010! The workshop wouldn t have been possible without our generous sponsors, namely being:

• Debian
• Linuxhotel
• Linux Information Systems AG
• Netways
• Spotify
• Thomas-Krenn.AG

My project. This post assumes that the reader has read my last post. I m posting this weekly report earlier this week because there are too many things to tell already. I ve found the reason behind the segfault I m looking for so much time. The last problem, which is the only I know exactly when was fixed, because the programs started working, was related to allocating two times in the same region of memory. This happened because in each major GC the list of free line groups is generated again, but my old code was still allocating in the same line group of the last generation. So the last part of the line group, which was not yet used, would be a part of a line group in generated in the new collection, and it will be used for allocation two times: one in the allocation of the current line group, and another when this new line group starts being allocated. The implementation of the allocation of memory in lines is not very complicated, but it has some details that should be paid attention, and that were the cause of most trouble last weeks, and still need improvement. Initially I was allocating one object per line, just to see if it would work. As it didn t, I kept on improving the approach until I could find the problem. The next attempt was by setting ws->todo_free and ws->todo_lim in alloc_for_copy() in rts/sm/Evac.c. I think this is not ideal, because I didn t want the code to become too inconsistent with the way memory was allocated using these pointers before my changes. So I created new variables, line_free and line_lim, at first in the gen_workspace ws, the same place that todo_free and todo_lim are, but because of the last problem I described in the previous paragraph I changed it to generation. I m still not sure about where to place these pointers, this is something that can be improved. Another problem that I took a long time to understand is that the object need to be scavenged after being allocated. When it was allocated in todo_free, it was being scavenged by scavenge_block() in rts/sm/Scav.c, because the block in which it s in, todo_bd, is scavenged by this function. As I didn t wanted to the whole block where the free line group is to get scavenged again, I didn t want to send it to this function. So I thought about creating a way to scavenge only part of a block, that is, the space in the free line group that was allocated. This is still a valid idea, but I noticed that it was easier to use the mark stack. So I mark the object that is allocated in the line and push it in the mark stack. The main problem with this approach it s only possible to allocate in lines during major GCs, since only in this kind of GCs the mark stack is active. This is certainly the place where I can make more improvement. The patch of these changes and another one for the sanity checking explained in the last post. I m now benchmarking these changes with nofib, to see how much it affects the performance.

Following the update of antlr3 in unstable, I have now uploaded forked-daapd to Debian. For now, and until the new version of antlr3 is ready to migrate to testing, forked-daapd will only be available in experimental. I have also uploaded the ANTLR v3 C runtime (aka libantlr3c) to unstable. It s currently blocked from migrating to testing, until antlr3 is ready to migrate. Hopefully, the issues affecting antlr3 3.2 in unstable will be resolved soonish and all of this will migrate to testing and be part of Squeeze.

Some time ago, someone posted to a development mailing list about a piece of code, Cc:ing some people who had modified the code in recent past. Within hours, the original author replied to complain about not being Cc'd. Several thoughts went through my mind:

• Is this man a walking colostomy bag?
• Why is this clown proud of his code? It is sloppy and riddled with bugs, and half of the changes made to it by other people have been obvious fixes.
• Why did his parents not raise him better? Doesn't he realize that demanding recognition in this way makes him a bad person?

This is Free Software, and people are motivated by all kinds of factors. Some bad reasons include

• Recognition, praise, or other Community ego-stroking
• Gaining a competitive advantage for one's for-profit business
• Line items on one's CV
• Finding a husband
• Accumulating increased influence in other communities

Looking at it through an Open Source lens, all these bad reasons are good reasons. For whatever motives, people are getting work done and accomplishing things. Of course, there are always an irritating number of hangers-on, and people who do not actually achieve much but are filled with such self-importance that they believe they deserve to be highly influential through sheer force of will, and the majority of the Community will concede this influence to them due to a variety of flaws in the Human Condition, but I digress. It would be nice if everyone had motives that were pure and good, especially since at least 90% of problems in Free Software projects are due to ego issues, but when the world works, the world works. So I am happy to get code contributions for whatever reason (assuming there are no strings attached). The problem I have is when people insist that their bad reasons are legitimate and should be catered to and accommodated by others.

I am staring at three different plug computers and wondering how many more will appear. They are in varying states of Debianness: all are running squeeze, but the GuruPlug is the only one that's pure . The SheevaPlug is running a wacky version of U-Boot, because the u-boot package in Debian can't cope with the SD card device. Hopefully this can be fixed some year. Worse yet is the FreeAgent DockStar, which is running a foreign kernel and two (TWO) foreign U-Boots (the first one chains to the second one which then boots off a USB stick). Unfortunately I'm afraid to correct these things because, unlike the other two, it is not a developer board, and messing it up would mean trying to deal with 3.3V RS-232 or something equally unpleasant. As it is, bricking one of the plugs where I actually have serial/JTAG causes enough emotional turmoil.

I m a Debian developer since 1998 and have been fostering cooperation between Debian and Ubuntu since its inception with various levels of success. I recently decided to go one step further and join Ubuntu as an official member. I wanted to be able to vote and to promote best cooperation practices from the Ubuntu side as well (and not only in Debian). Joining as a Debian developer should be relatively easy, it is well known in the Debian community that Mark Shuttleworth once said:

Every Debian developer is also an Ubuntu developer, because one way to contribute to Ubuntu is to contribute to Debian.

So I sought to verify that too! Here s the result. I prepared my application page collecting 2 testimonials from Lucas Nussbaum and Colin Watson, I registered for the next EMEA membership meeting (which happened yesterday, irc logs here), and I attended. (I just followed the instructions on the wiki) During the meeting, when it was my turn, I pasted my short introduction and got immediately +1 votes from several board members, they didn t even want to hear from me, they were already convinced. I still responded to a few questions and elaborated on my goals and was promptly accepted even if I made it clear that my technical contributions will continue to be done within Debian. A few minutes later I was added to the ubuntumembers launchpad group and this morning I followed the instructions to add my feed to Planet Ubuntu and here I am The whole process has been very smooth and the meeting shows a very welcoming attitude. Even someone who was denied the Ubuntu member status was cheered for his work and encouraged to try again when he can present some testimonials from other members. Hello Planet Ubuntu! 11 comments Support my work

Today, virtually all of the free software available can be found in packaged form in distributions like Debian and Ubuntu. Users of these distributions have access to a library of thousands of applications, ranging from trivial to highly sophisticated software systems. Developers can find a vast array of programming languages, tools and libraries for constructing new applications. This is possible because we have a mature system for turning free software components into standardized modules (packages). Some software is more difficult to package and maintain, and I m occasionally surprised to find something very useful which isn t packaged yet, but in general, the software I want is packaged and ready before I realize I need it. Even the long tail of niche software is generally packaged very effectively. Thanks to coherent standards, sophisticated management tools, and the principles of software freedom, these packages can be mixed and matched to create complete software stacks for a wide range of devices, from netbooks to supercomputing clusters. These stacks are tightly integrated, and can be tested, released, maintained and upgraded as a unit. The Debian system is unparalleled for this purpose, which is why Ubuntu is based on it. The vision, for a free software operating system which is highly modular and customizable, has been achieved. Rough edges This is a momentous achievement, and the Debian packaging system fulfills its intended purpose very well. However, there are a number of areas where it introduces friction, because the package model doesn t quite fit some new problems. Most of these are becoming more common over time as technology evolves and changes shape.

• Embedded systems need to be pared down to the essentials to minimize storage, distribution, computation and maintenance costs. Standardized packaging introduces excessive code, data and interdependency which make the system larger than necessary. Tight integration makes it difficult to bootstrap the system from scratch for custom hardware. Projects like Embedded Debian aim to adapt the Debian system to be more suitable for use in these environments, to varying degrees of success. Meanwhile, smart phones will soon become the most common type of computer globally.
• Data, in contrast to software, has simple requirements. It just needs to be up to date and accessible to programs. Packaging and distributing it through the standardized packaging process is awkward, doesn t offer tangible benefits, and introduces overhead. There have been extensive debates in Debian about how to handle large data sets. Meanwhile, this problem is becoming increasingly important as data science catalyzes a new wave of applications.
• Client/server and other types of distributed applications are notoriously tricky to package. The packaging system works within the context of a single OS instance, and so relationships which span multiple OS instances (e.g. a server application which depends on a database running on another server) are not straightforward. Meanwhile, the web has become a first-class application development platform, and this kind of interdependency is extremely common on both clients and servers.
• Cross-platform applications such as Firefox, Chromium and OpenOffice.org have long struggled with packaging. In order to be portable, they tend to bundle the components they depend on, such as libraries. Packagers strive for normalization, and want these applications to use the packaged versions of these libraries instead. Application developers build, test and ship one set of dependencies, but their users receive a different stack when they use the packaged version of the application. Developers on both sides are in constant tension as they expect their configuration to be the canonical one, and want it to be tightly integrated. Cross-platform application developers want to provide their own, application-specific cross-platform update mechanism, while distributions want to use the same mechanism for all their components.
• Virtual appliances aim to combine application and operating system into a portable bundle. While a modular OS is definitely called for, appliances face some of the same problems as embedded systems as they need to be minimized. Furthermore, the appliance becomes a component in itself, and requires metadata, distribution mechanisms and so on. If someone wants to install a virtual appliance, how should that work? Packaging them up as .debs doesn t make much sense for the same reasons that apply to large data sets. I haven t seen virtual appliances really taking off, but I expect cloud to change that.
• Runtime libraries for languages such as Perl, Python and Ruby provide their own packaging systems, which manage dependencies and other metadata, installation, upgrades and removal in a standardized way. Because these operate independently of the OS package manager, all sorts of problems arise. Projects such as GoboLinux have attempted to tie them together, to varying degrees of success. Meanwhile, each new programming language we invent comes with a different, incompatible package manager, and distribution developers need to spend time repackaging them into their preferred format.

Why are we stuck?

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.
Abraham Maslow

The packaging ecosystem is very strong. Not only do we have powerful tools for working with packages, we also benefit from packages being a well-understood concept, and having established processes for developing, exchanging and talking about them. Once something is packaged, we know what it is and how to work with it, and it fits into everything else. So, it is tempting to package everything in sight, as we already know how to make sense of packages. However, this may not always be the right tool for the job. Various attempts have been made to extend the packaging concept to make it more general, for example:

• Portage, of Gentoo fame, offers impressive flexibility by building packages with a custom configuration, tailored for the needs of the target system.
• Conary, from rPath, offers finer-grained dependencies, powerful revision control and object-oriented build recipes.
• Nix provides a consistent build and runtime environment, ensuring that programs are run with the same dependencies used to build them, by keeping the relevant versions installed. I don t know much about it, but it sounds like all dependencies implicitly refer to an exact version.

Other package managers aim to solve a specific problem, such as providing lightweight package management for embedded systems, or lazy dependency installation, or fixing the filesystem hierarchy. There is a long list of package managers of various levels which solve different problems. Most of these systems suffer from an important fundamental tradeoff: they are designed to manage the entire system, from the kernel through applications, and so they must be used wholesale in order to reap their full benefit. In other words, in their world, everything is a package, and anything which is not a package is out of scope. Therefore, each of these systems requires a separate collection of packages, and each time we invent a new one, its adherents set about packaging everything in the new format. It takes a very long time to do this, and most of them lose momentum before a mature ecosystem can form around them. This lock-in effect makes it difficult for new packaging technologies to succeed. Divide and Conquer No single package management framework is flexible enough to accommodate all of the needs we have today. Even more importantly, a generic solution won t account for the needs we will have tomorrow. I propose that in order to move forward, we must make it possible to solve packaging problems separately, rather than attempting to solve them all within a single system.

• Decouple applications from the platform. Debian packaging is an excellent solution for managing the network of highly interdependent components which make up the core of a modern Linux distribution. It falls short, however, for managing the needs of modern applications: fast-moving, cross-platform and client/server (especially web). Let s stop trying to fit these square pegs into round holes, and adopt a different solution for this space, preferably one which is comprehensible and useful to application developers so that they can do most of the work.
• Treat data as a service. It s no longer useful to package up documentation in order to provide local copies of it on every Linux system. The web is a much, much richer and more effective solution to that problem. The same principle is increasingly applicable to structured data. From documents and contacts to anti-virus signatures and PCI IDs, there s much better data to be had out there on the web than down here on the local filesystem.
• Simplify integration between packaging systems in order to enable a heterogeneous model. When we break the assumption that everything is a package, we will need new tools to manage the interfaces between different types of components. Applications will need to introspect their dependency chain, and system management tools will need to be able to interrogate applications. We ll need thoughtfully designed interfaces which provide an appropriate level of abstraction while offering sufficient flexibility to solve many different packaging problems. There is unarguably a cost to this heterogeneity, but I believe it would easily outweigh the shortcomings of our current model.

But I like things how they are! We don t have a choice. The world is changing around us, and distributions need to evolve with it. If we don t adapt, we will eventually give way to systems which do solve these problems. Take, for example, modern web browsers like Firefox and Chromium. Arguably the most vital application for users, the browser is coming under increasing pressure to keep up with the breakneck pace of innovation on the web. The next wave of real-time collaboration and multimedia applications relies on the rapid development of new capabilities in web browsers. Browser makers are responding by accelerating deployment in the field: both aggressively push new releases to their users. A report from Google found that Chrome upgrades 97% of their users within 21 days of a new release, and Firefox 85% (both impressive numbers). Mozilla recently changed their maintenance policies, discontinuing maintenance of stable releases and forcing Ubuntu to ship new upstream releases to users. These applications are just the leading edge of the curve, and the pressure will only increase. Equally powerful trends are pressing server applications, embedded systems, and data to adapt as well. The ideas I ve presented here are only one possible way forward, and I m sure there are more and better ideas brewing in distribution communities. I m sure that I m not the only one thinking about these problems. Whatever it looks like in the end, I have no doubt that change is ahead.

Thanks to some recommendations from Jeremy, Pierre and Jeff, this blog should be running a lot less slowly. I ve installed memcached, set up wordpress plugins, tuned apache MPM parameters, tweaked iptables and tc rules and beaten on the blog with load testing scripts. It seems that it will now reliably handle around 10 concurrent requests and take less than 5 seconds to bring the page up. This is much better than the previous <2r/s 20s load time. I m no amazon.com (thank goodness), but I think this will meet the blog s needs quite well.

Thanks to some recommendations from Jeremy, Pierre and Jeff, this blog should be running a lot less slowly. I ve installed memcached, set up wordpress plugins, tuned apache MPM parameters, tweaked iptables and tc rules and beaten on the blog with load testing scripts. It seems that it will now reliably handle around 10 concurrent requests and take less than 5 seconds to bring the page up. This is much better than the previous <2r/s 20s load time. I m no amazon.com (thank goodness), but I think this will meet the blog s needs quite well.

The current version of mksh had use of arc4random(3) removed, including set -o arc4random , to speed it up (on some architectures, a lot) this will break some existing scripts (such as /etc/rc *cough* on MirBSD ). Hence I decided to publish the next version of mksh(1) as R40 based upon current development, and defer plans for associative arrays (and multidimensional arrays) for mksh R41. There s also already the change to Build.sh arguments, so this suits me quite fine. (Read: if running MirBSD, don t upgrade mksh at the moment.) There will be a new MirBSD snapshot once this is fixed, maybe a few more changes to the shell for better POSuX compliance, and the recently mentioned patent on LFNs (long filename) in FAT will be taken into account with a patch to msdosfs. I ll travel to LSM/RMLL 2010, the Libre Software Meeting (Rencontres Mondiales du Logiciel Libre) tomorrow until the weekend, to hack some on FusionForge (this is worktime for me), visit XTaran, Uriel, and maybe a couple of other usual people . Thundersday, between 10:00 UTC and 12:00 UTC, eurynome will be shut down by gecko2@ due to power supply maintenance on the host system data centre. We have a new mirror in the Americas, thanks a lot to Mike 'Fuzzy' Partin! Benny will mention it on the webpages once it s working.

On Sunday 5 September I will be entering a 10K running event, probably the Middlesex 10K in Victoria Park. This will be my first running event, so I've put together a training schedule based on a few sources on the web. Whilst nine weeks preparation appears quite conservative I have been running for about 5 weeks already I will be in North America for about a month of that with an unpredictable schedule. Below is what I'm currently shooting for. Units are in miles, a hyphen indicates a rest day and "cross" stands for cross-training. I haven't decided what I'm going to do for that yet although swimming is tempting. Tasks for this week:

• Complete week 1
• Get actual running shoes

My project. This post assumes that the reader has read my last post. This week I could get my focus on the project again, since most of my classes are already over. I ve investigated the segfault that was happening in GHC with +RTS -w -DS, and I noticed that the code in rts/sm/Sanity.c assumed that all objects in the allocated area of the block are being used.

    for (; bd != NULL; bd = bd->link)  
	p = bd->start;
	while (p < bd->free)  
	    nat size = checkClosure((StgClosure *)p);
	    /* This is the smallest size of closure that can live in the heap */
	    ASSERT( size >= MIN_PAYLOAD_SIZE + sizeofW(StgHeader) );
	    p += size;
	    /* skip over slop */
	    while (p < bd->free &&
		   (*p < 0x1000   !LOOKS_LIKE_INFO_PTR(*p)))   p++;  
	 
     

Since this is true for copy collection and mark compact, it was only with mark sweep that the error happened. The only way I could manage to make the segfault disappear by now was marking the swept blocks with a new flag, and avoid running this code in them. So I went back to my old problem with the allocation of memory in lines. I noticed that (one of) the problem(s) may be that the object that is allocated in the free line is not scavenged after the evacuation. When the object is allocated in a block, using the current allocation method, it will eventually be scavenged, because all blocks that were being used to allocation are scavenged. I m planning to implement a list of lines that need to be scavenged and code to scavenge the lines in this list, and the current line. It ll be very similar to the code that does this with blocks.

Wednesday, after some weeks with a flickering screen (or more precisely, something is causing GTK+ to redraw and the kernel to print ^@ in the terminal when you touch the screen, see http://jak-linux.org/tmp/20100628_002.mp4) in my 3 years old HP Compaq 6720s (which seems to be a software-hardware combination problem, at least it works in Ubuntu); I decided to buy a new laptop. It took me a few minutes to find the Lenovo ThinkPad Edge 15 NVL7VGE; which I ordered at notebooksbilliger.de at around 14:30 CEST using Express. On Thursday morning at 6:40 CEST, the device arrived. The laptop features an Intel Core i5 430 M processor with integrated graphics, 4 GiB of RAM, 320 GB hard disk storage, matte screen, and of course a TrackPoint. The machine comes pre-installed with Windows 7 Professional which is not very nice (and Lenovo does not take Windows licenses back). I booted Ubuntu 10.04 from an USB disk which worked a few times, but not always, so I upgraded the BIOS to make USB booting reliable (as Lenovo published multiple BIOS updates already, one fixing this USB booting issue). The first step was booting Ubuntu 10.04 from an USB hard disk to check the Linux support. Using Ubuntu, everything worked out of the box, including stuff like HDMI audio support or output switching. An exception may be the DVD drive, which only works if you set the SATA mode to compatibility. Non-DVD media also works in AHCI mode, but only if you start with a disc inserted in the drive. Playing DVDs requires setting a region code using setregion(8) [otherwise they do not work at all] and SATA compatibility mode [otherwise they only work partially]. Another issue is the HDMI output, as the TV is not correctly detected and some parts at the top and the bottom of the image are missing (that is, the GNOME panels) (but that happens with different screens and ATI cards as well). Oh, and closing the lid with a screen attached and the internal screen disabled works only as long as you do not open it again, because that causes the GPU to stop. On the input side, we have one keyboard, one TrackPoint, and a touch pad. All of them work out of the box, but they have a few problems on the hardware side: The cursor keys on the keyboard are incredibly small and the touchpad is very large. The keyboard is different from the one in the HP Compaq: First of all, there is space between the keys (like Apple keyboards); and secondly, you need to apply more pressure in order to use the keys. The device feels more solid than the HP Compaq 6720s, especially touching the screen does not change the colors on the screen. I don t know how long the keyboard will last, but I hope that I will still have symbols on them in 5 months (which was not the case for the HP Compaq 6720s keyboard). Power consumption is another interesting point. When the device is idle and the screen is disabled, the device seems to consume about 10W. With screen enabled, consumption varies between 13W (idle) 35W (building source code); if the values received via ACPI are correct. The device ships with a 48 Wh battery which after some time was displayed as 53 Wh in gnome-power-statistics (only the when loaded state, capacity is displayed as 47.5Wh). According to Lenovo, the laptop lasts 4.5 hours on battery, meaning consumption of 47.5 Wh / 4.5 h = 10.5 Wh which is a bit overly optimistic if you want to work on the device. After checking everything, I took over my Debian sid installation from my old laptop (which I installed in 2008). It runs happely now, although I still need to get accustomed to the new keyboard and the large touch pad. All in all, the price of 765 I paid for the device seems to be OK. Now I only need to remove the Windows 7 sticker and hope that the device lasts long enough until I want to buy a new one (probably 2013, if the aliens don t invade us 2012).
Filed under: General

Getting key verification working for pbuilder. It's been on the TODO list, and after some prodding, I've finally gotten around to getting something implemented. The last time I thought of implementing it was 1 1/2 years ago. Time flies. Not quite sure if this is enough or if I broke something. I have a feeling that something will break, so please try out verifying the signatures for the packages. I am hopeful that since main users of pbuilder are Debian Developers I will get useful patches.

I was planning to attend DebConf New York this year, but for a number of reasons I decided not to go. Fortunately, Ferdinand Thommes organized a MiniDebConf in Berlin at LinuxTag and I managed to attend. Thanks, Ferdinand! There were a number of interesting Talks. I especially liked the talk of our DPL, and those about piuparts and git-buildpackage. In contrast to the other LinuxTag talks, we had a livestream of our talks and recorded (most) of them. The kudos for setting this up goes to Alexander Wirt, who spent quite a few hours to get it up and running. I have to apologize for being late in bringing my Notebook, which was intended to do the theora encoding of the livestream. This was a misunderstanding on my part, I should have known that this is not going to be setup in the night before show time So to compensate the extra hours he had to put in for me, I offered to do the post processing of the videos. Basic approach for post processing The main goal of post processing the videos was (of course) to compress them to a usable size from the original 143 GB. I also wanted to have a title on each video, and show the sponsors at the end of the video. My basic idea to implement that consisted of the following steps:

1. Create a title animation template.
2. Generate title animations from template for all talks.
3. Use a video editor to create a playlist of the parts title talk epilogue.
4. Run the video editor in batch mode to generate the combined video.
5. Encode the resulting video as ogg theora.

As always with technology, it turned out that the original plan needed a few modifications. Title animations Originally I wanted to use Blender for the title animation, but I knew it is quite a complicated bit of software. So I looked for something simpler, and stumbled across an article that pointed me towards Synfig Studio for 2D animation. This is also in Debian, so I gave it a try. I was delighted that Synfig Studio has a command line renderer which is just called synfig and that the file format is XML, which would make it simple to batch-create the title animations. My title template can be found in this git repository. Batch creation of title animations I used a combination of make and a simple python script to replace the author name and the title of the talk into the synfig XML file. The data for all talks is another XML file talks.xml. Basically, I used a simple XPath expression to find the relevant text node and change the data using the ElementTree API of lxml python module. The same could be done using XSLT of course (for a constant replacement, see this file) but I found it easier to combine two XML files in python. Note that I create PNG files with synfig and use ffmpeg to generate a DV file from those. Originally, I had synfig create DV files directly but those turned out quite gray for some reason. I am now unable to reproduce this problem. Combining the title animation with the talk For joining the title animation with the talk, I originally went with OpenShot, which somebody of the video team had running at the conference. My idea was to mix a single video manually and just replace the underlying data files for each talk. I expected that this would be easy using the openshot-render command, which renders the output video from the input clips and the OpenShot project file. However, OpenShot stores the video lengths in the project file and will take those literally, so this did not work for talks of different play times I considered working with Kino or Kdenlive but they did not look more appropriate for this use case. I noticed that OpenShot and Kdenlive both use the Media Lovin Toolkit under the hood, and OpenShot actually serializes the MLT configuration to $HOME/.openshot/sequence.xml when rendering. I first tried to read that XML file from python (using the mlt python bindings from the python-mlt2 package) but did not find an API function to do that. So I just hard coded the video sequence in python. I ran into a few gotchas on the way:

• it s easy to segfault MLT from Python (something is wrong with the refcounting)
• generating ogg theora using MLT from Python (which uses libavformat) resulted in a bad audio/video synchronisation of the output (more than 1 second for 1 hour of video)
• generating dv from MLT and running ffmpeg2theora on the result is disk-bound on my quad core system so use a pipe to ffmpeg2theora

Things to improve While the results look quite okay for me now, there is a lot of room for improvement.

• What bothers me the most is the unequal audio volume between talks. We should apply something like replay gain for normalization. I did not find an easy way to add this to the processing pipeline, otherwise it would have been included already.
• Original recordings have different aspect ratios, so after combining them, there is a letterbox added. This is the case for Goswin s talk on Multiarch for example. I d rather have the title distorted or letter boxed, if I get around to it, I will fix it in the script.
• Most of the recordings concentrate on showing the slides. As we did not keep the raw camera data, but recorded the DVswitch output, we missed the opportunity to show the speaker more often in full screen. Looking at the same slide becomes quite boring after half a minute, so it would be better
  to show the speaker more often. That would require quite a bit of man power for post processing though, I have to admit that I did not watch all the hours of video.
• We could cut a bit off the talk at start and end, when the speaker discusses where he is available for questions. It is not of interest anymore after the conference is over.

Availability

• git repo (browsing): http://www.landschoff.net/git/?p=mdc-video.git;a=summary
• git repo (clone): git://landschoff.net/mdc-video
• Videos: http://meetings-archive.debian.net/pub/debian-meetings/2010/minidebconf-berlin/ (note: most of these are in german, apart from the DPL and the LXDE talk)