VirginMega.fr is a one of the French music download service which offers MP3 files (without DRM). They previously prevented all the browsers except MS Internet Explorer on MS Windows platform from accessing their website. I (we) used to change the browser id thanks to the FF User Agent Switcher plugin.
It now seems they have changed their mind under the pressure of Linux users. However, it seems there is still a problem for me (us) to access their website as they do not use the engine (Gecko in our case), but the user agent name… No, Iceweasel is not Firefox! Nor is Galeon! But why blocking access to a website which works perfectly with these browsers which share the same engine?

Following my previous entry about playing around with LVM, LUKS and the device mapper, I documented myself about internals involved in pvmove, which is more of a challenge, considering there is no such documentation. I could find no useful documentation for either the device mapper or LVM. A bit of good old UTSL later, I could elaborate how to do what I wanted, and realized it was even possible to do in shell script. So here you are : a shell script to transform an LVM physical volume to a LVM over LUKS physical volume,. I’ll detail in between how it works. As the previous script, use at your own risk, it comes with no warranty.
Note you theorically can still access the filesystems underneath without problems. It’s an in-place and live conversion. Also note this is more a proof of concept than a proper, risk-less and well-written solution.


set -e
dev=$1
luks=$(mktemp)
cryptdev=$(basename $dev)_crypt
pvsize=$(pvs -o pe_start,pv_size --units s --noheadings --nosuffix "$dev" awk ' print $1 + $2 ')
devsize=$(blockdev --getsz "$dev")
mchunk=8


The script takes the physical volume device to convert as an argument. Note there is no check for the validity of its value.

• luks will be the temporary file where we are going to create a temporary LUKS volume.
• pvsize is the full size of the LVM physical volume, i.e. the size of all the physical extents (pv_size) + the LVM headers and metadata (pe_start).
• devsize is the raw device size.
• mchunk is the size in blocks of chunks for the mirror target, see below.


dd of="$luks" seek=$devsize count=0 bs=512 2> /dev/null
luksdev=$(losetup -f)
losetup "$luksdev" "$luks"
trap "losetup -d \"$luksdev\"; rm -f \"$luks\"" EXIT


Next, we create a sparse luks file the same size as the device (in case luksFormat would use the size somehow, but I believe it doesn’t), and a loopback device on this file. The trap is here to avoid leaving the loopback device and the file when an error occurs later (though during the conversion itself, it will be pointless).


cryptsetup luksFormat -q "$luksdev"
cryptsetup luksOpen "$luksdev" $ cryptdev _real
read start length crypt format key IVoff cdev offset <<EOF
$(dmsetup table $ cryptdev _real)
EOF


We create a LUKS device, so that we can get the encryption key ($key), and the size of the LUKS header ($offset). Note you need to add –showkey to the dmsetup table command on sid.


if [ $(expr $devsize - $pvsize) -lt $offset ]; then
  echo Not enough free space after LVM physical volume
  cryptsetup luksClose $ cryptdev _real
  exit
fi


Check we have enough space after the LVM physical volume to offset everything by the size of the LUKS header. If not, you can still try again after you reduce the size of the LVM physical volume by an extent.


if [ $(expr $devsize % $offset % $mchunk) -gt 0 ]; then
  echo Last
  cryptsetup luksClose $ cryptdev _real
  exit
fi


This is another check to avoid surprises at the end, when dealing with the last chunk. As the script is written for the moment, it doesn’t support cases where this last chunk is not a multiple of $mchunk. So we need to abort in these.


read major minor <<EOF
$(stat -t "$dev" awk ' print $10,$11 ')
EOF
maps=$(dmsetup deps awk -F: "/\($major, $minor\)/ print \$1 ")
dmsetup create $cryptdev <<EOF
0 $length linear $dev 0
EOF
dmsetup reload $ cryptdev _real <<EOF
$start $length crypt $format $key $IVoff $dev $offset
EOF
dmsetup resume $ cryptdev _real
for map in $ maps ; do
  dmsetup table "$map" sed s,$major:$minor,/dev/mapper/$cryptdev, dmsetup reload "$map"
  dmsetup resume "$map"
done


Here, we create the dev_crypt device mapper that is our fake LUKS device, which starts as a simple linear mapper and will end as a complete crypt mapper. This fake LUKS device is inserted as an intermediate mapper between the LVM device mapper and the real device. So, the LVM device mapper will have this fake LUKS device as backend, and, at the beginning, the fake LUKS device maps linearly to the real device. Note we look for all device mappers using the real device as backend before creating the fake LUKS device to avoid finding the fake LUKS device in the list. Also note dmsetup reload only loads a new table in the INACTIVE slot, and dmsetup resume makes this inactive table LIVE.


cursor=$length
chunk=$offset
while [ $cursor -gt 0 ]; do
  cursor=$(expr $cursor - $chunk)
  if [ $cursor -lt 0 ]; then
    chunk=$(expr $chunk + $cursor)
    cursor=0
  fi
  (
  [ $cursor -ne 0 ] && echo 0 $cursor linear $dev 0
  echo $cursor $chunk mirror core 1 $mchunk 2 $dev $cursor /dev/mapper/$ cryptdev _real $cursor
  [ $cursor -lt $(expr $length - $chunk) ] && echo $(expr $cursor + $chunk) $(expr $length - $cursor - $chunk) crypt $format $key $(expr $IVoff + $cursor + $chunk) $dev $(expr $offset + $cursor + $chunk)
  ) dmsetup reload "$cryptdev"
  dmsetup resume "$cryptdev"
  chunks=$(expr $chunk / $mchunk)
  while ! dmsetup status "$cryptdev" grep "$chunks/$chunks"; do
    true
  done
done


This is where the main work is done : moving the data around. We actually just let the device mapper deal with the data duplication, $offset blocks by $offset blocks ($offset being the LUKS header size), using a mirror target for the chunk being moved. So our disk looks like the following:

We use the extra dev_crypt_real device (the previously remapped LUKS device) as the encryption backend for the mirror.
I haven’t figured a better way to wait for the end of the mirroring than to do a loop checking with dmsetup status, dmsetup wait doesn’t seem to be very helpful here.
Anyways, this is the part of the script where you don’t want a crash to occur. Because if it does, all you can do is start on a rescue system, and try to find where the encrypted part of the disk start to setup a device mapper by hand.
And you’d better have the luks temporary file in a directory that is neither in RAM (think tmpfs) nor in the LVM you are converting (/tmp in a default etch install is, for instance ; note the script works nevertheless fine in this case). Also note the trap will remove the luks temporary file if the script exits…


dmsetup reload "$cryptdev" <<EOF
$start $length crypt $format $key $IVoff $dev $offset
EOF
dmsetup resume "$cryptdev"
dmsetup remove "$ cryptdev _real"
dd if="$luks" of=$dev count=$offset bs=512 2> /dev/null


Final steps of the conversion : our dev_crypt device becomes a full LUKS volume, so we can remove dev_crypt_real and add the LUKS headers at the top of the device we converted. At this moment, the LUKS volume is setup just as if it had been setup by cryptsetup. For LVM to recognize the change properly, you need to run pvscan. Once you ran it, you can do whatever you want with LVM. Now, you may want to add the following to your /etc/crypttab file:


$cryptdev $dev none luks


i.e. hda5_crypt /dev/hda5 none luks if the device was /dev/hda5. And if the LVM volume you converted contains your root filesystem, you should run (for Debian systems):


update-initramfs -u


I tested this successfully under qemu and will give it a shot on my laptop some time soon. Now, because I had a hard time not finding much about the mirror target of the device mapper, here is what I could gather about it. The target syntax is as follows:


<logical_start_sector> <num_sectors> mirror [ core disk ] <num_params> <param> ... <num_mirrors> [ <destination > <start_sector> ] ...


logical_start_sector, num_sectors, destination and start_sector have the same meaning as in other targets.
core and disk are two different log types (to track differences between mirrors), respectively in memory and on disk.
num_params and params depend on the log type:

• for core, num_params can be either 1 or 2.
  The first parameter is region_size, which is the size (in blocks) of the chunks the mirror log tracks for synchronization. It can’t be less than the page size divided by 512, i.e. 8 on x86, must be a power of 2, and must not exceed num_sectors. Apparently, it is not a problem if num_sectors is not a multiple of region_size.
  The optional second parameter is either sync or nosync, meaning of which I’m not sure. I think it determines whether the mirror should do the initial synchronization (sync) or not (nosync).
• for disk, num_params can be either 2 or 3, with params being log_device (device where the logs are kept), region_size, as above, and an optional sync or nosync, as above too.

num_mirrors is the number of mirrors and for each mirror, we have a pair destination and start_sector. [ Update: after a quick look at the device mapper source code in Linus’s git tree, updated the mirror target description ]

Apparently there’s lots of interest in the TiVo-Comcast product that I took a picture of. Already that picture has nearly 350 views, and the blog entry got a couple questions about it. And it’s a crap photo of something that was seen at CES. The two questions that were asked were: When can I get it? I have no idea. All I know is that we got 1.0 done and Comcast currently has it. Most likely they’re going to be doing some testing of it privately, and then start market trials in select areas, but I don’t know when or where. (It’s also going to be available for Cox customers, but I have even less information about that.) Should I wait for it or get a Series 3? I can’t really answer that question. I have a Series 3 at home and it’s awesome. For me, if the choice were between this or a Series 3, I’d take the Series 3, despite the several hundred dollar price tag (especially since it’s currently available). Before I got the Series 3, I had the old Comcast DVR; and if I had to choose between that and the TiVo, I’d definitely choose the TiVo. (I’m biased, but I was a TiVo fan even before I got hired. That’s part of the reason I wanted to work here.) Disclaimer Please keep in mind that I’m not speaking on behalf of TiVo. I only work here.

Web applications are in constant competition for a user's attention. Unlike shrink-wrap software, there's often no captive audience, and web apps must gently guide and woo users to help them to the best experience of the software. This is particularly true when considering the problem of collecting personal details from a user. Collect too little information, and your app might not be able to function effectively. Collect too much information, and you raise the barrier for participation with your application. Users may either not be bothered, be alarmed at the apparent intrusion into their privacy, or fill out bogus information.  One size doesn't fit all
I originally stumbled on the issue in the development of Expectnation. The amount of information a conference speaker needs to give the organizers is significantly more than that of, say, a reviewer. An organizer will often want postal and phone details for a speaker, which are largely irrelevant to the review process. All conferences and events rely on the willingess of participants, so the path to participation needs to be as frictionless as possible.
Thus, a one-size-fits-all personal details form didn't make sense for us. At best, we could have simply made more fields optional, but that would result in missing information that we really did need. Another solution would be to provide a dedicated personal details form per user type, e.g. different forms for speakers and reviewers. This is a non-starter, however, as users can take on many roles within a conference. So, we had a problem on our hands: different levels of interaction with our application requires different levels of personal information. We also had a second problem, which is that not all events and organizers want the same level of information. For example, a user group event is unlikely to need the same amount of information from speakers as an academic conference. Thus we have two axes on which the amount of information required varies: by role, and by event.  Our solution: progressive disclosure We concluded that we wanted to adhere to two principles in the collection of user information for Expectnation:

• collect as little information from users as possible, and
  
• allow organizers to determine the level of information they wish to collect.

The first objective was achieved by constructing a matrix of information items against user role, with the possible values of required, optional, unused for each item. We then achieved the second objective by allowing this matrix to be varied over each event. We've named this concept "progressive disclosure." What this means in practice is that a user is only ever asked for the information required to fulfil the roles they have within Expectnation. If they take on another role, they are asked for the additional information at that point, and not before. Implementation concerns
Extract from disclosure level editing UI It would be untrue to say that progressive disclosure is simple to implement. A user in our system will have disclosure requirements that are the union of all of their roles in all of the events they attend. Within Ruby on Rails, this means the conventional validation logic is useless to us, as it only deals with static requirements. Secondly, disclosure requirements are expensive to compute, so we need to cache them, with the inevitable addition of issues such as cache invalidation. Beyond the data model issues, the user interface issues are, if anything, more troublesome. From an end-user's point of view, the difficulty arises when we need extra information from them, and presenting this in a way that makes sense. From the event organizer's point of view, we have the challenge of presenting a controlling user interface for rather a complex concept. Organizers must be able to change the level of required information either globally, or per-event. We opted for a grid presentation, an extract from which is shown in the screenshot in this article. Over time we'll be able to see how effective this interface is, and what we can do to improve it. Finally, we have chosen sensible defaults for all disclosure levels, as the best result would be that organizers have little need to reconfigure the settings in the first place. Making it look easy Because Expectnation is a general application for organizing events, rather than targetting one specific one, we've had to construct a framework for minimizing user information collection, as opposed to simple implementation of known scenarios. At the end of the day we have added a fairly complicated subsystem and an extra degree of complexity to our codebase. However, the result from the user's point of view is increased simplicity and a streamlined experience. Every time you embark on something that wide-reaching and complex, you have to ask yourself whether the feature is really worth it. In this case, though, the answer is a resounding "yes". The user is the most important factor in our software, and making their life easier in an unobtrusive manner is a priority.

• Expectnation blog: Only ask users for the information you need

For a long time I've been vexed by the gender bias of computing towards men. It's clear that computing has multiple issues, ranging from its perception to the nature of its society. As organizer of computing conferences, I care about ensuring that participation is meritocratic and inclusive.
As various debates have taken place over blogs in the past on women in computing, I have stayed silent. Not because I don't care, but because I really didn't know what to say. I try to understand, but at the end of the day I'm a man, I tend to misunderstand important points, and I suspect my contributions would come across as at best naïve, and more likely, patronising. So many conversations between men and women on this topic end up in mutual uncomprehension. Better to stay silent than be a fool.
This is why I'm very grateful for Jeni Tennison's analysis of women in computing. It is the most helpful blog post I have read recently on the topic. Perhaps it's because Jeni's British, and we share some of the same cultural assumptions and constraints. But more importantly, it's because her article builds bridges to my experience and understanding.
An argument of femininity Rather than simply appealing to tribes of men and women, Jeni places the discussion in terms of feminine and masculine traits. Like many men, I identify with certain attitudes and preferences more closely allied to my feminine side. By not excluding me, her wording helped me view the points she unfolded in terms that my own experience, however poor, could identify with. I've often found contemporary computing communities to be suffocatingly macho. There's a strong pressure to succeed in terms of attention from the chattering bloggers, privileged inner circles and the capacity for vicious excoriation of those who fall from grace.As I grew up through school and university, I held these sorts of social forms in contempt, as well as occasionally suffering at their hands. Yet now, bafflingly, I find myself under the impression I need to belong and compete in these spheres.Self-efficacy One of Jeni's central points is concern about one's own self-efficacy. This was the real revelation to me. I tend to assume most capable people I meet are aware of their own capability. Jeni turned this on its head for me when she wrote

... look at how Ruby on Rails is marketed. A big play is made of how easy it is. But if a language or framework is easy then people with low self-efficacy can’t win: if they manage to do something with it then they haven’t really achieved very much because anyone can do it; if they don’t manage to do something with it then they’re complete idiots. I’m not saying that we should advertise languages or frameworks as being hard, because obviously that can put people off as well, but a recognition of the barriers that people might face may, in a strange way, make them more approachable.

As Jeni brings her article to a close, it's with some shock and shame that I get the punchline loud and clear: "this isn't about you."It's about empathy, inclusivity and selflessness. Human qualities that are unrestricted to either gender.

I finally decided myself to switch my laptop to using a LVM over LUKS setup instead of LVM alone. The problem it involves is that there is currently no way that I know to do this in-place. I do have an external disk on which I could pvmove everything and back, but that would not be very challenging ;). So I started wondering how doable that would be. LUKS has a 1032 sectors (a sector being 512 bytes) overhead in which it store the LUKS header. Following the header, encrypted data just takes as much space as its decrypted counterpart, linearly. So all it would take to do in-place LUKS conversion would be to move data by 1032 sectors, and encrypt it. Which means the main problem is to have 1032 sectors free after the current volume. Fortunately, LVM doesn’t take up all the space on the partition a physical volume is setup, except if you’re unlucky or created partitions specifically so that it uses all the space. Because LVM physical volumes use the space by big chunks, 4MB by default), if the size of the partition is not 4MB aligned (modulo the LVM header), you have some free space between the physical volume end and the partition end. We thus need this free space to be greater than 1032 sectors. On my laptop, there is more than 1.8MB free there, so there would be no problem.
Note the debian-installer seems to do more than a simple cryptsetup luksFormat, as LUKS volumes it creates have an overhead of 2056 sectors instead of the 1032 sectors I got creating a volume by hand. Next problem is how to move the data and encrypt it in-place. Bonus points if that can be done without unmounting the filesystems. My first guess was to play around with the device mapper and a specially crafted FUSE filesystem to create a fake LUKS volume that could be half encrypted and half not, and would evolve, with time, into a fully encrypted LUKS volume. Since I didn’t know much about the device mapper, I looked at it a bit, and discovered a FUSE intermediate would be unnecessary: not only can you remap on-the-fly (which I already guessed was somehow possible, considering how LVM can do it, though I didn’t know much about the interaction between LVM and the d-m), but you can also mix dm-crypt and linear chunks in the same d-m table. It also appears LVM over LUKS has an overhead that is actually not necessary. How sad both use the device mapper but yet can’t do an efficient setup with it. When you setup LVM over LUKS, you open a LUKS volume, which will create a dm-crypt device mapper, and use this device mapper as the physical volume for LVM. Logical volumes created with LVM are actually device mappers using the physical volume as destination device. When you access a block on an LVM logical volume, your access is first mapped through the LV device mapper and then mapped again through the dm-crypt mapper. So it goes through 2 device mappers, but it could do with only one: the linear mapping used in most of the cases with LVM could be replaced by crypt mappings, that are just as linear, but also handle the encryption/decryption part. Playing around under qemu with a freshly installed etch system, I was able to implement this and came up with the following script. It should work with most setups, but use at your own risk, it comes with no warranty.


MAJOR=$(awk '/device-mapper/ print $1;exit ' /proc/devices)
dmsetup ls --target linear while read map extra; do
  TABLE=$(dmsetup table $map while IFS=" :" read start length linear major minor offset; do
    map2=
    if [ "$major" -eq "$MAJOR" ]; then
      IFS=":" read map2 extra <<EOF
$(dmsetup info -c --noheadings -j $major -m $minor)
EOF
    fi
    if [ -z "$map2" ] [ $(dmsetup table "$map2" wc -l) -gt 1 ] [ $(dmsetup table --target crypt "$map2" wc -l) -eq 0 ]; then
      echo $start $length $linear $major:$minor $offset
    else
      IFS=" " read cstart clength crypt format key IVoff dev coffset <<EOF
$(dmsetup table "$map2")
EOF
      echo $start $length crypt $format $key $(expr $IVoff + $offset) $dev $(expr $coffset + $offset)
    fi
  done)
  if (dmsetup table $map; echo "$TABLE") sort uniq -c grep -v "^ *2 " > /dev/null 2>&1;
  then
    echo Diverting $map
    dmsetup reload $map <<EOF
$TABLE
EOF
    dmsetup resume $map
  fi
done


[ Update: There was a dmsetup resume missing in the script ] After running this script, you should be able to run cryptsetup luksClose on your LUKS volume. Once you’ve done it, pvdisplay and other LVM tools will show you nothing, because the physical volume will just have disappeared… but everything will still be in place. Note that if you leave the LUKs volume open, you’d better be careful NOT to do operations with LVM (such as pvmove) because it could break everything. Note that on sid, you would need to add a –showkey option into the $(dmsetup table "$map2") line. This option is necessary for dmsetup to display the encryption keys. I wonder if LVM upstream would consider supporting LUKS volumes directly… Anyways, back to the in-place conversion to LVM over LUKs. The idea would be to:

• Create a file the size of the LUKS headers + one block.
• Create a loopback device for this file.
• luksFormat the loopback device.
• luksOpen the loopback device, so that dmsetup can give us the key to use. I wonder if there are ways to get the key not involving the creation of the device mapper.
• luksClose it, because we actually don’t need it
• Create a device mapper that linearly maps the whole LVM physical volume (i.e the real partition, but without the empty tail).
• Remap the LVM device mappers so that they use our own device mapper instead of the partition as physical volume.
• Here is the trickiest part, and I still have to take a look at how pvmove does to know exactly how this can be done (I guess it will require an additional device mapper): move data chunk by chunk (offsetting with the size of the LUKS headers), starting from the end of the LVM physical volume, and encrypt at the same time. Once a chunk is moved, adjust our device mapper so that it maps to the linear unmoved data followed by the moved encrypted data. As I said earlier, this is a setup that works.
• Once everything is moved, we are in a situation where our device mapper is a single crypt mapping, in the same shape as a luksOpen would have made it. We can now copy the LUKS headers at the beginning of the partition, and add the appropriate configuration to /etc/crypttab.

If I got it all right, both LUKS and LVM should be believing they have done this setup themselves, so unmounting, lvchange, etc. should just be fine. It might be necessary to regenerate the initramfs, though. I’ll tell you when I fuck up my hard disk try it. To be continued…

Just got back from Barcelona. On our last day, someone broke into our apartment and made off with two X40s, three cameras, four iPods, two N800s, a reasonable amount of cash, et al. One each of the X40s, cameras, and N800s, were mine.

That wouldn't be too catastrophic in and of itself, but unfortunately, the laptop was my master machine, where I'd been doing development, and I lost at least four active/useful X branches, in addition to the irreplacable photos/etc. Distributed development wasn't at fault here: one of the branches was actually done in CVS last year, but as a forked tree which thus had no history, and was just maintained as a huge diff until my hard drive crashed. Guess it's about time to start making some backups.

Anyway, long story short, if you were waiting for an xkbcomp-less X server, for RMLVO over the wire XKB, for an X server using HAL, or in fact basically any code from me, you'll have to wait a while longer.

From the Python 3000 Status Update:

Did I mention that lambda lives? I still get the occasional request to preserve it, so I figured I'd mention it twice. Don't worry, that request has been granted for over a year now.

Phew.

Did some work on d-i stuff, went to Yuri's mind-blowing Gentoo/Debian mashup talk. Very proud that "squeezed joeyh brains" went into that. Lots of other tech stuff, too much to blog about. But I spent a couple hours this afternoon on tourist stuff. Went up to Edinburgh Castle. Here's the view for the people at home: (Some more pics from there are in Maykel Moya's gallery Tomorrow: Geeks On A Train
(150 of them. In the Scottish countryside.)

This post is a sequel to my “What I want in edgy+1” post, which was written when the developer summit was first announced. Now that the summit (and the following company All Hands meeting) is over, and we’re all back home, this seems as good a time as any to review what was discussed and get a good idea of what feisty might look like. I’ve touched on the problem of predicting time-based releases before. It’s both the gift and the curse of a time-based release schedule that work not completed in time can be deferred to a later release. So take the following with a pinch of salt, some of this may still not make it. General Themes The general theme of dapper was to be a release that could be supported for a long term, conservatism was the goal. We did do some quite exciting work under the hood, such as the switch away from hotplug to a fully udev based system, but in general it wasn’t innovative or ground-breaking. Edgy was intended to be more ground-breaking, but the practical matter of having only a few months to develop it and our own pride in shipping something that still worked meant that it turned out as a shinier, improved dapper. So what’s feisty going to be like? Judging from the discussions at UDS, and the specifications that have been written, the general theme of feisty is to lead the way again with new technologies. The Desktop For the users, perhaps the most obvious change will be the active use of 3D acceleration to draw the desktop where hardware can support it (the issue of binary drivers has not yet been resolved). Windows are more visually distinct from each other through shadows behind them, and transparency for the non-active windows. The relationship between different workspaces/viewports is much clearer as the transition is animated on a cube or sliding pane. And for the bling crowd, window s can wobble, burn, explode or dissolve. There are two different compositors being considered at this point, compiz and beryl; we’re likely to decide which to use at Feature Freeze based on how well they’ve been fixed, developed and supported until that point. Underneath the hood, the configuration of the X server will be simpler and more robust; so even the worst case will not leave you confined to a console without any help. Networking Networking in feisty should be a much more pleasurable experience. The Network Manager project, which has been waiting on the side lines for a couple of releases, may finally get a shot at being in the default installation. For the average user, this makes switching between wired and wireless networks, including setting up WEP and WPA much, much easier. And what if there’s no network infrastructure around? Out of the box support for RFC 3927 link-local networks, and multi-cast DNS resolution (aka. Zeroconf), means that you just need to agree on a network name with others around you to be able to communicate. Of course, once you’re on a network, you still need to be able to share files and access local services. The integration of the Avahi project gives you one-click access to other people’s shared music or files; and lets you share your own, should you choose to do so. Customisations One of the most encountered problems with edgy was it being difficult to install various common packages that aren’t part of the default installation, especially codecs. Projects such as Automatix attempt to tackle this, but can cause problems with upgrading to later releases. Some effort will be going into feisty to make performing these common customisations much simpler, including being able to install codecs or viewers by just trying to open the file. Boot Sequence A long-running project within Ubuntu has been to get the boot and shutdown sequences as fast and efficient as possible. At the time we started, it was common for a Linux distribution to boot in a mere two or three minutes. If you thought edgy booted fast, wait until you see feisty. Feisty is the release where we take full advantage of Upstart, not only bringing the system up as fast as possible but also more robustly than we can do today. And if that weren’t enough, it should look slicker too; without some of the nasty flickering and mode changes that happen today.

Since I'm not at Debconf, I've been filling in time in the early mornings by randomly browsing Facebook. Chris Lamb had a post on a function called fnreduce, which clearly wandered past on planet when I was away. Since I was bored, I wrote a simple haskell version:

fnreduce :: [(a->a)]->a->a
fnreduce [] value = value
fnreduce (a:as) value = freduce as $ a value

Line by line, its clear how it works. The first line is the type sig, which says that the function fnreduce takes a list of functions, and a value of type a, and a gives back a value of type a. The functions all need to be of the type a->a, which means they take a a and give back a a. a is a type varible, which means it can be replaced with any type you feel like, for example: Int or String. The second line merely defines a base case, which says that if the list of functions is empty, it just returns the value given. The last line does all the work, and says that fnreduce on a non-empty list is computed by taking the first element of the list, computing the new value using it, then calling fnreduce on the rest of the list and the new value. This is how most list processing works in haskell. I'm left with two quick questions: the first is how to define it in terms of a fold, which it clearly is. The second is how to escape the a->a resriction, which lamby's can do. Since I'm writing haskell, I'ld like to do this in a type safe way, to have my cake and eat it too. This cant be done with haskell's type system, but I may run it past Edwin for his two cents if I remember...

The DebConf video team is using a large number of programs, some common and some of our own, spread across 20-30 computers. Some may be interested to know what this is.

• FAI for package installation and configuration on DebConf machines.
• dvgrab for video grabbing. We are using a variety of DV cameras, but dvgrab works well with almost all of them.
• dvswitch for switching between multiple cameras and files.
• ffmpeg2theora for transcoding to Ogg Theora/Vorbis, both for live streams and for the downloadable files slowly appearing on meetings-archive.
• A small number of scripts for starting the above three programs and recording to files.
• Icecast 2 for streaming. We are currently using the sarge package rebuilt for etch because the server in the etch package is unstable in our double-relayed configuration. (To limit bandwidth use between DebConf and the Internet, we have an internal server for internal use and an external relay that serves as master for all other relays.)
• GeoDNS and a monitoring script to resolve streams.video.debconf.org to the nearest working relay server.
• A newly-developed extension to the Pentabarf conference management system that presents recordings for review. Reviewers match up recordings to events, give the start and end time of the event in the recording so we can cut out extra material, and rate the quality of the recordings and transcoded files.
• A script to import the details of new recordings into the database ready for review, run as a cron job.
• A script to transcode recordings into multiple versions using the timing and naming information provided by reviewers. This spreads the work across multiple servers controlled using ssh (again!).

Most of the custom software and configurations should be included in the debconf-video project on Alioth.

IPMI stands for "Intelligent Platform Management Interface", an interesting feature deployed on nowadays servers. I can't forget that question I made some months ago about an extra ethernet port on the IBM server machines we were working with on those days. The answer was quite simple: it is almost of no use at all; at most you can get some stats and diagnostics but it is mainly intended for hardware technicians.

Liars (or ignorants)! Anyway, it is quite useful. It is the IPMI port which can be used for several managements actions (even from the running system) such as power cycle the machine, get some stats, establish a watchdog interface and the like.

Uses?, are they need to be told?. As a first glance you can get reports about the machine from the operating system itself without the need of physical access to the datacenter. You can also restart a machine which is failing to reboot by itself in case you can login one more time in the system. But... even more, you can do all that (and more) from a remote system (don't exhitate, it is password protected, at least, in case you configure it so). So the next obvious uses are for cluster fencing and stonith (shot the other node in the head) without the need of power control hardware.

It is a pity we didn't knew that when we configured the SAP cluster in my previous job despite the serial cable almost did it work. Sure Mr Navas will be interested in knowing about this technology :-)

Oh, a simple example:

# ipmitool -I open chassis status
System Power : on
Power Overload : false
Power Interlock : inactive
Main Power Fault : false
Power Control Fault : false
Power Restore Policy : always-off
Last Power Event :
Chassis Intrusion : inactive
Front-Panel Lockout : inactive
Drive Fault : false
Cooling/Fan Fault : false
Sleep Button Disable : not allowed
Diag Button Disable : allowed
Reset Button Disable : not allowed
Power Button Disable : allowed
Sleep Button Disabled: false
Diag Button Disabled : true
Reset Button Disabled: false
Power Button Disabled: true

Another one:

ipmitool -I open chassis power reset

Ooooooops. Next post after crash :-)

Conversation starting tool

1. Run ept-cache info and make sure that the popcon and debtags data sources are enabled
2. Run ept-cache search -t clean -s t- less

This will show you all packages that are not shared libraries or application data, and will show you at the top of the list those packages that you have installed and other people are very unlikely to have installed. You can use that list in case someone asks you "tell me of the cool packages that you use". If you need to improvise a talk at some Linux event, that list could very easily get you started and talking for hours.

I went out drinking last night, and got fairly plastered in process. I staggered back to the hostel at about 10pm, and amazingly, had the presence of mind to plug my laptop in to charge - to mean feat, given I had to assemble the three bits of the iGo charger. I woke up this morning feeling pretty pleased with myself (albeit in a seedy kind of way), until I discovered that I'd either failed to plug the cable into the laptop, or it fell out overnight, so it was all for nothing in the end. Oh well. It gets light here extremely early, and I woke up at about 4:30am and couldn't get back to sleep, so after a few more hours of lying around being seedy, I headed out for a brief reconnoitre before heading in to the morning sessions. I love this city! It's just so old. Edinburgh Castle looks really cool. I'm looking forward to checking it out.

No, the m68k port is not dead. Yes, there's an issue with etch-m68k, it mainly being that we haven't set it up properly after these months that we could've done so. That doesn't make it dead. I'd also have appreciated some inquiry from you before sending out such rumours, Joey. One of my goals for DebCamp/DebConf was to talk about it with the people involved and getting a buildd host up and running. That sortof is underway now. At the same time, it isn't. In any case, I planned yesterday, after my BoF, to give this the highest priority now. Expect some results RSN(tm).

It's the time of the year I feel a bit nostalgic since for the second, consecutive, time since 2004 I'm not at debconf. This time I've decided so, since I'm currently focused on develop my academic profile trying to catch-up this unfinished business I wanted to pursue sometime ago when I was younger. By May, things were quite on-schedule so I decided to drop this trip, since I didn't wanted to be involved in another unfortunate situation due time contraints, now even more constrained by my new responsibilities in life. I've tried to follow the conf over the stream since yesterday, but I felt asleep by 1am (8am there). However, as video team previously announced, by 17's night I went to the meetings arhive and downloaded Bdale's talk. I was impressed by the quality and professionalism of the video and the on-time (or time-to-market, aka me) publising. I want to congratulate the video team for this, very good work guys!. It's the second time I follow the conf over the stream and it helps to the being-there feeling. Anyway, I'm already arranging stuff and scheduling the time for debconf8 in Argentina. Hopefully I'm able to skip some days of class. I believe it would be a wonderful time, I also plan to visit my family there also. So, see you next year!. Now the usbmgr package. I have quite a feeling for this package (you might guess this knowing there were more mature options available since a couple of years ago). This was the first package I was in charge in Debian, and as popcon says there are still some users. However, things come to an end and I decided, as previously prepared and stated to some users, to orphan it post-etch. Now that the time has come, I'll do so and request its removal from the archive also in the following days. There are more mature options such the udev+hald mix and they support much more devices and give the user much more pleasant experience. usbmgr is a piece of software that was written when there was no good/integrated support for USB devices, specially pendrivers and memory sticks. There's no activity on upstream from long time ago, in fact I never was able to reach him, but I was trying to keep it useful for new software environments from time to time.
In case you still want to keep or maintain it, please drop me an email: rudy at d.o.

Right, so as announced I finished my thesis this week. Its translated title is (if anyone want to know)

"Design and Implementation of a Model for the Retrieval of Free/Libre Open Source Software using a Case-Based Reasoning System".

Writing it certainly was an educational experience but I am still happy it's done now. Especially the last weeks where rather unfunny, more so since about ten days before my deadline we had a massive heat wave here in Lower Saxony. Now that it's done I am still waiting for enthusiam to kick in. So far weariness still prevails. Maybe also because finishing my thesis does not yet end my studies. We also have to do written and oral examinations which will take place in July and after. So at the moment I spend my days mostly sleeping and trying to muster up enough wanderlust for my trip to DebConf7. (At the moment the prospect of traveling long distance with trains and cheap flights and then be surrounded by a few hundred happy people is still rather daunting.) Anyway, in order to give an impression of my last weeks and months and to sum this all up, a list of random notes, taken during the last weeks, of what I learned while writing my thesis:

• 6 months are NOT a long time!
• 10.000 chars are not enough to model a Debian package description
• 50 chars are not enough to model a Debian package name (YES, THAT MEANS YOU, libmaypole-plugin-authentication-usersessioncookie-perl!)
• osi.org does not belong to the Open Source Initiative but to Ontario Swine Improvement (Ontario Swine Improvement (OSI) is an industry-run organization dedicated to providing swine improvement and genetic products, programs, information and expertise. Our goal is to help Ontario pork producers achieve competitive advantages and excellence in the global market place.)
• I must not procrastinate.
  Procrastination is the mind-killer.
  Procrastination is the little-death that brings total obliteration.
  I will face my thesis.
  I will permit it to pass over me and through me.
  And when it has gone past I will turn the inner eye to see its path.
  Where the thesis has gone there will be nothing.
  Only I will remain.
• Dr.-Ing. Roland Bless rules! (So does his LaTeX class for theses.)
• Working without internet is possible. It is even better than working with internet. Simply leave gaps where you need to look up stuff and fill them in in the evening.
• Losing half a litre of blood does not impair my fitness for (brain) work.
• Working with version control requires a minimum of intelligence attention. Unfortunately mostly so when ending work (dead tired) and when beginning to work (dead tired).
• Side effects: Insomnia, itchy area on left leg, belly aches, little blisters on the sides of my fingers, headaches, slight but constant petulance.
• "I know what I want to say, getting it on paper is the least work." - WROOOONG!
• German is such a crappy language to write scientific texts in. Special greetings to the generic masculine. Gah!
• When the cat's writing her thesis ...
  The fun time's over now, darling ;)
• "I have don't really like proof-reading your thesis in PDF, can't you just send me the Word file?" is not what you want to hear from your proof reader.
• Good: Debian, vim, subversion, LaTeX, bibtex, evince, xpdf, pdftk, PostgreSQL, OpenOffice.org, dia, gimp, phpESP, tea, coffee, ramen, chocolate, electric fan, eating at the uni's cafeteria twince a day, raw cornflakes straight from the box, frida (my trusty laptop), power naps, pizza, people understanding my true problems, the freedom to burry myself in my work and absolutely and utterly neglect anything else until I am finished.
• The flat can look worse. Much!
• Bad:
  • The FSIL (student council for teaching students) belt grinding(!) tables on the balcony in front of the students' working rooms.
  • People trying to be nice and asking how it's going with the thesis.
  • Fellow students greeting me with "Hey still here?"
  • Our evil DPL blogging about his highly addictive monsterz just when I got to work.
• I'll do it after I handed in the thesis.
• I'll see a doctor about it after I handed in the thesis ...
• I should commit much more often (svn that is)
• How come there are so many extra 'i's in your thesis?" Because it's Linux!
• Sense of time is overrated.
• Print shop guy is mean.
  A = Me, overtired, having formatted and doing last changes to this document the whole night through and on until 20 minutes before the printing appointment.
  B = Print shop guy
  
  B: You know that's not A4 ...
  A: WHAT?!
  B: Yea, it's too big.
  A: WHAT?! But, but, I ... That can't be!
  B: I'm telling you, it's too big. See, 0.3mm too long and 0.12mm too wide
  A: ... *death stare*
  
  
• The examination office is now located in the fourth floor in the building next to the post office at the Hindenburgplatz.
• Tolimar is the most patient and understand guy I've ever known.
  Thank you!
• 6 months are SUCH a long time!

It actually seemed as though I wouldn't be going to DebConf this year. I didn't get to work much the last two months, since I wanted to spend all my time on the near completion of my thesis so I didn't have the money to pay for a flight (or train) to EDI and back myself. Due to that same thesis I also didn't have the time to prepare a talk this year so I was also not a speaker. Since I am also neither a DD nor in NM I was pretty sure I wouldn't be eligible for travel sponsorship this year. And, as expected, I was indeed very far down the queue, so when I still hadn't heard anything about being sponsored until about a week before DebConf was starting I had pretty much given up on the whole thing and resigned to the fact that this year's DebConf would happen without me. However, on 10 June I had a mail in my Inbox. Subject: "Your travel cost sponsorship request for DebConf7". It seems the DebConf team was able to dig up some more money and do another run on the sponsorship queue. This was three days before I had to get my thesis to the print shop and I wasn't really doing anything else except for frantically going over my proof readers' annotations, doing last minute additions, changes etc. and occasionally lie down for a few hours of rather uneasy sleep. So my first reaction was something along the lines of "What?! Now?! I don't have time for this now! Go away!" and I was pretty close to answering the mail saying "Thank's but that's too short notice. I won't be coming, have fun, see you all next year." Luckily Tolimar prevented this though and actually managed to find a flight for me that not only goes on the actual days that I had originally planned to arrive and leave but that is also not far over my travel sponsorship! So, long story short, I'll be flying to EDI on Wednesday. It'll only be for the very last few days and I am probably already missing out on lots and lots of stuff, but at least I'll be able to attend for a few days. And in time for debian-women's third birthday! :) I just hope everything will go well, it seems there's a lot of trouble with flying to DebConf this year, at least judging from what I read on planet and heard on irc. See you all soon!

Just stopped off in Derby on the way to Edinburgh. It was a good move in one way, because my inbound train was running over 30 minutes late (diverted due to the floods in the Midlands) and my outbound one is only 10 minutes late... but the manager just announced flooding in West Yorkshire, so I guess we'll see. The inbound train was called Vyjaer Kernewek (or something like that), which was nice to see. Derby was useful in another way (besides a good wifi-enabled light meal at the big blue coffee shop). I bought a laptop case. I didn't plan that when I set off, but by Bristol it was already obvious that I couldn't carry all my widgets in one bag. I also couldn't get the laptop out on the train, for fear of a packing explosion. So, Derby has shops. It also seems to have a market-free market place as the focus of the city centre. There's a view up to the cathedral, a small waterfall... and the Derby Mountain Rescue. I suppose the peaks are near, but mountains? The market wasn't in the marketplace. It seemed to be under cover in two parts. One is in the guildhall and the other is in a shopping centre that seems to have an identity crisis - is it the Eagle Centre, or is it Westfield? - and also has a building site attached that greets you as you approach from the railway. Not good tourist promotion. Until the building site, the City-Rail Trail is signposted well, about 5 minutes walk, but the signposts are a bit odd. I noticed the funny ridges first and dismissed them as an anti-spin thing, but then I noticed some bronze floppy disks (I'll post a photo once I find the right USB lead). Other attachments included spark plugs and glasses. There's no sign explaining it that I saw. What the heck is that all about? Coda: No, it wasn't a good move. Apparently there's more flooding in South Yorkshire, so I arrived about midnight and this is posted the next day. At least I'm having a better time of it than damog, who's stuck in New York and I wonder whether other tales of woe will appear on Planet Debian as people find net access. Now for breakfast and planning the day.