As some of you know, yesterday we started the two weeks of activities for the Fifth On-line Encounter of Free Education, Culture and Software. This year, the main topic for the conference is social networks, but not in the twitter-facebook sense Social networks as ways of understanding and modelling human-to-human interactions. Of course, there is interest from many attendees in using and taking advantage of said social network systems (and others), but the topic is far broader. One of the core tools of EDUSOL, and the only tool that is constantly active through our two weeks of activity, is IRC Which is somewhat of a challenge, as we receive some not-tech-savvy users from all over the world. The first year or two we asked everybody to connect via an IRC client (and we had even decided that it would be a technological filter We thought we would not cater for people without the technical skills to install and use IRC). Life proved us wrong, and we ended up using two web-to-IRC gateways: One of them, via an instance of CGI:IRC, and the other one via a third-party service, Mibbit. Now, the core mind behind EDUSOL is Alejandro Miranda, Pooka; I am not really into education (it is a very interesting topic but far from what I am good at). Pooka has been invovled in on-line communities analysis for several years already, since we worked together at FES Iztacala. For a long time he wanted to graph the interactions at EDUSOL, which was obviously possible given it was all centered on three tools (IRC for live chat, Drupal for generic information and Moodle for the talks and workshops' material This has changed this year, as we are giving much more weight to Drupal and taking it off Moodle), and so last year he was finally able to generate a EDUSOL 2008 conference photo (warning: 2MB image, 4311x3352px), where each person's avatar appears where most of his interactions were. This photo was (unfortunately) achieved using non-free software, but is very much looking at! Now, this year Pooka stumbled upon a great tool, PieSpy (available in Debian package piespy). Piespy generates very interesting interaction graph for IRC channels, as you can see here: Piespy is a bot that sits in the IRC channel logging everything that any given user "says" directly to another one, and graphs it. Not much to say here, only that it generates a very good (and practically real-time) view of what happens on the channel. For this particular channel, and during EDUSOL, you can see here the latest IRC snapshots. Very fun to see - and somewhat addictive how it morphs across time :)

Attachment	Size
EDUSOL 2008 - Conference photo	1.97 MB
Current (Monday+Tuesday) interaction graph for our IRC channel	39.89 KB

Of course, the event model I described before is far too simple to be worthy of a place in the ACPI spec. At the most basic level, there's more possible events than there are GPEs to attach them to, so there's a need for some further complexity. This manifests itself in the form of the ACPI embedded controller (EC).

The EC is typically a small microprocessor sitting on your motherboard, often implemented in the same hardware as the keyboard controller. It shares a lot in common with the keyboard controller - on PCs it'll usually appear in system io space, with one register for writing a command or reading a status, and a second register for passing data back and forth[1]. There's 256 registers available, so a typical interaction might be to write the READ command (0x80) to the command register, write the EC register address to the data register and then read back from the data register to get the EC register contents.

The embedded controller will often be responsible for tracking information about the hardware, such as the temperature. Attempting to read the temperature through ACPI will execute an ACPI method - in the case of the temperature being monitored by the embedded controller, this method will attempt to read from an EC register. The EC driver then performs the read and returns the result, which gets converted into decidegrees kelvin and passed back to whatever made the temperature query.

But, as mentioned above, the EC also generates events. These may be in response to a user initiated event like a hotkey press, or may be triggered by some change in hardware state like a thermal trip point being passed. The embedded controller will then raise a GPE.

Unlike normal GPEs, the EC GPE is not handled by looking for a _Lxx or _Exx method. Instead, the ACPI tables provide information about the GPE that the EC is using. This may be in the form of a _GPE definition in the EC object in the main ACPI tables, or alternatively may be provided in an ECDT (Embedded Controller Descriptor Table), an optional table that provides all the EC information. In either case, the OS knows which GPE will be triggered by the EC. It then installs a handler that will be called whenever the EC raises that GPE.

Things get a touch confusing at this point. The first thing this handler does is read the command byte, which functions as a status byte on reads. It then checks whether the SCI_EVT bit is set. This informs the system that the GPE was in response to a hardware event, and so the EC handler writes a query command to the EC command register and then reads back a value between 0 and 255 from the data register. This is then mapped to a _Qxx method, with xx representing the number of the EC event read from the data register. Like the _Lxx and _Exx methods, the _Qxx method is then executed.

The problem with all of this is that the EC isn't that fast. When a byte is written to it, it's necessary to read back the status byte and check whether the IBF bit is set. This is set when the OS writes a byte to the data register, and cleared once the EC has processed it. The straightforward way to deal with this is to poll the status byte until the bit is cleared, and then write the next byte, but polling is slow and wastes CPU time. The EC can instead be set to interrupt mode, where it'll fire a GPE when the IBF bit clears.

The EC has one additional function. The ACPI spec allows for an i2c bus to be implemented through the EC, with EC registers mapping to i2c registers. The observant among you will realise that this means that there's an indexed access protocol being implemented on top of indexed access hardware, which is more layers of indirection than seem sane. For additional humour, this is usually only used to add support for ACPI smart batteries. ACPI batteries are generally abstracted behind a set of ACPI methods that provide information. Smart batteries instead speak i2c directly to the OS[2] for no real benefit. Linux handles these devices fine, and while the chances are you probably don't have one, the chances are also that if you do you haven't noticed.

The final quirk of ACPI events is that there's yet another means of delivering events. The term "fixed feature" is used to describe an ACPI device that isn't described in the ACPI tables. A power button may be implemented as a fixed feature device rather than a normal ("control method") device. This is indicated by a flag in the fixed feature block. Hitting a fixed feature power button will generate an ACPI interrupt, but no GPE. Instead the OS has to read the fixed feature block and note that the power button flag is set there. It then notifies userspace appropriately. Sleep buttons can also be implemented this way, but other devices will be in the normal ACPI tables and will generate either GPEs or EC events.

[1] On my laptop, these are ports 0x62 and 0x66 - compare to the keyboard controller's use of ports 0x60 and 0x64

[2] As directly as indirection via the EC can be...

ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events. ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged. Then things get more interesting. The majority of GPEs are implemented in the ACPI tables via methods with names like _Lxx or _Exx. The xx is the number of the GPE in hex, while the leading _L or _E indicates whether the GPE is level- or edge-triggered. If an ACPI interrupt is fired and GPE 0x1D is flagged as being the source of the interrupt, the ACPI interpreter will then look for an _L1D or _E1D method. Upon finding one, it'll execute it. What this method does is entirely up to the firmware - on most HP laptops, GPE 0x1D is hooked up to the lid switch[1] and so executing it will send a notification to the OS that the lid switch has changed state. The OS will then evaluate the state of the lid switch (generally by making another ACPI query) and send the event up to userspace. How does the lid end up triggering GPE 0x1D? Things get pretty hardware specific at this point. Intel motherboard chipsets have a set of general purpose io (GPIO) lines that can, for the most part[2], be used by the system vendor for anything they want. For a lid switch, one of these lines is hooked to the switch and the BIOS configures the GPIO as an input. Pressing the switch will cause the GPIO line to become active. The GPIO lines are mapped to GPEs in a 1:1 manner, though with an offset of 16 - ie, GPIO 0xd will map to GPE 0x1d. If GPIO 0xd becomes active, GPE 0x1d will be flagged and an ACPI interrupt sent. The ACPI code will then do something to quash the interrupts, such as inverting the polarity of the GPIO[3], as well as send the notification to the OS. Why are the GPIOs offset by 16 relative to the GPEs? The lower 16 GPEs (again, talking about Intel hardware) have pre-defined purposes[4]. These range from things like "Critically low battery" to "PCIe hotplug event" down to "This device triggered a wakeup". And the latter is what I'm most interested in here. Various pieces of modern hardware can be placed into power saving states when not in use. The problem with this is that the user experience of having to turn on hardware before you can use it is not a good one, so in order to make this the default behaviour we need the hardware to tell us that something happened that requires us to wake the hardware up. There's something of a chicken and egg problem here, but thankfully most of the relevant modern hardware has out of band mechanisms to tell us about things going on. The PCI spec defines something called Power Management Events (PME), which are driven by an additional current that's supplied to the hardware even when it's otherwise turned off. On plug-in PCI Express cards, firing a PME generates an interrupt on the root bridge and a native driver can interpret that, but for legacy PCI devices and integrated chipset devices the notification has to come via ACPI. The example I've been working on is USB. It's a good choice for various reasons - firstly, there's already support for detecting when the USB controller is idle. Secondly, modern USB host controllers have support for generating PMEs on device insertion, removal or (and this is important) remote wakeup. In other words, as long as the USB bus is idle we can power down the entire USB controller. If the OS tries to access a USB device, we'll power it back up. If the user unplugs or plugs a device, we'll power it back up. If a previously idle device suddenly responds to some external input, we'll power it back up. And it's all nicely invisible to the user. How does this work? The controller retains a small amount of power even when nominally pwoered down. This is used to keep the detection circuitry alive. When it receives a wakeup event, it asserts the PME line. The chipset detects this and fires a GPE. The OS runs this GPE and receives a device notification on the ACPI representation of the USB controller, telling us to power it back up. We do so and process whatever woke us - if the bus then goes idle again, we can power down once more. The astonishing thing is that this all works. The only problem we have is that it relies on the machine vendor to have provided the ACPI methods that are associated with the GPEs. If they haven't, we can't enable this functionality - even though the hardware is capable of generating the GPEs, we have no method to execute to let us know which device has to be woken up. The GPE is never answered, we never acknowledge the PME and the hardware keeps on screaming for attention without getting any. And, more to the point, it never gets powered up and your mouse doesn't work. There's a pretty gross hack to deal with this. In general, we know what the GPE to device mappings are - they're pretty static across Intel chipsets, and while AMD ones can be programmed differently by the BIOS we can read that information back and set up a mapping ourselves. This trick also comes in handy when some vendors (like, say, Dell) manage to implement one of the GPE events wrongly. Everything looks like it should work, but the method never sends a notification because it's buggy. In that case we can unregister the existing method and implement our own instead. This code isn't upstream yet, but patches have been posted to the linux-acpi mailing list and with luck it'll be there in the 2.6.33 timeframe. My tests suggest about 0.2W saving per machine, which isn't going to save all that many polar bears but seems worth it anyway. [1] _L1D = lid. Sigh. [2] There's a few that are reserved for specific purposes [3] So where before it had to be high to be active, it now has to be low to be active - this means that it'll now trigger on the switch being opened rather than closed, so you'll get another event when you open the lid again. [4] You can find a list in the documentation for the appropriate ICH chip - the relevant section is "GPE0_STS" under the LPC interface chapter.

I have been reading a great many posts about Android lately, some consisting of criticism, some of praise and some simply addressing issues in the Android community . Let s have a look at those. Matt Porter s Android Mythbusters presentation and Harald Welte s reaction
I haven t seen the presentation live, but I had a look at the slides. Impressing work done by Matt putting all this information together. However, we all knew that Android only (ab-)uses Linux, without making use of the GNU userland for a long time, didn t we? In his presentation Matt has shown things such as Android s udev replacement that uses hardcoded values for device node creation and (on his blog) Harald has then come up with a statement I have found to be very strong:

The presentation shows how Google has simply thrown 5-10 years of Linux userspace evolution into the trashcan and re-implemented it partially for no reason. Things like hard-coded device lists/permissions in object code rather than config files, the lack of support for hot-plugging devices (udev), the lack of kernel headers. A libc that throws away System V IPC that every unix/Linux software developer takes for granted. The lack of complete POSIX threads. I could continue this list, but hey, you should read those slides. now!

Now both of these statements target technical details, but the root of the problem seems to be elsewhere. Where is my Android 2.0? Okay, that heading might not be making any sense in the context of this post at a first glance, but let me elaborate on that. Google and the Open Handset Alliance refer to Android as being an Open Source operating system, but the project is different from real Free Software projects: development takes place in a closed group and the results are shared with the community later on, when they are deemed to be ready. This means that innovation also takes place behind closed curtains and that the community is not involved in the actual development process at all. Lately we have seen the result of that, as Motorola is bragging about working close with Google on Android 2.0 ( Eclair ), but the AOSP source trees, open for everyone to have a look at, show no signs of version 2.0. In fact no changes that might even remotely suggest the release of a new major version have been made public in the past few weeks. So where is the openess there?
Actually, the Motorola Droid has already shipped with Eclair on 6th, but still, there is no indication that Eclair will be made available to the broader public. In short Android seems to be developed behind closed curtains, with hardly (read no) community input whatsoever and is sometimes released as Free Software, not what I would describe as an open development process. The Android Market problem As we have seen in the past Google is enforcing their copyright on proprietary applications that ship with pretty much every Android device, such as the Android Market. This has become really clear when Steve Kondik received a cease and desist letter when packing the Google-proprietary applications into his ROMs. Okay, it s Google s right to enforce their copyright and there is nothing wrong with actually doing so, the thing I really have a problem with is something else: the Market is proprietary. Now what this means should become rather clear. You can have an Android device without Google s proprietary bits, but with default settings you just do not have any way of installing additional software. In my opinion the Market should be freed by Google themselves, or the community has to react and come up with a free replacement to overcome the vendor lock-in. Oh, you might know a replacement called SlideMe (or Mobentoo) already. Well, that bugger is proprietary too, so not a solution at all. Nokia and Maemo to the rescue In most discussions about the openness of Android someone throws in Nokia and Maemo, as a solution to the dilemma. Reading all those positive comments I simply had to give it a try, but all my hopes were destroyed within a few minutes. Let s start with the good news and let alone the reason why my hopes were destroyed for another minute or two. Maemo is based on Debian GNU/Linux and various Free Software components, such as GTK+, gstreamer, esd and friends. Most of the system is Free Software which is a good thing(tm) and reading all of this really got me into Maemo. Okay, some applications seem to be proprietary, but I am sure that could be fixed rather easily, so I could once for all use a truly open phone. and then came the SDK installer shell script:

#!/bin/sh
# Copyright (C) 2006-2009 Nokia Corporation
#
# This is proprietary software owned by Nokia Corporation.
#
# Contact: Maemo Integration <integration@maemo.org>
# Version: $Revision: 1110 $

Now there is one question you should ask yourself: Why would someone trying to promote his platform as being open make the *installer* script for its SDK proprietary? Come on, it s an installer script, how much of your secret juice could be in there? What s the problem with people modifying it and working on this installer script in an open development environment? I had high hopes for Nokia actually doing a bit better than Google, but it seems they ve failed to do so. It may be me overreacting, but a proprietary SDK installer shell script scares me enough not to install the SDK and have a look at it for now nor to think about buying a Maemo-based device in the near future. Please Nokia, either get the facts straight or provide us with a free SDK to your free & open platform. So, in short, Google is bad at working with the community and creating a truly open development process, and Nokia simply fails in terms of not scaring off prospective developers for their open platform with the proprietary SDK installer. Do you have any solutions in terms of an open phone environment, apart from what OpenMoko has come up with?

The mailbox in my building is broken. Nobody can remember it being any other way. The lock is busted so anyone in the building can get access to every apartment's individual boxes in the same way that the mailman does. It's not a huge problem since there are only four apartments in the building and the box is behind a locked door to the street. I saw the mailman come one day to deliver mail. He used a key to unlock a box on the outside of the building from which he retrieved a key to first unlock the outside door and then another to "unlock" the mailbox. Every day, my mailman unlocks a mailbox that is always unlocked and, in fact, unlockable. As far as I can tell, he's been doing it for years. I don't have the heart to tell him the truth.

A periodic security review at our co-op suggested switching PasswordAuthentication no on even more hosts. One of those caused a bit of a heated discussion about the benefits of increased security and the drawbacks of making emergency access harder, reminding me of the old joke about a secure computer being one encased in a block of concrete, not connected to anything and buried in a secret location. More usefully, I found a Discussion summary on improving SSH which seemed to cover the basics pretty well. It also suggests that nearly two thirds of all SSH private keys were stored on disk with no password protection which is scary and can server admins even detect that sort of risky behaviour? One thing I ve not really noticed before is how common brute-force ssh attacks are. There are some denyhosts statistics which make interesting pictures and Strong passwords no panacea as SSH brute-force attacks rise summarises some data from last year. So what can we do? libpam-cracklib seems like a first step, along with denyhosts. While searching around, I noticed sslh ssl/ssh multiplexer which looks like a useful trick that I might try somewhere. It entered debian testing earlier this week and looks simple to backport to the stable version. What tips would you give? Do you think it s worth having at least one reasonably secure host with PasswordAuthentication enabled, just in case of disasters disabling private keys? Are you a fan of port-knocking and other more sophisticated things? Do you know more numbers about ssh security?

RCBW - week #8 This week I'm behind schedule. Part of the reason is that, after all the traveling (first the GSoC mentors summit, where I met several other Debian folks, then a work meeting in Lisbon), I got sick again for a while: plain flu, no H1N1 yet Now that I'm finally feeling better ... here is a (delayed) weekly RCBW update; as usual, dates are mostly indicative.

• 26/11/2009 bug #507201 - lsb-appchk3 - snprintf usage and undeclared NULL (Ubuntu patch)
• 27/11/2009 bug #507204 - lsb-pkgchk3 - snprintf usage and undeclared NULL (Ubuntu patch)
• 28/11/2009 bug #527754 - gsetroot - update ltmain.sh (fix FTBFS)
• 29/11/2009 bug #543553 - bum - verify with maint that his address does work
• 30/11/2009 bug #395609 - html-template - add debian/rules target "binary-arch"
• 31/11/2009 bug #533435 - pida - specific to home build Python (downgrade)
• 01/11/2009 bug #395725 - openwince-include - add debian/rules target "binary-indep"

This week, you might have noticed an upward spike of about 200 bugs in the RC bug graph: that's Manoj reporting a lot of neglected violations of policy "MUST" requirements. No matter what's your take on how those bugs have been reported, all of them (up to false positives) are indeed bugs of RC severity. So roll up your sleeves and join RCBW, we will be freezing soon.

About a year ago, Eric Poscher invented the IP address blog and installed his one at http://192.168.epe.at/. Every hour his netbook notes down the IP address of the interface which currenntly the default route goes through and if it has an internet connection, it uploads the list of IP adresses it had. Additionally, he filters the list to IP addresses in 192.168.0.0/16. June this year he published the source code behind his IP blog under GPL and Creative Commons. I modified his script slighty to just write down the new IP address if it s different to the previous one, but without any filter. This makes the list much more colorful (and my online times less traceable :-) as you can see at http://192.168.noone.org/. But the biggest disadvantage of Eric s code design is not the fact that it s a (quite nice to read :-) shell script but that it doesn t save the list of IPs separately and is not able to regenerate everything if you want to change the design, but always just adds a line to the HTML page. So I rewrote the whole thing in Perl last Saturday while sitting the dog of my parents. If you change the templates and call the script again, it regenerates the whole list with the new templates. The code is also under GPL, the HTML parts are under Creative Commons, too. And hey, this is one of the very few (if not only) applications which are much more fun with IPv4 than with IPv6. ;-)

Regular readers of this blog (yes, both of you!) may remember the computer/broadband/ directory that this post appears in as the collection of my Comcastic (yeah right) experiences with my ISP. But I think this week may top everything. I'll just try to jot down some notes before I forget all the gory details:

• On Tuesday, I edited one of the internal nvram configuration variables of my trusted wrt54g router in order to add the older daughter's shiny new iPod Touch to the set of 'permitted' MAC addresses. This router, running a custom Linux variant called OpenWRT had essentially not been upgraded since I first installed it, and still required a quick reboots after updating of configuration values. However, that worked fairly flawelessly for 4 1/2 years. Until Tuesday.
• Upon reboot, I got what appeared to be an invalid network setup from the cable modem. IP and Gateway assigned , but no DNS and no ability to ping anywhere. Crap. So I fiddled with this all evening, including a service call to Comcast but to no avail. When a laptop was directly plugged into the cable modem, it got correct settings albeit on a completely different subnet. So for the next day, we left one machine directly plugged so that my wife could at least telecommute.
• Somehow it became apparent that waiting insanely long for the router to remain powered-down -- and we're talking five minutes or longer -- helped. So by now we were suspecting the cable modem. I use a standard Motorola SB5101 I once had to buy in a rush because of the Comcastic ones who all of a sudden changed their minimum requirements which meant they would no longer connect to my existing modem. Anyway. So on Wednesday I called Motorola and had a decent service call with them but as I was at work I couldn't follow up with part numbers etc pp. At least I learned that I seem to have two months of the two-year warranty left...
• So by Wednesday evening I decided to fall back to the really cheap and old Speedstream router I had used before the Linksys wrt54g. That worked, albeit sloooooowly. Wired and wireless ethernet, direct assignment from the cable modem. All well. But did I mention it was sloooow though?
• Thursday evening was skipped as I was at the Chicago R meeting we organize to complement our R / Finance conferences in the spring.
• Given that the cable modem worked with the old Speedstream and with directly-connected machines, I decided to finally go for a long overdue update of the wrt54g software. So that happened on Friday, i.e. yesterday. And similar to my previous wrt54g notes, I needed to flash the new software with the tftp protocol and a helper script on a laptop connected to the router. All this took a while as I needed to remember to also send to a ping flood to the router to be able to catch the tftp request, needed to test which of the atftp and tftp binaries worked reliably, and whether the router prefers .bin images over .trx images when using the tftp protocol. But lo and behold this worked, and I configured a shiny new Kamikaze aka 8.09.1 version of OpenWRT. This even connected to the cable modem once I helped with DNS entries. OpenWRT generally rocks, and this new release is a lot nice than the more bare-bones version I used to run.
• Unfortunately, I had picked the bcrm47xx variant -- the 2.6.* kernel version of the OpenWRT Project's software for my WRT device. And guess what, that one does not include wireless support due to issues with Broadcom drivers and the kernel. Grrr. So once I had that confirmed this morning, I quickly switched to the bcrm-2.4 variant of the same 8.09.1 release. At least now I can flash from within using the mtd command from the commandline.
• But once up and running with the bcrm-2.4 release, I ran into the same issue we have had with the Motorola cable modem and Comcast behind them. Each time I connect with the wrt54g, I end up on a specific subnet, without DNS and with no ability to connect. The Speedstream still worked. So what to do? Well, MAC Cloning to the rescue. Now the Linksys wrt54g pretends to be the Speedstream, and all, at last, is well again.
• So after four days of intermittent service, which means that my few web pages, blog, and goodies like CRANberries were invisible, I now have better router software. That could have come a little easier, and I still don't quite know why Comcast decides to no longer service the wrt54g under the MAC address it presented itself with for 4 1/2 years. I have paid thousands of dollars over that time to get broadband access. But this, I don't quite call service. To top it all off, guess who cold-called to sell VOIP service while I wrote this up? Oh, it's Comcastic ...

Here we are, still writing code, still doing 16hour days, still keeping the archive cronjobs disabled, still having fun. Yesterday we enabled the rsyncable option for all the gzip contents we are creating in our dists/ directory, which will lead to nice bandwidth savings during the mirror push. Initial tests showed a size increase of about 2 til 4 percent but the transferred amount of data going down to 1/8th for Packages files, 1/4th for Contents files. We also started implementing an ACL system in DAK which basically will allow the sane implementation of the already mentioned buildd autosigning, decrufting the code handling DMs and later on easily allowing the throw away binaries feature, while still allowing for exceptions, like bootstrapping a new architecture. The majority of it is there and already working, just a few edges still have to be smoothened. Yesterday evening we started the last major work for this meeting, in terms of coding. We will end up with a completly different way of handling the lifecycle of a package in the archive, before it enters the public viewable pool/ directory. To wrap our heads around the problem we had to write down how the archive works currently, which you can see in the first picture. And then we had to go and define how it should look in the future, for which you can look at the second picture. As all of us here love the simple clarity of this drawing we went and declared it the one and official ftpmaster documentation of our archive queues. :) As you might miss enough Milk/IceTea/Coffee/Drugs to understand what the fuck we smoked when drawing it, lets try a short explanation, starting with the old red version:

• Packages get in as usual, via our queue daemon, which copies them into unchecked/
• From there packages either go to NEW/BYHAND, get rejected, end up in a policy-queue (which is what you knew by (o-)p-u-new in the past, but more generally implemented) or get accepted and end up in accepted/. accepted/ is what you see behind incoming.debian.org.
• From NEW/BYHAND or a policy-queue packages either go to rejected, should they be unwanted for the archive, for whatever reason. Or into newstage if they are accepted. Newstage will simply end up in the accepted/ queue the next time we process files and is a neat way to avoid a class of race conditions we had in the past.
• From the accepted queue packages are usually installed into the pool/. This is the real acceptance into the archive, after this they are considered public. Very seldomly there is a problem doing this and then we end up with an automated unaccept of them, but an unaccept there is usually never done by hand.
• When a package is removed from the archive it will end up in the morgue. The same happens for files out of the reject queue.
• Every now and then we have a huge shellscript running which we refer to as dinstall, which is triggering the installation of files into the pool, is responsible for the Packages/Sources files creation, pushing the mirrors, etc.

This is how the archive worked for the last years now, and while it works it has a few drawbacks. For example, files do have to be processed multiple times before they end up in their final location, which is just unneccessary. It also opens us to a kind of race condition between the unchecked and accepted queue, leading to automated unaccepts. And it duplicates some code and also checks we have to run, wasting time and resources. Thats not all of it, but a good start. So the idea is to get rid of the extra queue and switch over to install files directly into the pool when processed from the unchecked queue. And that is what the third picture shows in green, the changed handling. As you see we completly lose the complexity around accepted by moving directly into pool. So, as usual, saying it in two sentences doesn t mean the actual implementation is easy, even if it will make future handling easier for us. Of course this hasn t been all we worked on today. We are 6 persons here, there is much more done than I can condense into those few blog posts. Especially as I want to do my own share of work (shut up mhy :) ), and not just blog around. So, to still keep you informed I included our official meeting minutes in this blog post. Picture 3 is your friend. :) (Ok ok, just joking, but it contains a good overview of our various discussion about the various projects we work on). We are currently running a script that is importing data about all known changes files the archive ever has seen (since about 2002), together with the most important data from them. While this allows only a small function for us in the archive (have we seen this file already? Reject upload) using it (prior it used to look for file existance in a subdir), it might allow other people to do creative things with it. We also added a new Release file to our mirrors. They are called InRelease and are the same as the Release file, but they are signed inline (cleartext signature). Using those files it can no longer happen that one downloads a Release file, the mirror updates and one downloads a different Release.gpg. This will help reducing problems experienced there, even if it is only the first step towards the final solution. For the final solution we have to do something about the Packages files related to the Release files to, and allow multiple versions of them to exist and then let the Release file tell for which of them they are. We probably will end up with the Packages/Sources files of the last 2 or 3 runs and the Release file mentioning them all, but that won t be done during this meeting. Oh, and as you have all be so patient with us during the last days of reduced archive service: Our current plan is to come back to a mostly normal service during Saturday, so we have the rest of Saturday and the few hours on Sunday which we are here to monitor what it is doing. Thats it for today, but you can sure expect a more detailed report send to d-d-a sometime next week.

Compleat is an easy, declarative way to add smart tab-completion for any command-line program. For a quick description, see the README. For more explanation and a brief tutorial, keep reading... Background I'm one of those programmers who loves to carefully tailor my development environment. I do nearly all of my work at the shell or in a text editor, and I've spent a dozen years learning and customizing them to work more quickly and easily. Most experienced shell users know about programmable completion, which provides smart tab-completion for for supported programs like ssh and git. (If you are not familiar it, you really should install and enable bash-completion, or the equivalent package for your chosen shell.) You can also add your own completions for programs that aren't supported but in my experience, most users never bother. When I worked at Amazon, everyone used Zsh (which has a very powerful but especially baroque completion system) and shared the completion scripts they wrote for our myriad internal tools. Now that I'm in a startup with few other command line die-hards, I'm on my own when it comes to extending my shell. So I read the fine manual and started writing my own completions. Over on GitHub you can see the script I made for three commands from the Google Android SDK. It's 200 lines of shell code, fairly straightforward if you happen to be familiar with the Bash completion API. But as I cranked out more and more case statements, I felt there must be a better way... The Idea It's not hard to describe the usage of a typical command-line program. There's even a semi-standard format for it, used in man pages and generated by libraries like GNU AutoOpt. Here's one for android, one of the SDK commands supported by my script:

 android [--silent   --verbose]
   ( list [avd target]
     create avd ( --target <target>   --name <name>   --skin <name>
                   --path <file>   --sdcard <file>   --force ) ...
     move avd (--name <avd>   --rename <new>   --path <file>) ...
     (delete update) avd --name <avd>
     create project ( (--package --name --activity --path) <val>
                       --target <target> ) ...
     update project ((--name --path) <val>   --target <target>) ...
     update adb )

My idea: What if you could teach the shell to complete a program's arguments just by writing a usage description like this one? The Solution With Compleat, you can add completion for any command just by writing a usage description and saving it in a configuration folder. The ten-line description of the android command above generates the same results as my 76-line bash function, and it's so much easier to write and understand! The syntax should be familiar to long-time Unix users. Optional arguments are enclosed in square brackets; alternate choices are separated by vertical pipes. An ellipsis following an item means it may be repeated, and parentheses group several items into one. Words in angle brackets are parameters for the user to fill in. Let's look at some more features of the usage format. For programs with complicated arguments, it can be useful to break them down further. You can place alternate usages on their own lines separated by semicolons, like this:

android <opts> list [avd target];
android <opts> move avd (--name <avd> --rename <new> --path <file>)...;
android <opts> (delete update) avd --name <avd>;

...and so on. Rather than repeat the common options on every line, I used a parameter <opts>. I can define that parameter using the same usage syntax.

opts = [ --silent   --verbose ];

For parameters whose values are not fixed but can be computed by another program, we use a ! symbol followed by a shell command to generate completions, like this:

avd = ! android list avd   grep 'Name:'   cut -f2 -d: ;
target = ! android list target   grep '^id:'  cut -f2 -d' ' ;

And any parameter without a definition will just use the shell's built-in completion rules, which suggest matching filenames by default. The README file has more details of the usage syntax, and instructions for installing the software. Give it a try, and please send in any usage files that you want to share! (Questions, bug reports, or patches are also welcome.) Future Work For the next release of Compleat, I would like to make installation easier by providing better packaging and pre-compiled binaries; support zsh and other non-bash shells; and write better documentation. In the long term, I'm thinking about replacing the usage file interpreter with a compiler. The compiler would translate the usage file into shell code, or perhaps another language like C or Haskell. This would potentially improve performance (although speed isn't an issue right now on my development box), and would also make it easy for usage files to include logic written in the target language. Final Thoughts Recently I realized that parts of my work are so specialized that my parents and non-programmer friends will probably never really get them. For example, Compleat is a program to generate programs to help you... run programs? Sigh. Well, maybe someone out there will appreciate it. Compleat was my weekends/evenings/bus-rides project for the last few weeks (as you can see in the GitHub punch card), and my most fun side-project in quite a while. It's the first "real" program I've written in Haskell, though I've been experimenting with the language for a while. Now that I'm comfortable with it, I find that Haskell's particular combination of features works just right to enable quick exploratory programming, while giving a high level of confidence in the behavior of the resulting program. Compleat 1.0 is only 160 lines of Haskell, excluding comments and imports. Every module was completely rewritten at least once as I tried and compared different approaches. This is much less daunting when the code in question is only a couple dozen lines. I don't think this particular program would have been quite as easy to write at least for me in any of the other platforms I know (including Ruby, Python, Scheme, and C). I had the idea for Compleat more than a year ago, but at the time I did not know how to implement it easily. I quickly realized that what I wanted to write was a specialized parser generator, and a domain-specific language to go with it. Unfortunately I never took a compiler-design class in school, and had forgotten most of what I learned in my programming languages course. So I began studying parsing algorithms and language implementation, with Compleat as my ultimate goal. My good friend Josh and his Gazelle parser generator helped inspire me and point me toward other existing work. Compleat actually contains three parsers. The usage file parser and the input line tokenizer are built on the excellent Parsec library. The usage file is then translated into a parser that's built with my own simple set of parser combinators, which were inspired both by Parsec and by the original Monadic Parser Combinators paper by Graham Hutton and Erik Meijer. The simple evaluator for the usage DSL applies what I learned from Jonathan Tang's Write Yourself a Scheme in 48 Hours. And of course Real World Haskell was an essential resource for both the nuts and bolts and the design philosophy of Haskell. So besides producing a tool that will be useful to me and hopefully others, I also filled in a gap in my CS education, learned some great new languages and tools, and kindled an interest in several new (to me) research areas. It has also renewed my belief in the importance of "academic" knowledge to real engineering problems. (I've already come across at least one problem in my day job that I was able to solve faster by implementing a simple parser than I would have a year ago by fumbling with regexes.) And I'll be even happier if this inspires some friends or strangers to take a closer look at Haskell, Parsec, or any problem they've thought about and didn't know enough to solve. Yet.

A lot of people used to configure their ssh servers to listen to a port different from 22 to prevent their logs getting filled with login attempts by script kiddies. I just stumbled over sslh which is a very nice alternative to that in case you have a webserver running which supports ssl. sslh is also a good choice if you are behind a corporate firewall and ssh is filtered while SSL is probably not (and this is simpler than tunneling).

The idea is pretty simple but cool. The daemon listens on port 443 and depending on whether the client has written something into the socket (SSL) or not (SSH) after connecting the daemon "forwards" the connection to the listening service running on the same host. In case of SSL/HTTPS the client will send a request after the handshake and in case of ssh it won't. Pretty simple. I just did a quick test on one of my hosts and it seems to work great. The tool is written in C, consists of < 500 SLOC and the code looks clean to me. If you're using Debian you can just apt-get install it.

For non-Dutch speaking readers: this is about a petition seeking to limit what information communication providers should retain in accordance to the EU Data Retention Directive.
It is only useful to sign this petition if you actually live in Belgium.

Voor al wie enigszins met z'n privacy begaan is: gelieve hier de petitie tegen algemene bewaarplicht voor internetproviders te gaan tekenen.

So Tuesday was another very productive day seeing lots of things done. And actually today also started (and will end) like this. I am now of the opinion that we need such a kind of meeting of our team every 6 months, if only we could manage to get the time off from our real lifes. Sitting together, discussing, working on the same thing, being able to just call over the table and get a different opinion just gains so much that we are missing otherwise As you already read, Monday was our day to change the database layer while Tuesday left us with adding the v3 source formats. Which was declared finished today, somewhere around noon. The first picture in this blogpost shows our still to fix up/todo list from Monday evening. Besides the v3 format we also worked on other things, like the already mentioned lintian autorejects. This patchset is also fully committed and tested working since today, so we could start on other issues. We had lots of general enhancements of our code committed, from improving our logging to fixing some issues with expiring old files from the pool all the way up to reducing code duplicates all over the place. I must confess that we did not start right away working on dak today. . We had to start by using hightech to fix a broken shoe first. (Some glue and a heavy laptop. NOT MINE). :) Back to reality - we are currently 6 persons all working on various parts of the dak code. We still have the archive mostly disabled (basically one large dinstall run in the evening always) and are happy with this, as this allows us immediate testing without the risk of accidently destroying someones package upload. And we do test a lot, haven t seen that many python backtraces before in one place :) The third picture shows our todo list status from today. We still have a good number of points open, but as we have 6 persons working it does seem manageable to at least get a very good start on each of them. What we currently work on is ACLs in the archive, which will soon allow us to fix another issue the old DM code has but even more important will allow automated signing of packages built by buildds soon. We basically are able to limit such keys to only be able to upload for the one architecture they are allowed to, as well as forbid them to upload any kind of source. (Together with various other considerations around the setup of the buildds (like, for example, only buildds administrated by DSA can have that, but other points too), the handling of the keys, etc, this will lead to much faster uploads of the build binary packages). And will take a tedious work away from buildd admins (who will still have enough to do to deal with the general administration of it, the build failures, such things). Another thing we work on is the handling of arch all packages, in a source that produces arch all as well as arch any binaries. Currently, as soon as a new source is uploaded for one architecture, every architecture will see the new arch all deb. Which can lead to uninstallability of other packages. Not very nice. And another one of us is also working on our contents generation. Something started long ago, but not yet finished. Recently we did take big steps in this, getting our SQL queries for it down from about 10 minutes to something around 10 seconds. That is for one suite, one architecture, the full contents file contents. While this is SQL only and we still need to write it out to disk, it is a nice improvement. And to my right we have someone working on getting the Packges/Sources files data stored within projectb too, so we can also use our own python scripts to write those files. This will finally enable us to completly ditch apt-ftparchive. And finally, to my left site sits someone who is working on a dak policy wrapper, with which we can allow other groups then ftpmaster to control suites. The initial usage will be to restore the (oldstable-)proposed-updates handling with their own queues in front, as that is no longer part of our NEW handling script. In the future it will also allow easy handling of other situations, like suites for volatile or backports.org. Basically everything where the people running the archive software are different to those people actually setting the policy for the suite. Another nice detail is the ability to throw away uploaded binaries and only ever take buildd built binaries into the archive. We will not go to source-only uploads, as that will only lead to lots of untested uploads (sorry people, we are all humans, it WILL happen), but we can have them uploaded, do regular checks (including the lintian ones), then take the files from the buildds. (Of course there is a way to get things bootstrapped, the system behind this is flexible to allow for a bootstrap mode for a new architecture). (I have multiple pictures, will probably put some more up when I am back home).

Couchdb came onto my radar since distributed stuff is interesting to me these days. But most of what was being written about it put me off, since it seemed to be very web-oriented, with javascript and html and stuff stored in the database, served right out of it to web browsers in an AJAXy mess. Also, it's a database. I decided a long, long time ago not to mess with traditional databases. (They're great, they're just not great for me. Said the guy leaving after 5 years in the coal mines.) Then I saw Damien Katz's talk about how he gave up everything to go off and create couchdb. Was very inspirational. Seemed it must be worth another look, with that story behind it. Now I'm reading the draft O'Rielly book, like some things, as expected don't like others[1], and am not sure what to think overall (plus still have half the book to get through yet), but it has spurred some early thoughts: ... vs DVCS Couchdb is very unlike a distributed VCS, and yet it's moved from traditional database country much closer to VCS land. It's document oriented, not normalized; the data stored in it has significant structure, but is also in a sense freeform. It doesn't necessarily preserve all history, but it does support multiple branches, merging, and conflict resolution. Oddly, the thing I dislike most about it is possibly its biggest strength compared to a VCS, and that is that code is stored in the database alongside the data. That means that changes to the data can trigger processing, so it is mapped, reduced, views are updated, etc, on demand. This is done using code that is included in the database, and so is always available, and runs in an environment couchdb provides -- so replicating the database automatically deploys it. Compare with a VCS, where anything that is triggered by changes to the data is tacked onto the side in hooks, has to be manually set up, and so is poorly integrated overall. Basically, what I've been doing with ikiwiki is adding some smarts about handling a particular kind of data, on top of the VCS. But this is done via a few narrow hooks; cloning the VCS repository does not get you a wiki set up and ready to go. There are good reasons why cloning a VCS repository does not clone the hooks associated with it. The idea of doing so seems insane; how could you trust those hooks? How could they work when cloned to another environment? And so that's Never Been Done[2]. But with couchdb's example, this is looking to me like a blind spot, that has probably stunted the range of things VCSs are used for. If you feel, like I do, that it's great we have these amazing distributed VCSs, with so many advanced capabilities, but a shame that they're only used by software developers, then that is an exciting thought.

---

[1] Javascript? Mixed all in a database with data it runs on? Imperative code that's supposed to be side-effect free? (I assume the Haskell guys have already been all over that.) Code stored without real version control? Still having a hard time with this. :) [2] I hope someone will give a counterexample of a VCS that does so in the comments?

Note: via Blog post (obviously different formatting) and sent by email roughly: Lying in one, lying in all. That is from Steve Waddington. Please see the attached screenshot I have obliterated service identifying information. I find it telling that what I said in my original post, of all the things there the one you decided to fact-check was the number of emails between us. And you got that wrong. What was is you said again? Falsum in uno, falsum in omnibus? Do you still stand by that statement? If you have looked at the original complaint, no one asked for anything but information on 'what went wrong'. It simply noted that fact that payment occurred, and we were disconnected and went on to say "We find the above experience of diconnecting [sic] our service by EXETEL a very poor customer relationship experience." The response to that could have been: "Yes, that is a poor customer experience. We will look into it and see if we can improve." Instead we had an accusatory email back commenting, firstly, of the amount of time of the disconnection. As if someone paying for something and not receiving it for only 1 minute is OK. The point being missed we paid for something and we did not receive it. The point being missed -- this is a poor customer experience The second email then, falsely, accused us of not actually paying in the first place! Later the individual concerned, after the TIO complaint, clarifies that 'yes', we did indeed pay. Very telling indeed, then, when you were pointed to my blog entry and rather than pick up on the stratgeic ("we are getting feedback, for free, on how we can improve our processes") you looked at the tactical ("he is lying! How I prove it and thus be happy in ignoring what was said"). As I said, It is easier to hear when your fingers are not in your ears. In light of your recent blog post, and the recently released TIO report, I find Exetels' behaviour all the more perplexing. Complain to Exetel, get your service contract summarily terminated. Complain about Exetel, and get libelled publicly. But I guess when you have 1/6th (163 out of 620) of your complaints related to Customer Service (from the TIO report) and 3% of your own existing customers (from your blog post) complaining, you get the reputation you earn.

For software, the question whether some software can be considered free is relatively easy to decide: Check the license of the code against the list of free licenses of your trust (the OSI list, the FSF list or the judgement applied by debian-legal) and that s (mostly) it. When it comes to web services in the broader sense, including hosters, communication providers, social networks etc., this are not so clear. In this blog entry, I ll try to list the various requirements that a service should fulfill and which I usually consider before deciding to use a service.The RequirementsA) Availability of source codeComing from the Free Software movement, this is the first requirement that comes to mind. If a web service claims to be free, then I want to be able to inspect how it is working, suggest and implement improvements and set up my own instance. For example, Canonical s software housing setup Launchpad was itself not Free Software until very recently. Having access to the source code is also the reason why I prefer gitorious over github for git hosting.B) Access to my dataDepending on the kind of service, it might hold data from me or about me that are important, such as my emails on a webmail service. If I won t have a way to easily retrieve my data to back it up or to move to a competitor, I ll think twice before trusting my data to this service.C) Free data formatsIf there is accessible data, it should be in a format that I can easily get and use with a choice of (Free) software. Not sure if such a service exists, but an online office suite only supporting its own proprietary file format (possibly with its own proprietary offline office suit) would be no option for me.D) An APIFor an extensively used service, I will want to have the possibility to optimize my user experience by automation. This requires that I can use the service not only in person, but also from tools I write. A very good example for this is Twitter: Besides registration, there is no need to use their web site at all and still use all the features, using the tools I find most fit for a task.E) Choice of providersCompetition is good. If a service is only provided by a single company, and I become dependent on the service, I become dependent on the company. A few years ago, SourceForge was mostly the only provider of gratis, comprehensive software project hosting. By now, there are a few alternatives, such as alioth, savannah or google code, which is appreciated. Note that when it comes to the proprietary instant messaging services (ICQ, yahoo messenger, MSN messenger), there is no choice of providers: To talk to my buddies on a certain network, I am bound to that provider. The same applies to social networks (Facebook, StudiVZ, XING), and is one of the reasons why I shun these yet. F) Federation between providersIf there is a choice of providers, and the service in question provides some kind of communication, it is very desirable if I can interact with users of the service on other providers. A very good example for this is e-mail: People @gmx.net can send mails to people @aol.com, and I can have @joachim-breitner.de and still communicate with everyone. This was not always the matter of course: Before Internet e-mail was common, people on AOL could not get in touch with people at CompuServe. These days, the problem persists with instant messaging (Jabber a.k.a. XMPP being the notable exception) and social networks. Also sites like gitorious or github could support federation if they would allow to create branches of repositories that are hosted elsewhere, allowing their users to use the familiar interface with all repositories they work with.Analysis of a few servicesSo far the theory. Let s see how various services fare.e-mailBeing from the old times of the Internet, when everything was better, it does not surprise that this passes all requirements: There is a large amount of Free e-mail servers and clients to choose from, my data is easily moved via well-known protocols such as POP or IMAP and save in standardized format. There is an abundance of service providers, and they all talk to each other. One could argue that the openness of email causes problems like spam, but I d say that this might be a design issue, but not inherent to the above criteria.Pretty much the same analysis applies to Jabber/XMPP.ICQI joined when I was young and unwise. Now I am stuck to AOL to keep in touch with a relevent part of my online correspondence. To use it on the operating system of my choice, other people had to reverse engineer the protocol and any time, AOL could change how its servers work and I ll be locked out. I can not get in touch with people on other networks, my buddy roster is not easily backed up and restored, and no, we can not see the source of the ICQ servers. I d wish I could switch over to Jabber completely, but there are always a few important people who do not use that yet.Github/GitoriousAlthough gitorious is slightly older, github managed to become more famous. These web sites allow developers to publish their source code and other contributors to easily branch, work on their changes, and propose them for merging in the main code. They are based on git, so I can get my data. There is a choice of providers. I am not sure how well I can access their functionality (merge requests etc.) programmatically. As said before, they do not interact well, and only gitorious has published its code.Facebook/StudiVZVery disappointing. I have heard that Facebook offers you a rich API, but otherwise, they fail every category. This is one of the reasons why I am not to be found on these networks yet.Twitter/identi.caTwitter, similar to Facebook, excels in the category API , but fails in the others. A better alternative is identi.ca: The software behind it is free, I can download contact data etc. It even supports federation via OpenMicroBlogging, although that did not break through yet it, it seems.OpenIDAnother very good example for what I want is OpenID, a service that allows websites to accept a user s credentials from another site. This saves the user from having to create accounts everywhere. There are free implementations, multiple providers, I can easily switch between them. A great feature of OpenID that goes a bit beyond federation is delegation: On the start page of my homepage http://www.joachim-breitner.de/, I saved who my OpenID provider is. Now I can use http://www.joachim-breitner.de/ as my OpenID, and even if I change my provider, this stays valid! This is a feature that not even XMPP easily provides.ConclusionWhether a web service is really free is a complicated question and has a graded answer. One has to balance the benefits of using a web service against the freedoms it is missing. I hope that in the long run, services fulfilling most or all of these requirements will prevail, so that I ll be networking socially and blogging microly with a good conscience.

This is just a reminder for all of you that have packages that depend on the udev package: I hate you. A Debian package depending on the udev package (with very few exceptions like for example the initramfs-tools package that actually uses udev) is so wrong.

• The majority of those packages will work just fine without. All that is lost is some autoconfiguration of devices, which can be properly be done by hand. So even if you feel like it would not be the same without, that is at most a reason for a recommend.
• Depending on a package whose sole purpose is to heavily modify the behaviour of the system and especially what modules are loaded at boot time and how device nodes are named is a no-go in my eyes.
  Installing udev on a system not having udev installed has a realistic chance of breaking something or even making the system totally unbootable. (That is no bug in udev, it's main purpose is to load all the stuff and do much guessing. Unless you are able to write omniscient code there is no way for such a system to know what has to be changed to make the system still bootable with udev suddenly (re)installed in all cases.).
• Before anyone wants to start to argue with some hypothetical uninformed user that could be confused by some magic not working: please switch on your brain and reconsider.
  Udev has not be installed in new installs since ages. Lenny shipped without yaird so it is all but easy to have a system without udev running (if wanting to have a kernel installed). So if there is any user not having udev installed ending up with your package installed, that system was either time-consuming modified to not have udev or never had and was upgraded multiple times (thinking of a squeeze package) with each upgrade doing additional complex steps.
  What do you think are the chances on such a system for a dependency on udev to do any good? And what are the chances for it to cause a catastrophic failure?

Howdy, i m sure most people are aware of the recent release of Moblin 2.0; a user experience for netbooks. I m going to write a few blog posts about how the Moblin user experience is built on the awesome technologies in the GNOME platform. So first up, let s look at the Myzone, we re starting here since this is the first thing I really worked on in the Moblin UX and i ve been able to see it through from early ideas to the 2.0 and 2.1 releases. So, deep breath, the idea behind the Myzone is to provide a springboard to things that matter to you most: your recent files and web pages you ve visited, your upcoming events and things you need to do, things that are happening on social web services and your favourite applications. Now then, that s the theory, how does it work:

• Recent files: Recent file information is pulled from the GtkRecentManager and the thumbnails are pulled from the XDG thumbnail specification directory. Metadata for the file comes courtesy of gio which I presume comes from shared-mime-info. Yay. By using the GtkRecentManager for all our recent activity metadata across the platform we re allowing legacy GNOME applications to just work. Sweet.
• Events and tasks: These are pulled from EDS using libjana, a calendaring library primarily developed by Chris Lord (of Dates fame.) A couple of months back (well, uh, March) I enhanced libjana to support tasks and thus we are able reuse the existing Tasks/Dates apps for interacting with the calendar.
• Favourite apps: Here I let the side down. I use some quite crazy custom format for doing this which frankly stinks. I m going to try and sit down with the GNOME shell guys to see if we can come up with some better way for dealing with user originated application metadata.
• Social networking/web service integration: This comes courtesy of Mojito and librest, two projects that I and the esteemed Ross Burton have been working on. Mojito is a project that pulls in content from a variety web services into a centralised place, abstracting some of the complexity and the makes it trivial to query. librest is a library for to keep developers happy even though they re having to deal with web services. It does this by making requests and parsing the result simple.

Myzone in action