A lot of people seem to think that open source is a magic solution to project management and that open source projects will automatically attract a large and healthy community of contributors and users who will improve the software. This, of course, is not the case. In fact, creating a successful open source project is a really major and difficult effort. You have to deliver an initial promise that people find interesting, attract other people, then facilitate and lead the community, etc. You just have to look at all the failed projects on SourceForge that never delivered any code to see that "open source" is not a guarantee for success. Even though project management is a key element of every open source project, there are only few resources about this topic. That's why I always enjoy reading about the experience from open source project leaders. Jim Hall, the founder of the FreeDOS project, recently posted a series of four articles which I find particularly interesting. Here are links to the articles along with a quick summary:

• Free and Open Source Software: the first article just describes what free software and open source are all about. It also looks at Raymond's model and compares the cathedral and the bazaar style of development.
• Open source software in the real world: this article summarizes a number of key lessons learned: every project needs a clear purpose (i.e. you have to solve an actual problem); initial users of the software should be recruited as developers; releases are important; and every project needs an active coordinator or maintainer.
• Cultivating Open Source Software: you need a web site; making the source code availability in an easy way is important; documentation is often hard to write but is vital; you need a bug tracking system and responding to bug reports is important to attract good feedback.
• Transitions in an open source software project: finally, when you need to hand over the project, make sure to communicate openly, arrange for your replacement and stick around to ensure a successful hand over.

I really like these articles from Jim Hall since they contain a lot of great insights that apply to other projects, so I suggest you check them out! (Originally published on FOSSBazaar)

Earlier in the year I bought some stuff from UK clothes retailer USC. USC kindly included some gift vouchers in my delivery. This is a good example of how to sweeten a deal and make your customers happy. The vouchers did not include any expiry date on them. I banked them until the autumn, so I could maximize their potential by using them in conjunction with a heavy winter coat. I recently spotted a coat I liked a lot. It's a bit pricey at nearly 200, so I decided to cash in my vouchers. Unfortunately, the vouchers didn't work Frustrating! I emailed the company to ask them why, and it turns out that -- despite not including a printed expiry notice -- the vouchers had expired. I asked the company if they would be willing to issue me with a replacement voucher, and they stopped replying to my emails. This is a good example of how to really sour a customer's impression of you and make them look elsewhere. I did so, and found another retailer selling the coat 20 cheaper.

Of course, the event model I described before is far too simple to be worthy of a place in the ACPI spec. At the most basic level, there's more possible events than there are GPEs to attach them to, so there's a need for some further complexity. This manifests itself in the form of the ACPI embedded controller (EC).

The EC is typically a small microprocessor sitting on your motherboard, often implemented in the same hardware as the keyboard controller. It shares a lot in common with the keyboard controller - on PCs it'll usually appear in system io space, with one register for writing a command or reading a status, and a second register for passing data back and forth[1]. There's 256 registers available, so a typical interaction might be to write the READ command (0x80) to the command register, write the EC register address to the data register and then read back from the data register to get the EC register contents.

The embedded controller will often be responsible for tracking information about the hardware, such as the temperature. Attempting to read the temperature through ACPI will execute an ACPI method - in the case of the temperature being monitored by the embedded controller, this method will attempt to read from an EC register. The EC driver then performs the read and returns the result, which gets converted into decidegrees kelvin and passed back to whatever made the temperature query.

But, as mentioned above, the EC also generates events. These may be in response to a user initiated event like a hotkey press, or may be triggered by some change in hardware state like a thermal trip point being passed. The embedded controller will then raise a GPE.

Unlike normal GPEs, the EC GPE is not handled by looking for a _Lxx or _Exx method. Instead, the ACPI tables provide information about the GPE that the EC is using. This may be in the form of a _GPE definition in the EC object in the main ACPI tables, or alternatively may be provided in an ECDT (Embedded Controller Descriptor Table), an optional table that provides all the EC information. In either case, the OS knows which GPE will be triggered by the EC. It then installs a handler that will be called whenever the EC raises that GPE.

Things get a touch confusing at this point. The first thing this handler does is read the command byte, which functions as a status byte on reads. It then checks whether the SCI_EVT bit is set. This informs the system that the GPE was in response to a hardware event, and so the EC handler writes a query command to the EC command register and then reads back a value between 0 and 255 from the data register. This is then mapped to a _Qxx method, with xx representing the number of the EC event read from the data register. Like the _Lxx and _Exx methods, the _Qxx method is then executed.

The problem with all of this is that the EC isn't that fast. When a byte is written to it, it's necessary to read back the status byte and check whether the IBF bit is set. This is set when the OS writes a byte to the data register, and cleared once the EC has processed it. The straightforward way to deal with this is to poll the status byte until the bit is cleared, and then write the next byte, but polling is slow and wastes CPU time. The EC can instead be set to interrupt mode, where it'll fire a GPE when the IBF bit clears.

The EC has one additional function. The ACPI spec allows for an i2c bus to be implemented through the EC, with EC registers mapping to i2c registers. The observant among you will realise that this means that there's an indexed access protocol being implemented on top of indexed access hardware, which is more layers of indirection than seem sane. For additional humour, this is usually only used to add support for ACPI smart batteries. ACPI batteries are generally abstracted behind a set of ACPI methods that provide information. Smart batteries instead speak i2c directly to the OS[2] for no real benefit. Linux handles these devices fine, and while the chances are you probably don't have one, the chances are also that if you do you haven't noticed.

The final quirk of ACPI events is that there's yet another means of delivering events. The term "fixed feature" is used to describe an ACPI device that isn't described in the ACPI tables. A power button may be implemented as a fixed feature device rather than a normal ("control method") device. This is indicated by a flag in the fixed feature block. Hitting a fixed feature power button will generate an ACPI interrupt, but no GPE. Instead the OS has to read the fixed feature block and note that the power button flag is set there. It then notifies userspace appropriately. Sleep buttons can also be implemented this way, but other devices will be in the normal ACPI tables and will generate either GPEs or EC events.

[1] On my laptop, these are ports 0x62 and 0x66 - compare to the keyboard controller's use of ports 0x60 and 0x64

[2] As directly as indirection via the EC can be...

ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events. ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged. Then things get more interesting. The majority of GPEs are implemented in the ACPI tables via methods with names like _Lxx or _Exx. The xx is the number of the GPE in hex, while the leading _L or _E indicates whether the GPE is level- or edge-triggered. If an ACPI interrupt is fired and GPE 0x1D is flagged as being the source of the interrupt, the ACPI interpreter will then look for an _L1D or _E1D method. Upon finding one, it'll execute it. What this method does is entirely up to the firmware - on most HP laptops, GPE 0x1D is hooked up to the lid switch[1] and so executing it will send a notification to the OS that the lid switch has changed state. The OS will then evaluate the state of the lid switch (generally by making another ACPI query) and send the event up to userspace. How does the lid end up triggering GPE 0x1D? Things get pretty hardware specific at this point. Intel motherboard chipsets have a set of general purpose io (GPIO) lines that can, for the most part[2], be used by the system vendor for anything they want. For a lid switch, one of these lines is hooked to the switch and the BIOS configures the GPIO as an input. Pressing the switch will cause the GPIO line to become active. The GPIO lines are mapped to GPEs in a 1:1 manner, though with an offset of 16 - ie, GPIO 0xd will map to GPE 0x1d. If GPIO 0xd becomes active, GPE 0x1d will be flagged and an ACPI interrupt sent. The ACPI code will then do something to quash the interrupts, such as inverting the polarity of the GPIO[3], as well as send the notification to the OS. Why are the GPIOs offset by 16 relative to the GPEs? The lower 16 GPEs (again, talking about Intel hardware) have pre-defined purposes[4]. These range from things like "Critically low battery" to "PCIe hotplug event" down to "This device triggered a wakeup". And the latter is what I'm most interested in here. Various pieces of modern hardware can be placed into power saving states when not in use. The problem with this is that the user experience of having to turn on hardware before you can use it is not a good one, so in order to make this the default behaviour we need the hardware to tell us that something happened that requires us to wake the hardware up. There's something of a chicken and egg problem here, but thankfully most of the relevant modern hardware has out of band mechanisms to tell us about things going on. The PCI spec defines something called Power Management Events (PME), which are driven by an additional current that's supplied to the hardware even when it's otherwise turned off. On plug-in PCI Express cards, firing a PME generates an interrupt on the root bridge and a native driver can interpret that, but for legacy PCI devices and integrated chipset devices the notification has to come via ACPI. The example I've been working on is USB. It's a good choice for various reasons - firstly, there's already support for detecting when the USB controller is idle. Secondly, modern USB host controllers have support for generating PMEs on device insertion, removal or (and this is important) remote wakeup. In other words, as long as the USB bus is idle we can power down the entire USB controller. If the OS tries to access a USB device, we'll power it back up. If the user unplugs or plugs a device, we'll power it back up. If a previously idle device suddenly responds to some external input, we'll power it back up. And it's all nicely invisible to the user. How does this work? The controller retains a small amount of power even when nominally pwoered down. This is used to keep the detection circuitry alive. When it receives a wakeup event, it asserts the PME line. The chipset detects this and fires a GPE. The OS runs this GPE and receives a device notification on the ACPI representation of the USB controller, telling us to power it back up. We do so and process whatever woke us - if the bus then goes idle again, we can power down once more. The astonishing thing is that this all works. The only problem we have is that it relies on the machine vendor to have provided the ACPI methods that are associated with the GPEs. If they haven't, we can't enable this functionality - even though the hardware is capable of generating the GPEs, we have no method to execute to let us know which device has to be woken up. The GPE is never answered, we never acknowledge the PME and the hardware keeps on screaming for attention without getting any. And, more to the point, it never gets powered up and your mouse doesn't work. There's a pretty gross hack to deal with this. In general, we know what the GPE to device mappings are - they're pretty static across Intel chipsets, and while AMD ones can be programmed differently by the BIOS we can read that information back and set up a mapping ourselves. This trick also comes in handy when some vendors (like, say, Dell) manage to implement one of the GPE events wrongly. Everything looks like it should work, but the method never sends a notification because it's buggy. In that case we can unregister the existing method and implement our own instead. This code isn't upstream yet, but patches have been posted to the linux-acpi mailing list and with luck it'll be there in the 2.6.33 timeframe. My tests suggest about 0.2W saving per machine, which isn't going to save all that many polar bears but seems worth it anyway. [1] _L1D = lid. Sigh. [2] There's a few that are reserved for specific purposes [3] So where before it had to be high to be active, it now has to be low to be active - this means that it'll now trigger on the switch being opened rather than closed, so you'll get another event when you open the lid again. [4] You can find a list in the documentation for the appropriate ICH chip - the relevant section is "GPE0_STS" under the LPC interface chapter.

I have been reading comments on Sandro's post about the use of email addresses. I am not sure about what he really wanted to express exactly, because I can't see anything wrong when someone from other distro upload a package into the archive and they use their distro's mail addresses so it is clear that I don't agree with the first point of his post. Now, regarding the second point, I don't understand why when being a DD and an Ubuntu Developer at the same time some people prefer uploading to Debian archive using @ubuntu.com but well, anyone is free of doing what they think is correct and this is not something which is in the policy so it's OK. But reading the first point made by Stefano's post made me understand why.

I got surprised when I read the Jon's post about debian kernel contributors and I looked for the word debian in http://lwn.net/Articles/247582/ and nothing was found. I subscribed for a while, just for curiosity, in the Linux Kernel Mailing List and I was able to read some mails from people I know they are in Debian and just because the patches weren't signed off with debian mail addresses Debian does not appear in such a post at lwn.net. I'd like to add that, for not being misunderstood, I do respect all kernel contributors, because their contributions are really important for the whole community and we are aware of Debian's contributions. My point is that it should be accepted that there exists an impact when using one or another email address and it should be considered (when you are part of both projects, in this case, Debian and Ubuntu).

In Ubuntu Server Edition 9.10: No hardware required, Matt Zimmerman announces that from now on, "every release of Ubuntu Server Edition is simultaneously available on EC2". Amazon's EC2 documentation says: "the ability to create and publish kernels and ramdisks is restricted to Amazon EC2 and selected vendors". Canonical appears to be one of those vendors. Their newest EC2 AMI's contain modules from an Ubuntu build of linux-xen (2.6.27-23). Eric Hammond, who has been for a while making both Ubuntu and Debian images for EC2 says

Historically on EC2, neither Debian nor Ubuntu have had completely Debian and Ubuntu stacks from the kernel up. Instead, we've had to make do with patching the runtime on top of older kernels (2.6.16, 2.6.18, 2.6.21) built for other distros (Fedora Core). Amazingly, we've managed to hold this together and keep it running well for a couple years on EC2.

He goes on to describe current problems with eg, Debian's new udev version and the available EC2 kernels. Furthermore, anticipating today's news:

When the point comes where I am not publishing the base Ubuntu server AMIs, it will not make sense for me to publish new Debian AMIs either.

Those wanting to run Debian on EC2 will be stuck with duct-taping together Debian with the available kernels. Maybe I'm wrong. Maybe Ubuntu is actually working with Debian's kernel team, and will be letting them use Canonical's access to publish AKI's of official Debian kernels. Or maybe Ubuntu is working with Debian's kernel team to include identical linux-xen kernel in Debian, so the Ubuntu AKI's could be used. But appearances are that this is another case where Ubuntu has managed to take a situation that was less than ideal for all concerned, and turn it into an advantage for Ubuntu, while at the same time being a unfixable loss for Debian. My mouse's perspective is that a) we maybe only have so many losses left in us (altho we did get thru that whole dinosaur thing ok!) and b) King Kong is still oddly dependent on his mammilian heritage to be up there with his head in the cloud, tearing at Godzilla and stamping on us.

Hello, Planet! (Thanks to lamby for adding me to Planet Debian.) If you don't know me... I'm not surprised. I do some occasional pkg-perl work, but I've really been slacking (and changing jobs, and moving halfway across the UK). Last week I fixed one RC bug, in libgnupg-interface-perl. Better than none, I suppose. It turned out to be quite interesting; the tests failed when the package was built on the buildds, but not locally. Other people had done the hard work, and figured out that this was happening when there was no tty; I just had to debug why the particular perl variable was getting unset. I'm somewhere near the end of the NM process; one of these days, I'll be able to upload my own packages, and those of other people. I'm still working out where best to apply myself within Debian - I do want to share some of the pkg-perl upload burden (because gregoa is an absolute hero, and deserves to be able to take holidays). The status of Debian mentoring in general intrigues me; there seem to be so many places where more minds are needed, and perhaps this makes encouraging and training new developers especially important. More on this later.

I was referred to Creating a World Without Poverty during a discussion on advogato some months ago, but it took rather a long time to obtain the book for various reasons and then I forgot about this review until a recent discussion on new Cooperatives-UK Chief Ed Mayo s site. Better late than never, here s my review of it. Let me know what you think of this book. About the Book Author Muhammad Yunus is founder of the Grameen Bank, Bangladesh, one of the microcredit pioneers, but that story has been told in another book Banker to the Poor. The prologue of this book starts with the creation of Grameen Danone a joint venture between the bank and the France-based Groupe Danone then part one describes problems in capitalism and introduces the idea of Social Businesses as a complement to what the book calls Profit-Maximising Businesses (PMBs), or what most people know as usual shareholder companies. Part two then covers the Grameen Group from its start in banking, through an impressive list of short company case studies some Social, some PMBs, some joint-ventures up to the first year or so of Grameen Danone. The final part then talks about creating a marketplace which is open to Social Businesses, harnessing Information Technology to spread these ideas (more on this in a later article on this site) and using those innovations to banish poverty from the world. The epilogue is the Nobel Prize Lecture Poverty Is a Threat to Peace which seemed to read almost as a simplified summary of the book s main ideas. Annoyance I got quite annoyed with this book because I feel there is a lurking absurdity in its main big idea:- On the one hand, the argument seems to be that profit-maximising businesses are problematic because of their single-minded focus on profit-maximisation. This is an argument that I largely agree with, as you may remember from my short review of Joel Bakan: The Corporation. On the other hand, it proposes to remedy this by creating Social Businesses with equally single-minded focus on social problems this seems like a recipe for disaster! If profit-maximisers threaten to wreck the environment, human rights or whatever else isn t in their goals, why won t Social Businesses using similar single-purpose company formation rules just as easily threaten to wreck whatever mentioned isn t in their goals? If this book is correct and entrepreneurs are multi-faceted human beings who will act to promote social aims (which they must if they are to invest in Social Businesses), then the only robust way forward is to promote businesses that do not try to simplify that multi-faceted nature, whether for profit-maximisation or for a subset of social aims. The best form of businesses that I know for trying to represent all interests of their owners are cooperatives. Dismissed after less than a page Cooperatives are examined in part one of the book and dismissed after less than a page (p35) because they are not inherently oriented towards helping the poor or producing any other specific social benefit (which is a misleading claim because concern for community is a basic principle of cooperatives which all should uphold), they can be owned equally by the wealthy as well as the needy (why is this a problem?) and they can sometimes be demutualised into PMBs. Ultimately, if that s the will of the people, that can t be completely prevented, but recent developments have found ways to make that much less attractive (so-called anti-carpetbagging rules). Doing what people want is not usually a problem in other spheres it s democracy. What s really infuriating is that the author proceeds to describe Social Business which are aimed at specific social benefits (rather than all), can only be owned by the needy but can access the resources they need by donations from the wealthy or by forming joint ventures with them (even very unequal ones) and can be desocialised into profit-maximisers (p178). The same problems which were used to reject cooperatives exist just as badly, if not worse! Conclusion: a bit disappointing So, ultimately, I found this book a bit disappointing. There are lots of good ideas, but when presented with a ready-made round wheel for the basic problem of tempering profit maximisation, this book reinvents an oval one. I suppose I shouldn t be surprised. The misleading criticism of cooperatives is sadly typical of much of the current social enterprise movement, which I mentioned as a drawback in the recent case study of our co-op by Cooperatives-UK. I don t know if social enterprise enthusiasts are informed by the thinking of people like Muhammad Yunus, or if the high-profile authors simply reflect the movement. Have you read this book? Did you spot this contradiction? Do you feel it has other problems that I m ignoring? Or is it a work of genius and I shouldn t be disappointed with it?

I know I wrote about Android already today, but there is another thing that concerns me right now. I am owner of an Android-based phone (an HTC Dream) and recently switched my mobile network provider. The problem is that my new provider is a virtual provider and as such there is no real network of that provider. Now Android has a feature to turn off broadband connections when in roaming mode, which itself is a great idea and can save you from paying quite a lot of money when the phone connects to 3G abroad, but this feature also turns off broadband connections when roaming locally. All this is being discussed in bug report #3499. After noticing this problem I became curious on how Android detects that it is roaming and I found the GsmServiceStateTracker.isRoamingBetweenOperators method to be responsible for that magic, but soon noticed that the method is not only inefficient, but also doesn t work as intended. This is hardly related to the bug mentioned above, but let s have a look at the code in question:

/**
* Set roaming state when gsmRoaming is true and, if operator mcc is the
* same as sim mcc, ons is different from spn
* @param gsmRoaming TS 27.007 7.2 CREG registered roaming
* @param s ServiceState hold current ons
* @return true for roaming state set
*/
    private
    boolean isRoamingBetweenOperators(boolean gsmRoaming, ServiceState s)  
        String spn = SystemProperties.get(PROPERTY_ICC_OPERATOR_ALPHA, "empty");
        String onsl = s.getOperatorAlphaLong();
        String onss = s.getOperatorAlphaShort();
        boolean equalsOnsl = onsl != null && spn.equals(onsl);
        boolean equalsOnss = onss != null && spn.equals(onss);
        String simNumeric = SystemProperties.get(PROPERTY_ICC_OPERATOR_NUMERIC, "");
        String operatorNumeric = s.getOperatorNumeric();
        boolean equalsMcc = true;
        try  
            equalsMcc = simNumeric.substring(0, 3).
                    equals(operatorNumeric.substring(0, 3));
          catch (Exception e) 
         
        return gsmRoaming && !(equalsMcc && (equalsOnsl   equalsOnss));
     

Okay, let me summarize what this piece of code does wrong, at least from my understanding:

• It takes both the network operator alphanumeric identifier and alphanumeric long identifier and compares both to the alphanumeric identifier coming from the SIM card, whilst
• it could simply use the network and SIM card numeric identifiers and compare those, which should be a lot cheaper than comparing those strings
• Then it takes the first three characters/digits of the numeric identifiers (which indicate the country) and compares those

Now in my case my SIM card doesn t seem to provide the phone with a alphanumeric identifier, so the first two comparisons always fail for obvious reasons and, looking at the inline-if in the last line of that method my phone will always indicate that I am in roaming mode, even when I am not. The problem is not only the logic which seems to be wrong, but I rather see the inefficient comparisons used there to be a major problem in embedded systems like mobile phones. This is the first piece of Android code I have had a look at, but if all other code is as ugly and inefficient as these few lines Android really needs some major fixes. Related to this I have reported bug #4590 and forked the git repository in question over at github, to fix this method, should be a matter of 5 minutes.

I ve recently been diagnosed with Asperger Syndrome (AS) [1]. Among other things this means that I am genetically predisposed to have an interest in solving technical problems and give lectures about how I solved them, but that I tend not to be a people-person . AS is generally regarded as an Autism Spectrum Disorder (ASD), but there is a lot of debate among the experts about the exact relationship. Some people (such as the psychologist who assessed me) believe that AS is a synonym for High Functioning Autism (HFA), however one theory I ve heard is that HFA people are more sensory oriented and Aspies differ by being information oriented in that case I would not be regarded as HFA. I m not bothered by this issue, I m sure that in a few years time the experts will have some consistent definitions for such things that most people can agree on. There is no Boolean assessment for AS, the assessment is based on a sliding scale of number of criteria. People who are almost Aspies but don t quite pass the test (or more commonly don t get assessed because they don t think they would pass) are sometimes referred to as Asperger Cousins (AC), this is a slang term that is not formally recognised but is often used in online discussions. I m sure that a significant portion of the readers of my blog would regard themselves as being at least ACs if they investigated the issue. The test at Glenn Rowe s web site [2] can give you an idea of how you rate by some criteria but note that this is not at all conclusive and it s based on the theories of Professor Simon Baron-Cohen [3], not all of which have general agreement. Leif Ekblad is running a project to analyse long-term changes to the Aspie score of adults [4]. The main quiz for that project seems quite popular for self-diagnosis, but again it s not conclusive. I think that diagnosing oneself for an ASD is not nearly as crazy as most things which might fall in the category of being one s own psychologist, but I still strongly recommend getting a formal assessment if you believe that you are an Aspie. In Australia it costs about $600 and there s a waiting list of about 3 months. Chaotic Idealism has an insightful post about the pros and cons of self-diagnosis [5], if you suspect that you may be an Aspie then I recommend that you read it before doing the tests.

• [1] http://en.wikipedia.org/wiki/Aspie
• [2] http://www.glennrowe.net/BaronCohen/MaleFemale.asp
• [3] http://en.wikipedia.org/wiki/Simon_Baron-Cohen
• [4] http://www.rdos.net/eng/Aspie-quiz.php
• [5] http://chaoticidealism.livejournal.com/70133.html

Earlier this year, my brother and I developed a documentation management server for a small software developing company. They were sick of creating their documentation by sending around Word documents, having to manually merge them, losing changes and not getting a clean, consistent layout.So we created zpub for them: It is based on DocBook and Subversion, and adds a fairly nice web interface to it. Now their work flow is

1. The editor checks out the DocBook source document via Subversion. With a client like TortoiseSVN, this is possible even for the less tech-savvy editor.
2. He works on the document using the editor of his choice. We recommended an editor with a proper DocBook mode such as the XMLMind XML Editor or Serna Free, which was recently published as Free Software, to our customer.
3. When satisfied, he commits his changes via Subversion, adding a comment describing his modifications.
4. On the server, zpub renders the document in the various output formats (.html, .pdf, .chm), and makes the result available via the web interface. The commit messages are put there, and all previous revisions of the document can still be accessed.
5. Optionally, an e-mail about the change is sent out to a per-document configurable list of recipients.
6. Optionally, the documents are rendered with a DRAFT -Watermark on the pages, to avoid leaking wrong revisions to the outside. Only users with extended rights are allowed to release a document, thus causing a version without that watermark to be rendered.

For more details on the feature and usage of zpub, check out the documentation that you can find on the demo instance.We are actually quite satisfied by zpub, and it would be a petty if that was just it. Of course, there are quite a view programs out that that can provide these features plus many more, much more than a company the size of our customer would want to have (or even to worry about). So there is a niche between sending Word documents by mail and buying a very expensive, complicated product where zpub fits in.Being a fan of Free Software myself, and since zpub is based on Free Software, we decided that we want to release zpub itself under a Free License. We chose the EUPL, which is a GPL compatible license created by the European Commission, since our customer comes from a municipal environment. The code is hosted on gitorious now, so if you have a need for zpub, just give it a try.Also, if you run a Free Software project, manage your documentation in DocBook (or want to start to do so) and think that zpub might be a neat idea to allow more documentation writers to contribute, talk to me. I might well offer free hosting in that case. If you are a commercial user, I m still offering hosting (and support or feature development), just not for free any more. Note that the zpub user interface and documentation is currently only available in German.

Things which should exist, but don't yet:

Transparent SQL Cache

Imagine a proxy listening on 127.0.0.1:3306, receiving SQL Queries. Any query that was "SELECT .." could return the result from a local cache for the appropriate table. Any query of the form "UPDATE" or "INSERT" would flush all caches for the table. Should be near-trivial: Hash the incoming query & parameters via SHA1sum to get a unique key then store/lookup results in Memcached. Would it be useful? I think so, but of course it depends on the application and the effort involved.

"Global status"

A single site that would rebroadcast a posted (short) status message to facebook/twitter/your chat client/etc/etc. Hard part would be receiving comments from the sites it re-served to. Scraping statuses from facebook is hard, not sure about twitter.

This concludes my Friday wishlist. ObFilm: Master and Commander

graphing DPKG source v3 adoption Given that now we can upload DPKG source v3 packages to the archive, prodded by Lucas and Martin, I've added yet another page to my Debian graph/stats collection: namely stats about DPKG source v3 usage in the archive. This time, I've done even less work than the other pages: Lucas was already providing the actual data via a UDD CGI. So thank him, I'm basically just aggregating/mirroring the graphs for bandwidth reasons. Of course, I've also added the new page to the stats index on wiki.d.o.

When I visited Taiwan last week to talk about Debian at a conference on smartbooks, I used the opportunity to meet up with the folks from QNAP. It was really nice to meet many of my contacts at QNAP in person. We talked about their roadmap and existing products and I found out that they had just released a number of new devices that may be of interest to Debian users. I really like the hardware from QNAP but one downside of their high quality is also that the devices are fairly expensive. Last week they introduced a number of lower cost alternatives: in addition to the TS-119 and TS-219, you now have the TS-110 and TS-210. They feature a 800 MHz CPU (instead of 1.2 GHz on the TS-119/TS-219), 256 MB (instead of 512 MB) and have a plastic case (as a result of which, the TS-110 now has a fan unlike the TS-119). Similarly, in addition to the TS-419 and TS-419U, you now have a TS-410 and TS-410U. Since the TS-110/TS-210 and TS-119/TS-219 are compatible, the Debian installer will work out of the box.

It is only a few days since my last post about the referendum which is due to happen on the same day as the Presidential elections, but I am forced to make another post because there are lots of incorrect views on the subject.

So let's try take the issues one by one and see what is the problem with every one of them, using reason and the information we have available from more mature democracies than Romania's.

What's the referendum about

The referendum, in this case, is about two issues:

• reducing the number of representatives (deputies and senators)
• transforming the Parliament from a two-chamber Parliament into a one chamber Parliament
  

Information on the subject

For an election such as the referendum (basically asking the population on a subject of national importance) every citizen has the right to be correctly informed about the options of the referendum, what each implies, what are the pros and cons for every option, what problems does the referendum tries to resolve.

There is no information on the subject, no videos, no spots, no fliers, nothing, just biased posters (I'll explain later).

Debate on the subject

In order to have a correct information, a clear public debate which should happen, without the pressure of a presidential campaign at the same time as the debate.


Well, we have no debate, so I guess there is no problem of overlapping (sad, but true), but on the other hand, there is no information on the subject.

Analysis of the options

Now let's analyse the two proposed measures in opposition to the current situation.

1. reducing the number of representatives
• pros
• reducing costs
• minorities have a bigger influence (they can defend their rights - see comments for explanations)
  
• cons
• easier to corrupt (obtaining a corrupt 5% in a 300 seats parliament is easier than in a 500 seats parliament)
  
• minorities have a bigger influence (they are disproportionally powerful, in regard to the population they represent - see comments)
• absence of a representative with a big vote power could allow passing an abusive law (to pass a regular law there is a need of a majority relative to the number of present representatives, not the total number), but, in a numerous parliament, the absence of a person isn't that risky (other colleagues would be present to stop the abusive initiative)
2. one chamber instead of two
   
• pros
  
• faster processing of decisions
• less bureaucratic
  
• cons
  
• in the current system there can be a passive vote (law passes silently if not discussed) which has just one occasion to be caught, while in the current system silent passing is allowed only in one of the chambers, but needs to be discussed in the other
• there is a chance that one chamber might amend/catch abusive initiatives with "two chambers", but with "one chamber" there is one chance only
• more prone to power abuse
  

So, there are pros and cons for all solutions, but now we can make an informed decision and know the risks for each option.

This shouldn't have been taken in haste without any public debate.


I don't want to influence anyone's vote, so I'll abstain from telling my options.

The president has the right to associate his image with one of the options.

OK, let's analyse that.

The law doesn't explicitly forbids this, but there are some recommendations from Venice on fair elections (sorry, I couldn't find the link) which say that in a vote such as the presidential elections there shouldn't be an unfair advantage for one of the candidates. Why is that relevant? Because the current president actually uses the referendum as his campaign and he's speculating people's innate hatred for political figures.

His election material is only comprised of virulent attacks at the representatives or calls to the population to vote in favour of both questions.


Why is that unfair? Because the current president is using his current position (president) as launch ramp for his campaign as a candidate.

OTOH, if the seal of the presidency was used instead his face, the problem wouldn't have been that bad.


Organisational issues

There were various rumours about how the voting will take place. Here is a list of things I heard:

1. there will be a single list for both presidential elections and referendum and there will be a single column for signature to confirm your presence at vote
2. there will be one paper, but two columns for signatures
3. no matter what the list looks like, you'll be given both sets of ballots, but you can annul the one you don't want to vote on

All the variants above are incorrect from a democratic pov, but to different degrees:

1. a single signature covering for either of the vaiants "presidential vote only", "refrendum only", "both" asks for trouble because unused ballots in "only" variants might be added later to the valid votes as if they were expressed by voters; since there is no distinct count for each of the ballots to confront that with signatures, before the counting period one could use unused ballots and vote for you on the issue you didn't want to express an opinion on
   
2. informing on what to do in some cases such as "only" vote is essential to prevent fraud; e.g.: strike through the cell corresponding to the the election you don't want to express a vote on, instead of signing
   
3. giving the ballot unconditionally to everybody, leaving you to decide to annul it or not is abusive, because vote is not compulsory in Romania; more than that, this artificial presence burst (even with void votes) could end up validating the referendum since a 50%+1 presence is needed to pass Constitution modification, as we have here

So, recap:

1. favours fraud by allowing inside people to obtain untraceable/unaccounted ballots
2. preventing fraud isn't of any concern, it seems (and could be achieved)
   
3. forced voting might lead to validation of the referendum because the voting presence is sufficient, even if there are many annuled ballots
   

Vote validation and artificial voter presence

Since validation of a referendum on the changes on the constitution depends on sufficient voter presence (there is some mandatory threshold, but I can't find references now), it is in the interest of people wanting to promote the changes to mobilize people to vote.

By different measures the vote presence at the referendum is artificially increased raising the risk of validation, when people, in fact, wouldn't want to validate it and would, to express neutrality, will be be forced to annul the ballot.

Also, a high presence in the referendum might increase the number of voters for presidency, and the people mobilized this way may have a high chance to be biased in favour of the current president because he associates himself with the violent message against the representatives.

What does it take to implement, if referendum passes

If the referendum passes, the changes aren't immediate. The proposed changes are discussed in the Parliament and a final proposal for the constitution changes is made; after approval of the proposals by the Parliament, the ball goes back to population for final approval.

This means it will take a while to see a changed parliament.

Conclusions

Romania is not a true democracy yet and there are areas where individuals should know what their rights are and not let authorities fool them into buying any argument uncritically.

Romanians must learn their rights so they can't be manipulated whenever somebody wants to trick them into some scheme by knowingly omitting issues and creating insane situations.


Romanians, wake up, learn you have rights, form your own oppinion, as distinct individuals, then we might see a change.

A periodic security review at our co-op suggested switching PasswordAuthentication no on even more hosts. One of those caused a bit of a heated discussion about the benefits of increased security and the drawbacks of making emergency access harder, reminding me of the old joke about a secure computer being one encased in a block of concrete, not connected to anything and buried in a secret location. More usefully, I found a Discussion summary on improving SSH which seemed to cover the basics pretty well. It also suggests that nearly two thirds of all SSH private keys were stored on disk with no password protection which is scary and can server admins even detect that sort of risky behaviour? One thing I ve not really noticed before is how common brute-force ssh attacks are. There are some denyhosts statistics which make interesting pictures and Strong passwords no panacea as SSH brute-force attacks rise summarises some data from last year. So what can we do? libpam-cracklib seems like a first step, along with denyhosts. While searching around, I noticed sslh ssl/ssh multiplexer which looks like a useful trick that I might try somewhere. It entered debian testing earlier this week and looks simple to backport to the stable version. What tips would you give? Do you think it s worth having at least one reasonably secure host with PasswordAuthentication enabled, just in case of disasters disabling private keys? Are you a fan of port-knocking and other more sophisticated things? Do you know more numbers about ssh security?

Recently Eric asked on the LUG Vorarlberg mailing list about netbook experience. I wrote a lengthy reply summarizing my experiences with the ASUS EeePC 701. And I thought this is something I probably should share with more people than only one LUG: I ordered an ASUS EeePC 701 (4G) with US keyboard layout at digitec in Spring 2008, got it approximately one month later and posted a first resum after one month in my blog. I m still very happy with the EeePC 701, despite two commonly mentioned drawbacks (the small screen resolution and the small SSD which I both don t see as real problems) and some other minor issues. What matters

• Very robust and compact case. And thanks to a small fan being the only moving part inside, the EeePC 701 is also very robust against mobile use.
• Very pleasing always-in-my-daypack size (despite the 7" screen it s the typical 9" netbook size) and easily held with one hand.
• Black. No glossy display. Neither clear varnish nor piano laquer. Short: No bath room tile. Textured surface, small scratches don t stick out and don t matter.
• Debian (previously Lenny, now Sid) runs fine on it, even the webcam works out-of-the-box.
• Despite all those neat features, it was fscking cheap at that time. And it was available without Windows.

Nice to have

• There s power on the USB sockets even if the EeePC is turned off but the power supply is plugged in.
• The speakers are impressingly good and loud for their size. (But my demands with regards to audio are probably not too high, so audiophiles shouldn t run to ebay because of this. ;-)
• It has three external USB sockets.

What doesn t matter

• The small 7" 800 480 screen: I like small fonts and do most things inside a terminal anyway. And even with 800 480, those terminals are still much bigger than 80 25 characters. Only some applications and webpages have no heart for small screens.
• The small disk size: Quite a lot of programs fit on 4 GB of disk space. Additionally I use tmpfs a lot. And music and video files are either on a external 500 GB Western Digital 2.5" My Passport disk (which I need quite seldomly) or much more come via sshfs and IPv6 from my home server anyway. :-)
• The small keyboard: I just don t have any problems with the size or layout (right shift right of the cursor up key, etc.) of the keyboard. Well, maybe except that any standard sized keyboard feels extremely large after having used the EeePC exclusively for some weeks. ;-)
• The to 630 MHz underclocked 900 MHz Intel Celeron: It s enough for most of the things I do with the EeePC. Also the original 512 MB RAM are somehow ok, but for using tmpfs, but no swap space at all, 1 GB or 2 GB are surely the better choice.
• A battery runtime of 2.5h to 3h is fine for me.

What s not so nice

• The n key needs to be pressed slighty stronger than other keys, otherwise no n appears. So if one of my texts in average misses more n than other letters, I typed it on the EeePC. ;-)
• Home, End, Page-Up, and Page-Down need the Fn key. This means that these keys can only be used with two hands (or oe very big hand and I have quite small hands). This is usually no problem and you get used to it. It s just annoying if you hold the EeePC with one hand and try to type with the other.
• What looks like a single mouse button is a seesaw and therefore two mouse buttons below one button. This makes it quite hard to press both at the same time, e.g. for emulating a middle mouse button press. It usually works in about half of all cases I tried it. My solution was to bind some key combination to emulate a middle mouse button in my window manager, ratpoison:

  bind y ratclick 2

  And that mouse button bar already fell off two times.
• The battery reports only in 10% steps, and reporting in percentage instead of mAh is an ACPI standard violation because reporting in percentage is only allowed for non-rechargable batteries. It also doesn t report any charging and discharging rates. But in the meanwhile nearly all battery meter can cope with these hardware bugs. This was quite a problem in the early days.
• Now, after approximately 1.5 years, the battery slowly fritzes: When charging there are often only seconds between 10% and 40%. Rigorously using up all power of the battery helped a little bit. Looks like some kind of memory effect althought the battery is labeled Li-Ion and not Ni-MH and Li-Ion batteries are said to have no memory effect.
• The SD card reader only works fine if you once completed the setup of the original firmware or set the corresponding BIOS switch appropriately. No idea why.

Similar models Technically, most of this also counts for the EeePC 900SD (not 901) which only differs in screen, resolution and disk size as well as CPU, but not on the the case. So same size, same robustness, same battery, same mainboard, bigger screen, resolution, disk and faster CPU. (The 901 has a different CPU, a different battery, and a different, glossy and partially chromed case.) See Wikipedia for the technical specifications of all EeePC models. ASUS only big FAILure Stopping to sell most EeePCs with Linux and cowardly teaming up with Microsoft after having shown big courage to come out with a Linux only netbook. Well, you probably already know, but it s better without Windows So basically you no more get these really neat netbooks from ASUS anymore and you get nearly no netbooks with Linux from ASUS in the stores anymore. It s a shame. Would I buy it again? Sure. Well, maybe I would also buy the 900SD or 702 (8G) instead of the 701, but basically they re very similar. See Wikipedia for the differences between these EeePC models. And of course I still prefer the versions without Windows. But despite the low price, the EeePC 701 is surprisingly robust and still works as on the first day (ok, except battery, the mouse button bar and the n key ;-), so I recently bought a second power supply (only white ones were available *grrrr*) and ordered a bigger third party battery plus an adapter to load the battery directly from the (second) power supply without EeePC inbetween. What desktop do I use on the EeePC? None. I use ratpoison as window manager, uxterm, urxvt, and yeahconsole as terminal emulators (running zsh with grml based .zshrc even as root s login shell :-), wicd-curses as network manager and xmobar (previously dzen2) with i3status as text-only panel. Installed editors are GNU Emacs 23, GNU Zile and nvi. (No vim. :-) And of course a netbook wouldn t be a netbook if it wouldn t have a lot of network applications installed. For me the most important ones are: ssh, scp, autossh, sshfs, miredo, conkeror, git, hg, and rsync.

Heyho. The last morning and we are all very tired. Yesterday we had a nice 20hour workday that only ended this morning at 03:30h. But it was entirely worth it, as we managed to get the last big patchset working. Something where the idea is years old, but noone ever came around to do it. Granted, it is a really big change, so not easy to do and it took us time and lots of energy, but it is really worth it. Especially as it will instantly kick out a lot of race conditions we could never properly fix with the old code. And will help also against the case where an orig tarball could get lost, this is no longer possible. Now I am way too tired writing up more in this post, just two things:

• We had 82 hours simply working on stuff this week. Combined with the number of people and their arrival times we can sum the total time spent working on the archive up to more than 400 man hours.
• 140 files changed, 11922 insertions(+), 9715 deletions(-) (We have nearly 30k lines of code).

So, lots of changes, stay tuned, we will post multiple times to d-d-a when we are back home and feel human again. :)

Regular readers of this blog (yes, both of you!) may remember the computer/broadband/ directory that this post appears in as the collection of my Comcastic (yeah right) experiences with my ISP. But I think this week may top everything. I'll just try to jot down some notes before I forget all the gory details:

• On Tuesday, I edited one of the internal nvram configuration variables of my trusted wrt54g router in order to add the older daughter's shiny new iPod Touch to the set of 'permitted' MAC addresses. This router, running a custom Linux variant called OpenWRT had essentially not been upgraded since I first installed it, and still required a quick reboots after updating of configuration values. However, that worked fairly flawelessly for 4 1/2 years. Until Tuesday.
• Upon reboot, I got what appeared to be an invalid network setup from the cable modem. IP and Gateway assigned , but no DNS and no ability to ping anywhere. Crap. So I fiddled with this all evening, including a service call to Comcast but to no avail. When a laptop was directly plugged into the cable modem, it got correct settings albeit on a completely different subnet. So for the next day, we left one machine directly plugged so that my wife could at least telecommute.
• Somehow it became apparent that waiting insanely long for the router to remain powered-down -- and we're talking five minutes or longer -- helped. So by now we were suspecting the cable modem. I use a standard Motorola SB5101 I once had to buy in a rush because of the Comcastic ones who all of a sudden changed their minimum requirements which meant they would no longer connect to my existing modem. Anyway. So on Wednesday I called Motorola and had a decent service call with them but as I was at work I couldn't follow up with part numbers etc pp. At least I learned that I seem to have two months of the two-year warranty left...
• So by Wednesday evening I decided to fall back to the really cheap and old Speedstream router I had used before the Linksys wrt54g. That worked, albeit sloooooowly. Wired and wireless ethernet, direct assignment from the cable modem. All well. But did I mention it was sloooow though?
• Thursday evening was skipped as I was at the Chicago R meeting we organize to complement our R / Finance conferences in the spring.
• Given that the cable modem worked with the old Speedstream and with directly-connected machines, I decided to finally go for a long overdue update of the wrt54g software. So that happened on Friday, i.e. yesterday. And similar to my previous wrt54g notes, I needed to flash the new software with the tftp protocol and a helper script on a laptop connected to the router. All this took a while as I needed to remember to also send to a ping flood to the router to be able to catch the tftp request, needed to test which of the atftp and tftp binaries worked reliably, and whether the router prefers .bin images over .trx images when using the tftp protocol. But lo and behold this worked, and I configured a shiny new Kamikaze aka 8.09.1 version of OpenWRT. This even connected to the cable modem once I helped with DNS entries. OpenWRT generally rocks, and this new release is a lot nice than the more bare-bones version I used to run.
• Unfortunately, I had picked the bcrm47xx variant -- the 2.6.* kernel version of the OpenWRT Project's software for my WRT device. And guess what, that one does not include wireless support due to issues with Broadcom drivers and the kernel. Grrr. So once I had that confirmed this morning, I quickly switched to the bcrm-2.4 variant of the same 8.09.1 release. At least now I can flash from within using the mtd command from the commandline.
• But once up and running with the bcrm-2.4 release, I ran into the same issue we have had with the Motorola cable modem and Comcast behind them. Each time I connect with the wrt54g, I end up on a specific subnet, without DNS and with no ability to connect. The Speedstream still worked. So what to do? Well, MAC Cloning to the rescue. Now the Linksys wrt54g pretends to be the Speedstream, and all, at last, is well again.
• So after four days of intermittent service, which means that my few web pages, blog, and goodies like CRANberries were invisible, I now have better router software. That could have come a little easier, and I still don't quite know why Comcast decides to no longer service the wrt54g under the MAC address it presented itself with for 4 1/2 years. I have paid thousands of dollars over that time to get broadband access. But this, I don't quite call service. To top it all off, guess who cold-called to sell VOIP service while I wrote this up? Oh, it's Comcastic ...

a tale of three tools: mairix, maildir-utils (mu), nmzmail I think/fear I'm getting into this Getting Things Done thingie. For weird reasons I'll explain later on, part of the GTD work flow I'm implementing requires quick lookup from Message-IDs to the corresponding mail, no matter in which mailbox (actually Maildir) I've stored it. Hence, I've looked for a mail indexing tool which is Mutt-compatible, handles Maildirs, and supports Message-ID queries. In Debian (where else should I look? g), I found three: mairix, nmzmail, maildir-utils (whose upstream name is actually "mu"). The first one I tried is mairix. Last upload in Debian was 2 years ago, it doesn't seem to be particularly buggy, and in popcon it has about 300 installations. The integration with mutt is good: searches can create a sort of virtual Maildir, whose files are symlink to the search results; with a couple of macros you can have Mutt easily open the result directory after query. The reason why I ditched mairix, is that it heavily suffer from the NIH syndrome. mairix is a self-contained executable with no external dependencies; that in principle being good, I found nowadays totally unreasonable to not use some third party full text search indexer, given that in the FOSS world we have several good ones. A good aspect of mairix, which is missing in the competitors, is the ability to index messages incrementally as they flow in, e.g. via procmail. Still, the problem of that is that it is difficult to pair that with the habit of moving messages across mailboxes. To that end, periodic re-indexing, or better batch index updates, offers a better work flow. Then I tried nmzmail, which claims at is killer feature the "better integration with Mutt" wrt competitors. Actually, this is false, it has the same level of integration of the other (virtual maildirs with macros) and it doesn't even offer a ready to use set of macros in the doc! (yeah, they're easy to write, but given that you claim you're so well integrated with Mutt ...). The reason why I ditched nmzmail is that I didn't particularly like its choice of (external at least) indexer: Namazu. The index it created was very big (something like 250 Mb for about 400 Mb of Maildirs). Also, I had a bad feeling that the indexing was somehow Japanese-specific (the project having support for that language) and I found no way to disable support for that, which I obviously do not need. Finally, I tried maildir-utils and I was finally happy. It is implemented on top of Sqlite3 (for mail metadata) and Xapian (for full-text indexing). There are some bugs, but Norbert, as the very reactive maintainer, have tackled down most of them now and I've been happy to help with various feedback. Integration with Mutt is granted by the following 2 macros:

    macro index <F8> "<shell-escape>rm -rf ~/.mu/results; mu-find -o l -l ~/.mu/results " "mu-find"
    macro index <F9> "<change-folder-readonly>~/.mu/results\n" "display mu-find results"

The first one query (you) for the search string, the second jumps to the results opening the virtual Maildir (which is useful, especially when you want to go back to the last query you did). I update the index every two hour with the following cron entry:

    31  */2 *  *   *     on_ac_power && mu-index -q

Beside a corner case bug which is close to solution, updating the index is very fast, usually a few seconds; index size is about 150 Mb. To conclude, my initial goal (message path lookup via Message-ID) is easily achieved as follows:

    $ mu-find -f P m:20091030112543.GA4230@usha.takhisis.invalid
    /home/zack/Maildir/INBOX/cur/1256902638_0.25702.usha,U=37563,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,S

In addition to that, I've now gained all-maildirs full-text search from within Mutt As a concluding remark, Enrico pointed me to Not much mail, but it seems to be (by authors' own admission) in early stage of development. Also, AFAIU it aims to be a MUA, whereas I'm perfectly fine with Mutt, I just need from time to time to integrate it with other components of my daily work flow.