To whom it may concern... Daniel and Rob are holding a party in June to celebrate their Civil Partnership. Yay! And I'm invited. Double-yay! Unfortunately, it's the same weekend that Debconf 7 finishes. And it's in in deepest, darkest Wales, almost at the opposite end of the UK from Edinburgh. Less yay...! As Daniel is a DD, I'm guessing (and he has confirmed) that quite a few of the party guests will be at Debconf. That means there will be quite a number of people will be needing to get from one end of the country to the other on Saturday 23rd June. As I'm a mug for this kind of thing, I'm looking into organising travel for a group. If you're looking at making this trip, mail me if you're interested in travelling together. Options at the moment include hiring a minibus and driving the whole distance, or maybe flying from Edinburgh to somewhere much closer (Bristol or Cardiff) and sharing a vehicle from there. Let me know...

As I'm nearing the completion of my PhD, I'm starting to write about the findings from my research. The aim of my PhD is to investigate quality improvement in free software projects. After some exploratory studies about quality in free software, I decided to focus on release management as one problematic area in which improvement is possible. My approach is to study the development process of projects and to investigate ways how these processes can be improved. The hope is that such improvements will lead to increased levels of quality in the software produced by free software projects. With regard to release management, I'm focusing especially on large and complex projects, in which hundreds of volunteers need to be coordinated during release preparations. I've been looking particularly at time based release management since my exploratory studies have shown that there is significant interest in this release management strategy. The main question of my research is how and why time based release management works for large free software projects. In my first series of postings, I will introduce the seven free software projects I have studied in depth during my research. For each project, I will briefly summarize what problems the project faced in the past, what changes they have implemented to address these problems, and what outstanding problems they still have.

Been trying out upstart today. It's a nice piece of code, but unfortunately it's totally useless today. Not because its concepts suck; rather, because Debian packages today ship with SysV-style initscripts, and those do not integrate well with upstart: most of the features that make upstart so interesting require you to create an event.d script, which has a totally different syntax. The Debian Developer inside me couldn't help but start to think about a way to fix this, but that seems harder than what it initially would sound like to the uninitiated. My first idea was to add some /sbin executable to upstart or one of its supporting packages, so that a SysV-style initscript could exit if that binary told it that upstart was running, before doing anything. That way, a package could ship both upstart event.d files and SysV-style initscripts during a migration period, and we could get rid of the SysV-style initscripts when and if Debian eventually decides to migrate to upstar. But when I sent that past #debian-devel, someone mentioned initng, which is yet a different init implementation. If this would work for upstart, it could also work for initng, and I don't think many packages would want to ship initscripts for three different init implementations. So scratch that. My next idea was to generate initscripts, rather than have packages ship them. The amount of common code in our initscripts is huge, anyway, and it's not as if it's all that hard to write an initscript. The things an initscript needs to do are limited, anyway: start a daemon when requested, stop it, restart it, perhaps send it a SIGHUP if or when configuration updates have been performed, and provide some useful output on all that. In most cases, that's about it. So, I thought, why not create an RFC822-style file (analogously to other files in the debian/ directory of a debian source package) containing all the metadata on an initscript, and generate the correct style of initscript out of that? That should work, I thought. So I started writing a specification, and had a look at the initscript of one of my own packages, nbd-server. Which does this at some point (nonrelevant content cut out):

i=0
while [ ! -z $ NBD_FILE[$i]  ]
do
  start-stop-daemon --start --quiet --exec /bin/nbd-server (...)
  echo -n " $ NBD_FILE[$i]  ]
  i=$(( $i + 1 ))
done

There's no way you'll be able to express that type of flexibility with an RFC822-style file containing all the metadata. So, scratch that too. I still do think generating initscripts is the right thing to do, however; the question that remains is "how". One idea I had was to use m4 in an autoconf kind of way, since autoconf allows one to mix autoconf macros with "regular" shell snippets, which would be exactly the kind of flexibility that one would need. However, that totally ignores the fact that autoconf is only able to pull this off because its output is shellscript in every case. An m4 macro that has to produce output which are not, in fact, shellscripts (such as would be the case for this project) could not do that. I think I'm kinda stuck now...

Debian used to be the cool kid among Linux distributions. Because our stuff worked much better, was easy to install and especially to upgrade. Dependencies would be automatically resolved while others were fighting dependency chaos, and our menus would have all the apps in it, where others had to fill their menus on their own. And at the same time, Debian was very flexible and could be customized to become e.g. Knoppix. Our bugtracking was great and we were openly discussing bugs and all this stuff. Many of these Debian achievements are now common among distributions. Especially among Debian-derived distributions. Debian however has become a reliable 'stable' distribution often even called 'stale', while others stand in the spotlight of innovation. Other distributions have come up with maybe more advanced bugtrackers (I'm not talking bugzilla, which IMHO is overengineered) and other tools. Technically, this isn't completely true. There are still many fields where Debian is technically leading. But our focus has recently been a lot about individual packages or components (e.g. the installer). Maybe we should try focusing more on infrastructure again. In fact, Debian still has some very good infrastructure components others might be lacking. Our QA tools (including bug counters, tracking which versions are affected by which bugs, all kinds of graphs, wotomae) are great. All our update-* commands are missing in many other distributions, also things like invoke-rc.d. We have a solid Python policy to support multiple Python versions and precompilation (judging from the SELinux policy, none of the other distributions has that yet). We're able to use dash as a minimal shell instead of bash in most places (which does make a speed and memory difference). We have Debtags. But these projects don't play a key role in Debian anymore AFAICT. Trying to start a major infrastructure thing requires a LOT of engagement, maybe too much. I'd love to see better SELinux support in Debian, but I can't do it on my own, it's just too much. And I find it very hard to find people to help me with that effort. If Enrico hadn't been pushing Debtags again and again (and with a lot of code), it would also have gone stale. Despite being IMHO something that can take Debian on top again (making it a lot easier to find appropriate software). One of the reasons why Debian is still living pretty much outside is that it would be a huge effort to get maintainers to add the corresponding Tag information to their packages and so on. I've always balked at even trying that, fortunately Enrico got some tag information added via overrides to the Packages file already. Another thing that should have been happening within Debian (but happened at Ubuntu) is 'upstart'. There have been a couple of packages dealing with new init systems, but none ever managed to get support into other packages (nor is there any package in Debian with upstart support that I'm aware of). Getting SELinux strict support into init ramdisks is another big thing I'm too scared of to even attempt... i.e. I doubt that I'll be able to do that on my own, yet even to get it into the actual packages... DPL candidates: any ideas on how to be come more agile in such points (e.g. adding support to sysvinit+upstart+initng+runit to all packages with init scripts)? It took us already ages to get LSB headers added to many init scripts. Or on any of the other transitions we should do (SELinux support, Debtags, ...), or even why we've lost this flexibility? The one idea I have for that is to try to do some hackfests. Like an init transition day, where a group of people tries to NMU most of the packages that have init scripts. Or adds "Homepage:" to most packages, which some important package metadata we only have on some packages. Or adding 'watch' files. Some of these would be easier if we had a common way of packaging and a central SVN repository. That would make it both easier to prepare the transition (e.g. committing init scripts beforehand) or especially for adding Homepage metadata and watch files without actually interfering with the maintainers work by not doing an upload immedeately, but just adding them to SVN so the maintainer includes them on his next upload. (Sorry, no comments enabled in my blog.) P.S. another thing I'd really love to see is a (semantic) PackageWiki for Debian, that has a Wiki-like structure with a page for each (Source-) package. Including things like e.g. Screenshots, Homepage link, Link to the Freshmeat page of the package, support lists/groups ... Kind of like the QA pages, but for the actual users, and editable by visitors.

München (Munich!) has a great swing scene - alive and kicking. Lindy Hop [wikipedia], Balboa [wikipedia], Boogie Woogie [wikipedia] (~ East Coast Swing) and Rock'n'Roll [wikipedia] (the acrobatic version) all over the place. Since starting dancing (RnR some years ago, Lindy a year ago, all just socially until about half a year where I took on the others as well and started going to some real trainings), they've become an essential part of my life that I wouldn't want to miss. If you happen to come to Munich, here are some links for you:

• Swing and the City, the homepage of Christine von Scheidt, a world class dancer and trainer as well as judge in some contests gives Lindy Hop classes that are excellent.
• The Cord (Sonnenstraße, very central location in Munich) has a great cosmopolitan flair and a swing night every monday, often with live music (which may or may not be great for dancing). Cat and Chris of 'Swing and the City' give teaser classes there, too.
• Salon Erna at Optimolwerke (near Ostbahnhof, also quite central, at the big party areal in Munich) is the home of Swing on Sunday. There is a beginners class as 19:30 with changing trainers and topic (usually Lindy Hop). While a bar like Cord, there are mostly dancers here.
• Swing it! is organizing social dancing events with livebands about once a month. Unfortunately, the previous location closed down (not due to Swing It, which got the place really crowded), but they'll hopefully find a new home soon.
• Boogie-Bären is the dancing club I'm in. They have four Lindy Hop classes on Tuesday, two Balboa classes on thursday and Boogie Woogie classes on most days.
• Near Fürstenfeldbruck (north-west of Munich) there is the Tanzfabrik Überacker where Fabi and Gino will take good care of you; I learned a lot from them in their classes at Salon Erna. They got me addicted to Lindy Hop. :-)
• Keep on Swinging, is the home of Fabi and Chrissi. They are giving occasional workshops at the Tanzfabrik
• If you are at the university for some time, got to the University Sports Programm. It's exceptionally cheap (like 7.50 EUR for a term for a regular student), and that includes lots of sport offers, such as a Rock'n'Roll and Boogie Woogie class. Thats where I started dancing before going to Berkeley, discovering Lindy Hop there and continuing it in Munich later on.
• Claudia and Horst, who also teach at Boogie-Bären, also give classes in Freising, north of Munich.
• And of course World of Swing: Markus and Bärbel are very well known dancers and trainers. If you are into swing, you probably have seen or met them already. They are giving Balboa and Boogie Woogie-Classes and organizing large dance events such as Rock that Swing or the "Munich Balboa Weekend".
• In summer, there will hopefully be a weekly dance event in Hofgarten park again, at the Diana temple.

P.S. My personal favourites / recommendations: Go to Cord or Salon Erna to see how much fun the community is, then take classes with Swing and the City or Keep on Swinging.

Russel Coker explained how to generate a local policy module from the error log. Note that this approach (audit2allow) suffers from the same problems that automatic policy learning suffers from (at least when not done very smartly). The generated policy will exactly cover the behaviour you had during logging; functionality that you didn't use is not covered, but misbehaviour that occured during this time is. Thats basically why SELinux doesn't use this autolearning approach considered a "benefit" of AppArmor by some (as you've just seen, you can do that with SELinux, too). So let me show you an alternate way: First of all, install the refpolicy-*-dev (from my experimental repository on alioth) or selinux-policy-refpolicy-dev (unstable) packages. Next you'll need an audit error to fix, e.g.:

audit(1173577161.426:3436296): avc:  denied    search   for  pid=23862 comm="amavisd-new" name="lib" dev=md2 ino=63745 scontext=system_u:system_r:amavis_t tcontext=system_u:object_r:var_lib_t tclass=dir

So that obviously is amavis trying to access /var/lib (you can verify this by checking that ino= is the inode number of this directory). Looking at /var/lib/amavis reveals that these files are labeled amavis_var_lib_t, so the amavis policy is lacking just this simple tweak. While Russel's approach would work fine, I'll try to show how this would be fixed in the actual policy. The approach I use is documented somewhat in the README.Debian at least in my packages. I'll create a file named amavisfix.te:

policy_module(amavisfix,1.0.0);
require  
        type amavis_t;
 
files_list_var_lib( amavis_t )

The name "files_list_var_lib" was looked up in the Refpolicy API documentation. Granted, it takes some time to get used to their naming scheme, but it's actually quite consistent. Now I run

make -f /usr/share/selinux/refpolicy-strict/include/Makefile

and it creates a compiled module, amavisfix.pp, for me that I can install with "semodule -i amavisfix.pp". The audit error should now be gone, and amavis should function. I can now file a bug report with upstream that the amavis policy is lacking "files_list_var_lib( amavis_t )" and they'll be happy to add that. :-) Note that "files_list_var_lib" has an actual semantic meaning what the process is being granted; audit2allow lines are just technical representations of the access violations seen. Some of the more advanced SELinux IDEs might be able to suggest you appropriate interfaces by looking at the audit errors; I havn't tried them yet.

Well, after reading the rebuttals of my platform for DPL, i noticed that from the others candidates only one really disagree with a proposal I've included - 'backports' related. I'm really happy that they agree with some of the proposals. Unfortunately, it seems that it's common sense between the others candidates that I won't be able to do everything that's in my platform during a one year term. They're right, i won't. I won't do everything that is in my platform simple because Debian isn't a single person project. :-)

My platform is a request to in a way lead some changes and if you disagree with any idea, feel free to email me and do your suggestions. Keep in mind that I can't please every single person using or contributing to the Debian project. I'm sure that i can motivate enough people in the project to help me work on that proposals and I'll be directly involved in a lot of stuff that is there, but I've no problem share power, influence or whatever you call for the better of our operating system.

I've been working on a public Google Calendar with my schedule (if elected), maybe it will help some of the others candidate and Debian Developers understand how and when:

XML:
http://www.google.com/calendar/feeds/qkkr1fd8b2qu4ms0cr9dkq56i0%40group.calendar.google.com/public/basic

ICAL:
http://www.google.com/calendar/ical/qkkr1fd8b2qu4ms0cr9dkq56i0%40group.calendar.google.com/public/basic.ics

HTML:
http://www.google.com/calendar/embed?src=qkkr1fd8b2qu4ms0cr9dkq56i0%40group.calendar.google.com

There's also my Live Platform online. The Live Platform is basically my platform as in vote.do containing some HTML fixes and additional comments (in bold) based on questions or general feedback about my platform content.

See you during the debate!

I was honoured to be on the ETech 07 programme committee. Alas, I can't get there this year due to the very enjoyable reason of recent parenthood. Nevertheless, here are some of the talks I'd earmarked to attend.
Creating Addictive User Experiences (Kathy Sierra)Is it possible to say too many good things about Kathy Sierra? Any project or product built with attention to her advice about empowering users gets a huge head start to popularity and success. I was fortunate enough to spend a day with Kathy last year on a training course, and it was great to see the passion and commitment from her writing runs through her as a person, even behind the scenes.Applied Web Heresies (Avi Bryant)I'd go to this one out of insatiable curiosity. I believe that web conventions enable rapid application design (c.f. REST in Rails) but here's an argument that not following them also reaps rewards. Attend this talk prepared for involvement. I've never met a Squeaker without attitude.
Making Offline Web Applications a Reality (Zimbra)A short product talk, but taking web apps offline is an area to watch. Even if you don't think offline is that important to you, many of the same principles apply to application mobility. Take that key app data off onto your phone, watch or memory stick.Collective Intelligence, Indeterminacy, and the Illusion of Control (Charles Armstrong)More user-facing stuff. How do you give a user of a large nondeterministic piece of social software the idea that they're in control? This is an important topic to me: note how many pieces of social software fall by the wayside right away. For some, it's lack of utility, but that's influenced too by the degree of engagement and control a user feels.
No Program Left Behind: Liberating TV from the Tyranny of the Ephemeral (Tom Loosemore)There is no good reason not to listen to Tom Loosemore. Especially when he feels strongly about something. The idea of this talk is a "vision of a cheap consumer device sitting under your TV that will capture, store, and share all TV, forever." The TV industry sure needs its visionaries.
Patterns: From Fabrics to Fabrication (Dale Dougherty)Dale was responsible for getting my XML career off the ground, and you can find him at work behind some of the smartest things O'Reilly's done over the years. Make magazine was one of those, and now he's moving on to Craft. Design patterns for, er, real designs.RFID Guardian: A Personal Platform for RFID Privacy Management (Melanie Rieback)
Jamming bad RFIDs. Having just taken delivery of an RFID-enabled passport, this is starting to come home to me.Super Ninja Privacy Techniques for Web App Developers (Marc Hedlund)Despite the cute title, lots of good stuff here. Expectnation is going to end up with a truck load of user registrations, and their privacy is important to me. I imagine there'll be some good debate in the Q&A session for this talk.JavaScript: It's Happening All Over Again! (James A. Duncan)I've recently got the JS bug, thanks to JQuery. It's actually a fun and malleable language to work in. I'd go to this talk looking for fellow travellers and hints, though I'm a bit skeptical about Zimki itself.

• ETech 2007 registration
  

Yesterday I received a Fulong Miniature Computer from China. This device is produced by Lemote and is based on the Loongson chip, a MIPS-like CPU developed in China and previously known as the Godson. The Fulong mini-PC has a Loongson 2E CPU running at 666 Mhz, 256 MB RAM and a 2.5" 40 GB hard drive. There are the usual I/O ports, such as USB, PS/2, Ethernet, audio, VGA and TV out. The Fulong mini-PC ships with a customized version of Debian. I'm really excited about the Loongson because it promises to make MIPS available on the desktop. In fact, the folks at Lemote already have a laptop based on the same motherboard used in the Fulong mini-PC. Lemote is also trying to establish a community around the Loongson. They have given 400 devices to partners and 600 to Loongson fans and developers in China. It seems that the community is quite active, but it's hard to tell because I don't read Chinese. Fortunately, there is a blog where regular updates about Loongson are posted. For now, I have taken some pictures of the Fulong, but in the coming months I look forward to integrating Loongson support into Debian.

Today are provincial elections in The Netherlands, one of the least visible governments in this country, so it's hard to decide what would be a good choice. Add to that that the chosen members will in turn choose the (national) Senate. I have never been fond of package deals. I do not necessarily vote for the same party on the provincial level as I do nationally, because these two layers deal with distinctly different classes of decisions. After casting my vote I returned home with a little doubt of whether I'd made the right choice, until I realised which 9 parties I didn't vote for. The fact that my vote did not go to the parties that I don't trust our province or senate with is far more useful than the relatively small difference between the candidates I considered.

One of the many things I'm working on in my copious amounts of spare time is setting up the server that HP donated Linux Australia after I think linux.conf.au 2006. Said server has iLO, which facilitates remote power management and remote console, both things I want to have working on this server before it goes into production. Jon's been madly poking holes in his firewall for me so I could access the iLO stuff. I'm already liking it better than Dell's BMC, because it has things like SSH, although the whole thing hasn't been without some adventures. I fiddled around via the web interface and added an account for myself and an SSH key. Then I tried to SSH in, and promptly got stuff like

buffer_get_ret: trying to get more bytes 4 than in buffer 0
buffer_get_int: buffer error

straight after what looked like a successful connection, and then the SSH connection got terminated. I did a pile of Google searching, and found people had to disable environment variable sending, and agent forwarding to get this to stop happening. Interestingly, I can't find a way to disable SendEnv on a per-host basis with OpenSSH, if it's been set in /etc/ssh/ssh_config, which is kind of annoying. So after solving the problem of gaining SSH access to the iLO, I then battled with the remote console. The host already had a throwaway Linux installation on it, but the remote console kept freaking out about the console being in graphics mode. I figured this was because it was using a frame-buffer. No problem, I just needed to boot with vga=normal. Problem was, GRUB was configured with hiddenmenu so I had 3 seconds to hit Escape before it charged off and booted up. Not that big a deal you'd think, except to get out of the remote console and back to the iLO prompt, you have to hit Escape (, so the iLO eats Escape keys for breakfast. After having a bit of debate with some mainly Red Hat-oriented sysadmins on #sage-au, it turned out that RHEL has a patch so that GRUB accepts any key, not just Escape to present the menu. That was no use to me, as the box wasn't running RHEL. And all I was asking for was if anyone had any iLO experience, no whether or not "any key" was the equivalent of Escape. Sheesh. It turned out that I needed to press Escape twice in that initial 3 seconds, with probably sufficient pause between the two presses for it to register properly, in order for an Escape character to be sent to GRUB. Fun.

Fortunately, I rarely use my Google GMail account. Because for about a month or so, I can't write replies in my preferred browser anymore. I can only guess it's due to some broken browser detection done by Google - my preferred browser is Epiphany, and uses XulRunner. So it's the same engine as my Firefox, and I can write replies with Firefox (but the Firefox UI is not as nice as Epiphanys, and it uses more memory). (Well, almost. Epiphany is Gecko/20070209, whereas Firefox is Gecko/20070208. So either some change in this one day breaks GMail, or Google broke it themselves with some stupid browser detection; many people still think it's sufficient to check for 'Firefox' to detect all non-IE users. Please use the engine ID, that is Gecko.) Anyway: if I'd be a heavy Google Mail user, that would be a desaster for me. Broken for a month now and counting! Fortunately, I don't rely on that friggin' Ajax stuff; I can either use the standard HTML version of GMail or use Firefox. I'd just like to emphasize that Ajax apps break much easier, and your users might be unhappy about that. Ajax is far from perfect, but an ugly hack. Don't overuse it.

There are days like this where you get inspiration. Since about January (strange that one cannot never remember the beginning of such annoyances), neither suspend-to-ram nor suspend-to-disk were working anymore on my Dell latitude X1. Suspend to RAM indeed stopped working back in August 2006 for an unknown reason. The laptop indeed goes into sleep mode but, when waking up, the video card is not restored properly. A quite common problem, unfortunately. Why it suddenly stopped working is hard to explainn but as my laptop runs unstable and, usually, the latest Debian kernel, there may be an explanation somewhere betwwen kernel stuff and Xorg stuff. Since then, I had suspend to disk working pretty well until it suddenly stopped working back in January. The system was printing "Snapshotting system" on the screen and that's all. So, since then, I keep my laptop ON all the time, which is pretty bad when travelling (and I travel a lot all day long as I use public transportation with 3 connections morning and evening). Moreover, I cannot even completely close it because some moron at Dell has decided to hardcode in the BIOS that the laptop should enter suspend to RAM when the LID is closed. So, closing the laptop means....no more screen when opening it up. As a consequence, during last weeks, I had my laptop hald-closed in my bag with some random device inserted between the screen and the keyboard to prevent it to be entirely closed. Absolutely stupid. However, this morning, the Holy Grace touched me. I finally remembered that, for last New Year's Eve week-end, I did setup the Conexant HSF modem drivers on the laptop because I was staying at my mom's place with nothing but a phone line to get my mail. And, definitely, this was the explanation. As soon as I unload the Conexant driver before going to suspend to disk mode, the suspend to disk magically works again. Dammit. Once again a proof that Thou Shalt Never Use Non-Free Drivers. I'm now again a happy suspend to disk user. If someone, some day, can bring me back a working suspend to RAM, I'll hug kiss her him to death.

You might expect that a vaccine against a disease that causes cancer would be widely embraced as soon as it was proven safe. If the disease in question was transmitted by contaminated food or water, sneezing, or most of the other ways that diseases spread then it probably would be widely accepted.

However recently there is a recently released vaccine against Cervical Cancer. The virus in question is only transmitted sexually. Apparently 80% of women in the US will catch it before the age of 50 (so it's obviously not scaring people away from unsafe sex).

There is a strong Christian lobby against the vaccine, their idea is that if sex doesn't cause debilitating and/or fatal conditions such as cervical cancer then their daughters will have less reason to avoid it. The fact is that religious people are statistically more likely to practice unsafe sex (see this link) so it seems unlikely that preventing one of the STDs that religious people might catch will affect the amount of unsafe sex.

One thing that seems strange about the entire discussion is that no-one has raised the possibility of vaccinating boys. Vaccinating boys could lead to the virus being eradicated. Even if an eradication attempt fails it will help save some of the Christian girls.

Hi from Chemnitz! As expected it's great here. As usual. :) My two talks went really well. They were mostly freely improvised since I was kind of slack regarding their preperation, but I got very positive feedback on both and plan to expand them for future events. There's still a lot of visitors walking around. The debian booth is well visited with questions in all ranges, babelbox is running merily on the demo machine in rotation with bb, which never ceases to fascinate people. ;) My saturday was really busy. I walked over here around 8:30, had breakfast and gave my first talk. After a two hour break I gave the other one and ever since then I've been switching from one conversation to the next without pauses. Most of these conversations were with "strangers" who visited my talks and wanted to tell me about projects and ideas of their own, their personal experiences with Linux and Free Software or give further suggestions for my talks. I've really enjoyed these conversations, though they got a bit tiring after a while since for some reason people always catch me when walking somewhere, and so most of these long conversations were held standing and my legs are pudding now. In a few minutes (hopefully) the social will start and I hope it will be as good as last year. I plan to take things a bit easier tomorrow, visit other peoples booths, take some pictures and maybe even visit a talk or two. The rest of this day will hopefully be spend having some more inspiring conversations, enjoying good food and drink and continuing to try to coax Noel into relinquishing to me his old Debian shirt, that he grew out of. (Small Debian shirts are such a rare and precious good!)

Shell scripts are good for a lot of things. It's quick and easy to design shell scripts that take input from one program, pass it to another program, munge it for filenames, etc.

But there are a few drawbacks to shell scripts.

The #1 drawback, in my opinion, is that it is extremely difficult to get quoting and escaping right. I often see things like $@ in shell scripts (breaks if a parameter has a space in it). I also see people failing to check for errors properly (set -e helps that). It's also difficult to do a more modern style of exception handling (do a sequence of actions in a temporary directory, and always remove that directory, even if there's an error, but stop processing and propogate the error). Command-line parsing is esoteric and odd, even with getopt. That's not to say that it's impossible to make a secure shell script that handles filenames with spaces in them properly. Just that it's difficult, and makes using common operators like backticks difficult.

Awhile back, I toyed with the idea of making Haskell a shell scripting language. This week, I spent some time to make this a reality. I released HSH, a shell scripting environment for Haskell.

HSH makes it easy to run shell commands, set up pipelines, etc. straight from Haskell. You can either use simple strings to invoke commands (they'll be passed to sh -c), or you can specify arguments as a list (like exec...() takes), which eliminates the strange filename problems.

But the really cool thing is that HSH doesn't just let you pipe from one external program to another. It also lets you pipe to/from pure Haskell functions. Yes, you can pipe the output of ls -l straight into a Haskell version of grep. I've found it to be very nice, especially for more complex processing tasks.

I put these simple examples on the HSH homepage:

run $ "echo /etc/pass*" :: IO String
 -> "/etc/passwd /etc/passwd-"
runIO $ "ls -l" - - "wc -l" 
 -> 12
runIO $ "ls -l" - - wcL
 -> 12

In this example, wcL is a pure-Haskell line-counting function.

The results were surprising. According to SLOCCount, porting hg-buildpackage from a shell script to a HSH script achieved a 20% reduction in source lines of code. And at the same time, gained better error handling, better safety of filenames, better type safety (compile-time type checking), etc. Yet it does exactly the same thing in almost exactly the same way.

Even greater savings will occur too. I decided to reimplement a small part of sed just for fun, and that code is still in my tree. If I removed that and replaced it with a call to sed as in the shell version, that would probably buy another 5% savings.

I didn't really expect to achieve a reduction in lines of code. I thought that I'd be lucky to come close to breaking even. After all, who'd expect something other than the shell to be better at shell scripting?

I don't know if these results are generalizable, but I'm really excited about it.

Never mind the wheels what about the fuel ? Update: Morrisons have withdrawn unleaded petrol from suspected stations as a precaution. Well done! Meanwhile, last I heard, Tesco were still denying that the contaminated fuel was anything to do with them, while flushing their tanks. Hey, it's not their cars which are kangarooing.

Why you won't be voting for me for DPL this year. Or, my plans for Debian stuff over the next year (or three). Most of these plans embody larger themes and directions that I feel Debian should take. I am a strong beliver that working code and getting stuff done is the best way to work toward larger goals, so I won't bother to try to explain the larger goals. constantly usable testing This is so important that it has its own page. This is my first public posting of that proposal, and it should not be considered finished. I plan to finish it, and get to work on it. website www.debian.org is deprecated, but this is not obvious to either users or developers. That's because it's not formally deprecated, we just treat it like it is: We don't use it for anything new. Instead we have wikis, planet, times.debian.net, etc, etc, etc, etc. Unfortunatly that has created a bit of a mess for users, who are still often stuck trying to use www.debian.org, or who can't find new stuff that is on other sites. A solution could be creating an alternate starting point for users, to help them learn about Debian, install it, and point them to docs and all the other useful sites once they're more involved and using Debian. I hope to find time to work on and promote such a thing. I plan to continue moving stuff that needs to be maintained actively and grow off of www.debian.org to the wiki (as I've done with some of d-i stuff, for example), and encourage others to do so as well. Anything under /devel/ on www.debian.org is an especially good candidate for the wiki. double the archive sync frequency again Now that we've switched to updating the archive twice a day, and dealt with the (minor) issues caused by it, it should be easier to increase it to a frequency that is more likely to cause noticible turnaround time improvements and thus speed up some development. As it is, many people sleep through one or the other archive update. So I support doubling it to four times a day, or more (six seems better). automated whole system testing Automated whole system testing is important because Debian needs to keep working in so many obscure and annoying to test situations. Does d-i work on m68k? Does the desktop task install properly and work? Does debootstrap install a usable system on every architecture? If we rely on user reports, things can fall through the cracks entirely or not be noticed for too long. I've been running a d-i lab for a few years, which tries to automatically test installs on 6 architectures. However, between the current kernel not supporting several of my test machines due to hardware and software bugs, and the general time sync of keeping a rack of aging and obscure test boxes working, this has not been very useful lately. I feel that the solution is emulation: It should be possible to run Debian under emulation for most architectures. I hope to get an absurdly powerful server donated, and switch my test lab over to running in emulation on it. developer (virtual) machines The other benefit of emulation is that everything needed for testing stuff under emulation can be easily distributed to others. Getting developer accessible machines for certian architectures has been a problem in Debian. Emulated systems can sidestep this problem. All that's needed is a collection of documents and/or bootable images that developers can use to set up virtual machines for as many architectures as possible. We have steps in this direction. Hercules contains documentation for how to install in it using d-i. There is a kernel flavor and d-i boot images for mips on qemu. Having to ssh out to a DSA adminned box to deal with a buld failure on $ARCH, and having to wait for build deps to be installed should begin to feel antequated and quaint. It is. find the next nslu2 Debian's support for the nslu2 is an excellent development. It opens up using Debian for a whole new set of tasks, that need low-powered, semi-embedded, commodity hardware, that before were the province of special-purpose embedded distros. It also blew the popcon numbers for arm through the roof. We need to find the next nslu2-like target and get to work on it. I'm hoping that it will be a wireless router, because that's the only thing I'm not running Debian on now. :-) But it could also be a cell phone, or PDA..