All the soarings of my mind begin in my blood.
-Rainer Maria Rilke I have made it back from NYC after a pretty intense few days spent at the Open Minds Conference and Icarus-hosted workshops, Get Your Freak On. Both were incredibly interesting and I felt the ambivalent rush of excitement and fear that follows the prospect of working on a new project. While I have spent some time researching critiques of psychiatry, I have not spent all that much time with the people involved, academically and personally, in the various movements. And there is nothing like seeing in the flesh and blood some very passionate political work to get the intellectual excitement brewing. The first time I went to a survivor conference and protest I was living in San Francisco, deeply involved in my hacker research. I learned about the movement from a civil libertarian John Gilmore, who drew inspiration from psychiatric survivors to develop his free speech arguments for drug legalization. I had never heard of them and was intrigued so I naturally poked around the web. And I was fortunate to find out that their conference was being held the following weekend less than 10 blocks from where I was living at the time. I made time to go and unsurprisingly, it was a powerful event, perhaps the most stunning conference I had ever been to. There was something awfully inspiring about a political movement in which the members had experienced heavy doses of trauma, often through heavy doses of “treatment” and yet found the energy, will, and life to engage in political action, and in a society that does most everything to dampen the fire of politics. This weekend was no less inspirational. Even at the academic conference, the personal stories of trauma, survival, and the complex ethical decisions of choice in a landscape dominated by one medical model, erupted frequently. These were important eruptions that sometimes probably for some felt out of place in an academic setting where such outpourings are discouraged, buried deep away never to disturb the calm and rationality of talks. But without them, the conference would have felt dry, staid, and sterile, which would precisely have been the most disappointing atmosphere to have created for this environment. All the talks were engaging and illuminating and it was great to see and hear many folks working on these topics and especially working to carve out a different political and somatic reality. For me, the talks by David Oaks and Jackie Orr were the most electric and hopefully I will soon get the videos of them as well as most others (as I (tried) to tape most of the conference) up for those who are interested. In the meantime, David Oaks has already
released his talk, which is worth checking out. One of the more interesting parts of the conference was that it gave a clear indication of one of the great successes and strengths of the mad movement, which is its staying power, its ability to survive (even if not necessarily known to wider publics) generation after generation, which is more surprising given that the vitality of so many of the political movements of the 1960s and 1970s, vanished or languished. This continuity was marked by David Oaks keynote speech entitled “Unite for a Nonviolent Revolution in the Mental Health System: What 30 Years in the Mad Movement Have Taught Me” and just by the fact that he and Celia Brown were there, both who have been involved for decades. Beyond that, it was also the new faces, the new groups, that include Icarus, and the
Freedom Center, that attest to the ability for this Mad Movement to thus far escape the cruel ravages of time and its intervention, often causing rifts within and across generations. Sasha Dubrul who gave a talk during the activist panel and founded Icarus four years ago talked about the deep alienation he felt when he first had encountered the rhetoric of psychiatric survivors and, even despite his deeply critical stance against psychiatry. He wanted to create an organization and a message and a place that resonates with our actual experiences of mental illness rather than trying to fit out lives into a conventional framework. It is a message that swoops into many territories that includes a stiff, unrelenting critique of psychiatry and the pharma industry all the while admitting that not everyone can survive without pharmaceutical drugs (and being deeply grateful for this), all the while also providing pathways to alternatives. It is not easy territories to navigate, due to the and cracks and crevices between positions but then again, most of life is filled with these bumpy textures and not a smooth plane free contradictions. If Icarus was born in part because existing organizations did not adequately address existing needs and desires, it did not grow and move away from existing organization but moved closer to them and is now is in alliance with
MindFreedom. And this is key. Difference in a political movement adds vitality, depth, opportunity but this can only be brought into healthy fruition with alliances. Otherwise, deep fragmentation follows, which given the already deeply fragmented nature of our lives, and of the political landscape, can end in political stasis. But so far, the Mad Movement has over years taken “mad anger” to fight a mad system, and given the last 30 years, I am sure will provide many more decadesof mad pride.

Finally, my relocation is now finished, and I’m “back on to the world”. The world? The whole world? No! I’m definetly not because my DSL-Connection is not yet activated by Swisscom. First of all I would like to do more for Debian, this is poorwise more or less impossible now, except some activity on the mailinglists during work. Then I’m moving a server from a private customer from Switzerland to Germany and setting up VHCS. Then I have also a stack of books I should read, maybe about Squid, Kerberos and Postfix. I have also to prepare a little Website for my grandfather (I will speak to him about it on thursday) and last but not least I should help my parents with relocating their office. This all is a never ending chain -> I wish the day would have some hours more or I wouldn’t need to sleep, I would like best to have more time for all my friends and just to relax. Here in the office it’s sometimes also a hurry, one person has gone and we’re only 3 persons now, and an “administrative officeman” would be very helpful. By the way I’m also preparing my trip to Berlin in December and preparing for my LPI test. Yeah, that’s life, sorry for my inactivity in Debian and other placesm there will be better times in the future where I have more time.

Well, I've been added to the python-modules project on alioth and am looking at getting the django development in there over the next week, just need to get my head round how this works first and checkout Raphael Hertzog's repackaging of the 0.95 debs that I did in to the new python policy style. Will also need to work out what to do about different python versions, django-admin.py would need to call the right python depending on some setting - not sure how to do that yet - with the previous format there where just 2 different python-django packages for the 2 different versions of python, and then an alternative to point to the "right" django-admin.py within site-packages of the correct python directory - with the central install this is going to be more interesting. Any thoughts on this gratefully recieved to my email address (linked from the bottom of my website.

Just seen this at the Arts Cinema in central Cambridge this evening. It's a well-presented documentary showing Al Gore's lecture tour where he talked about the problem of global warming. He delivers a compelling message, with a fair amount of data to back up his conclusions. I only had a couple of issues with the movie:

• A sequence about his presidential election defeat in 2000; in my opinion all this added was distraction from his points.
• Some of the scary-looking graphs he showed didn't have any clear, useful labels on the Y axis; my own experience from studying science and engineering makes me always suspicious of such things. I hope that this was just a deficiency in the film, and I'll check out the accompanying book to confirm this.

In my opinion, the conclusions from the movie are quite clear:

• Global warming is a problem.
• We need to do something about it, probably a mixture of trying to reduce the causes and coping with the consequences.
• There's a moron in the White House; this is not news to many of us...

Update It helps if you give sort a -n option apparently. The results make a bit more sense now. Well it kind of varies, because I have a weird mode of operation. I SSH into caesar, in my cupboard, which is my local general purpose box. It runs a screen session, which includes an SSH session with daedalus, my server co-located back in Australia, where I read my email via mutt, and then I do really local stuff on my laptop, icarus.

daedalus

     37 fg
     32 ls
     31 sudo
     19 blog
     12 cd
      7 df
      6 vi
      6 mutt
      5 less
      4 man

Unsurprisingly, fg wins out here, because I'm usually in mutt and just background it to get a shell prompt. blog is a funky shell script I wrote to better manage blog posts with Blosxom.

caesar

    152 sudo
     61 ls
     38 less
     35 cd
     27 screen
     19
     13 dpkg
     11 apt-cache
     10 man
      9 ssh

icarus

156 play 103 ls 54 sudo 45 ssh 12 cd 12 apt-cache 10 dpkg 8 wget 8 host 6 telnet play ranks so highly because I was recently listening to all of the audio files that come with Asterisk to stitch together a particularly hellish purgatory for telemarketers.

So it's interesting seeing how the different machines get totally different use.

A bit surprising, really.

: tfheen@thosu ~ > history 1 awk ' print $2 ' awk 'BEGIN  FS=" "   print $1 ' sort uniq -c   sort -nr  head -n 10
  21477 ls
  18170 cd
  10640 ssh
   9257 sudo
   5559 less
   3015 grep
   2407 bzr
   2101 ps
   1980 man
   1980 debuild

For those of you wondering why there is no editor on the list; I use emacs and it's in the 11th place (with 1903 runs). I tend to let it run for a while and open more than one file, so it doesn't get that high on the list. I also cd and ls a lot.

Well it kind of varies, because I have a weird mode of operation. I SSH into caesar, in my cupboard, which is my local general purpose box. It runs a screen session, which includes an SSH session with daedalus, my server colocated back in Australia, where I read my email via mutt, and then I do really local stuff on my laptop, icarus.

daedalus

      7 df
      6 vi
      6 mutt
      5 less
      4 man
      4 grep
      3 mkpasswd
      3 du
      3 cat
     36 fg

caesar

      9 ssh
      9 ps
      9 fg
      8 dict
      8 df
      6 I
      6 dmesg
      6 a
     61 ls
      4 Upload

icarus

9 dpkg 8 wget 8 host 6 traceroute 6 telnet 6 ping 5 less 5 ip 5 HEAD 52 sudo

So it's interesting seeing how the different machines get totally different use.

upstart is a replacement for the init daemon, the process spawned by the kernel that is responsible for starting, supervising and stopping all other processes on the system. The existing daemon is based on the one found in UNIX System V, and is thus known as sysvinit. It separates jobs into different “run levels” and can either run a job when particular run levels are entered (e.g. /etc/init.d/rc 2) or continually during a particular run level (e.g. /sbin/getty). The /etc/init.d/rc script is also based on the System V one (and is in the sysv-rc package), it simply executes the stop then start scripts found in /etc/rcN.d (where N is the run level) in numerical order. Why change it? Running a fixed set of scripts, one after the other, in a particular order has served us reasonably well until now. However as Linux has got better and better at dealing with modern computing (arguably Linux’s removable device support is better than Windows’ now) this approach has begun to have problems. The old approach works as long as you can guarantee when in the boot sequence things are available, so you can place your init script after that point and know that it will work. Typical ordering requirements are:

• Hard drive devices must have been discovered, initialised and partitions detected before we try and mount from /etc/fstab.

• Network devices must have been discovered and initialised before we try and start networking.

This worked ten years ago, why doesn’t it work now? The simple answer is that our computer has become far more flexible:

• Drives can be plugged in and removed at any point, e.g. USB drives.

• Storage buses allow more than a fixed number of drives, so they must be scanned for; this operation frequently does not block.

• To reduce power consumption, the drive may not actually be spun up until the bus scan so will not appear for an even longer time.

• Network devices can be plugged in and removed at any point.

• Firmware may need to be loaded after the device has been detected, but before it is usable by the system.

• Mounting a partition in /etc/fstab may require tools in /usr which is a network filesystem that cannot be mounted until after networking has been brought up.

We’ve been able to hack the existing system to make much of this possible, however the result is chock-full of race conditions and bugs. It was time to design a new system that can cope with all of these things without any problems. What we needed was an init system that could dynamically order the start up sequence based on the configuration and hardware found as it went along. Design of upstart upstart is an event-based init daemon; events generated by the system cause jobs to be started and running jobs to be stopped. Events can include things such as:

• the system has started,

• the root filesystem is now writable,

• a block device has been added to the system,

• a filesystem has been mounted,

• at a certain time or repeated time period,

• another job has begun running or has finished,

• a file on the disk has been modified,

• there are files in a queue directory,

• a network device has been detected,

• the default route has been added or removed.

In fact, any process on the system may send events to the init daemon over its control socket (subject to security restrictions, of course) so there is no limit. Each job has a life-cycle which is shown in the graph below: The two states shown in red (“waiting” and “running”) are rest states, normally we expect the job to remain in these states until an event comes in, at which point we need to take actual to get the job into the next state. The other states are temporary states; these allow a job to run shell script to prepare for the job itself to be run (“starting”) and clean up afterwards (“stopping”). For services that should be respawned if they terminate before an event that stops them is received, they may run shell script before the process is started again (“respawning”). Jobs leave a state because the process associated with them terminates (or gets killed) and move to the next appropriate state, following the green arrow if the job is to be started or the red arrow if it is to be stopped. When a script returns a non-zero exit status, or is killed, the job will always be stoped. When the main process terminates and the job should not be respawned, the job will also always be stopped. As already covered, events generated by the init daemon or received from other processes cause jobs to be started or stopped; also manual requests to start or stop a job may be received. The communication between the init daemon and other processes is bi-directional, so the status of jobs may be queries and even changes of state to all jobs be received. How does it differ from launchd? launchd is the replacement init system used in MacOS X developed as an “Open Source” project by Apple. For much of its life so far, the licence has actually been entirely non-free and thus it has only become recently interesting with the licence change. Much of the goal of both systems appears initially to be the same; they both start jobs based on system events, however the launchd system severly limits the events to only the following:

• system startup,

• file modified or placed in queue directory,

• particular time (cron replacement),

• connection on a particular port (inetd replacement).

Therefore it does not actually allow us to directly solve the problems we currently have; we couldn’t mount filesystems once the “filesystem checked” event has been recived, we couldn’t check filesystems when the block device is added and we certainly couldn’t start daemons once the complete filesystem (as described by /etc/fstab) is available and writable. The launchd model expects the job to “sit and wait” if it is unable to start, rather than provide a mechanism for the job to only be started when it doesn’t need to wait. Jobs that need /usr to be mounted would need to spin in a loop waiting for /usr to be available before continuing (or use a file in a tmpfs to indicate it’s available, and use that modification as the event). This is not especially surprising given that Apple have a high degree of control over both their hardware and the actual underlying operating system; they don’t need to deal with the wide array of different configurations that we have in the Linux world. Had the licence been sufficiently free at the point we began development of our own system, we would probably have extended launchd rather than implement our own. At the point Apple changed the licence, our own system was already more suitable for our purposes. How does it differ from initng? Initng by Jimmy Wennlund is another replacement init daemon intended to replace the sysvinit system used by Linux. It is a dependency-based system, where upstart is an event-based system. The notion of a dependency-based system is interesting to talk about at this point. Jobs declare dependencies on other jobs that need to happen before the job itself can be started. Starting the job causes its dependencies to be started first, and their dependencies, and so on. When jobs are stopped, if running jobs have no dependencies, they themselves can be stopped. It’s a neat solution to the problem of ordering a fixed boot sequence and the problem of keeping the number of running processes to a minimum needed. However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started. Initng uses run levels to ensure this happens, where a run level is a list of goal jobs that should be running in that run level. It’s also not clear how the dependencies interact with the different types of job, a dependency on Apache would need the daemon to be running where a dependency on “checkroot” would need the script to have finished running. Upstart handles this by using different events (“apache running” vs. “checkroot stopping”). Again while interesting, Initng does not solve the problems that we wanted to solve. It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot. A typical example would be that if the only dependency on the job that configures networking is the mount network filesystems job, then should that job fail or notbe a goal (e.g. because there are no network filesystems to be mounted) the result is that network devices themselves will not be configured. You could make everything a goal, and just use the dependencies to determine the order, however this is less efficient than just ordering the existing sysv-rc scripts (which can be done at install time). Another example is that often you simply don’t know whether something is a dependency or not without reading other configuration, for example the mount network filesystems may be a dependency of everything under /usr or may just be a dependency of anything allowing the user to login if it just mounts /home. The difference in model can be summed up as “initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to.” How does it differ from Solaris SMF? SMF is another approach to replacing init developed by Sun for the Solaris operating system. Like initng it’s a dependency-based system, so see above for the differences between those systems and upstart. SMF’s main focus is serive management; making sure that once services are running, they stay running, and allowing the system administrator to query and modify the states of jobs on the system. Upstart provides the same set of functionality in this regard, services are respawned when they fail and system administrators can at any time query the state of running services and adjust the state to their liking. Will it replace cron, inetd, etc? The goal of upstart is to replace those daemons, so that there is only one place (/etc/event.d) where system administrators need to configure when and how jobs should be run. In fact, the goal is that upstart should also replace the “run event scripts” functionality of any daemon on the system. Daemons such as acpid, apmd and Network Manager would send events to init instead of running scripts themselves with their own perculiar configuration and semantics. A system administrator who only wanted a particular daemon to be run while the computer was on AC power would simply need to edit /etc/event.d/daemon and change “on startup” to “on ac power”. What about compatibility? There’s a lot of systems administrators out there who have learned how Linux works already and will not want to learn again immediately, there’s also a large number of books that cover the existing software and won’t cover upstart for at least a couple of years. For this reason, compatibility is very important. upstart will continue to run the existing scripts for the forseeable future so that packages will not need to be updated until the author wants. Compatibility command-line tools that behave like their existing equivalents will also be implemented, a system administrator would never need to know that crontab -e is actually changing upstart jobs. Does it use D-BUS?

“To D-BUS people, every problem seems like a D-BUS problem.” —Erik Troan

The UNIX philosophy is that something should do just one job, and do it very well. upstart’s one job is starting, supervising and stopping other jobs; D-BUS’s one job is passing messages between other jobs. D-BUS does provide a mechanism for services to be activated when the first message is sent to them, thereby starting other jobs. Some people have taken this idea and extended it to suggest that all a replacement init system need do is register jobs with D-BUS and turn booting into a simple matter of message parsing. This seems wrong to me, D-BUS would need to be extended to supervise these services, provide means for them to be restarted and stopped; as well as deal with being process #1 which means cleaning up after children whose parent’s have died, etc. It seems far simpler to arrange for D-BUS to send an event to init when it needs a service to be started, and focus on being a very good message passing system. The IPC mechanism used by upstart is not currently D-BUS because of various problems, however it’s always been expected that even if init itself doesn’t communicate with D-BUS directly, there would be a D-BUS proxy that would ensure messages about all init jobs and events would be given to D-BUS and D-BUS clients could send messages to init to query and change the state of jobs. What is the implementation plan? Because this is process #1 we are changing, we want to make sure that we get it right. Therefore instead of releasing a fully-featured daemon and configuration to the world, we’re developing it in the following stages:

1. Principal development; at the end of this stage the daemon has been implemented and can manage jobs as described.
2. Replacement of /sbin/init while running the existing sysv-rc scripts. This is the shake-down test of the daemon, can it perform the same job as the existing sysvinit daemon without any regressions?
3. /etc/rcS.d scripts replaced by upstart jobs. These consitute the majority of tasks for booting the system into at least single-user mode, and contain many of the current ordering problems and race conditions. If the daemon solves the problems here, it will be a success.
4. Other daemon’s scripts replaced by upstart jobs on a package-by-package basis; this will be an ongoing effort during which upstart will continue running the existing sysv-rc scripts as well as its own jobs. During this time the event system may be tweaked to ensure it truly solves the problems we need.
5. Replcement of cron, atd, anacron and inetd. This will happen alongside the above and result in a single place to configure system jobs.
6. Modification of other daemons and processes to send events to init instead of trying to run things themselves.

The current plan is that we will be at least part of the way into stage #3 by the time edgy is released, with that release shipping with upstart as the init daemon and the most critical rcS scripts being run by it to correct the major problems For edgy+1 we hope to have completed stage #5 and be at least part of the way into the implementation of stage #6. From the start of development of edgy+2, no new packages will be accepted unless they provide upstart jobs instead of init scripts and init scripts will be considered deprecated. What state is it in now? The init daemon has been written and is able to manage jobs as described above, receiving events on the control socket to start and stop them. This has now been uploaded to the Ubuntu universe component in the upstart package for testing before it becomes the init daemon. We welcome any experienced users who want to help test this; install the package and follow the instructions in /usr/share/doc/upstart/README.Debian to add a boot option that will use upstart instead of init. If your system boots and shut downs normally (other than a slightly more verbose boot without usplash running) then it is working correctly. Other types of events will be added as required during development and testing. Currently only a basic client tool (initctl) has been written, compatibility tools such as shutdown will be written over the next week or two before it replaces our sysvinit package.

Here at Canonical Towers we have several staff who worship at the altar of Extreme Programming, and as such many of the methodologies and rituals prescribed by that religion find their way into our day-to-day working practices. A few of these came together in an interesting way a few weeks ago, and it was suggested by a cow-orker that I blog about it so he could give the URL to people. The first ritual is that of the sprint, I don’t think this is orthodox XP but rather something inspired from it that we picked up from the Python community. With all of the Canonical developers scattered across the globe mostly working from their own homes, it’s become a useful tool; particularly for the Launchpad team. The basic idea for those that haven’t heard of it is simple, and perhaps obvious; you get a selection of the team together in one place, sit them around the same table with particular goals to complete. In effect it’s highly compressed facetime and high bandwidth interaction to nail those tasks that need it, before you head off again and work at a more sedate pace. The second is directly from XP doctrine and is that peculiar observance known as pair programming, something that is often coupled with sprints. For anybody who hasn’t encountered this before, it’s something I’ve always found rather odd. You get two programmers together, both itching to code, and you take one of them’s laptop away; and you don’t just force them to fight over the keyboard either, the laptop-less soul isn’t allowed to touch it. The theory here is that it frees the deprived individual of the hassle of coding and allows them to direct the programmer in ways that may not be immediately obbious, or alternately think about the next bit of work that needs doing. Another interesting side-effect is it can be an interesting way to learn code you’ve not worked with before, if you’re the one doing the coding and you’re being guided what to do by a sage who already knows it. I have a funny story here, so I’m going to digress from the main stream for a moment to tell it. A few months ago I was working in the London office, Mark had been up all night coding and had then given me responsibility to get his changes landed onto the Launchpad mainline. I’m not that hot on quite a bit of Launchpad, and fade to utterly clueless the nearer the code gets to the web application itself; and when trying to merge the two there were conflicts which I needed Mark’s help to resolve. Now as anybody who’s met Mark knows, he’s a pretty good example of a Type-A personality, yet he suggested that we do a spot of pair programming to get the code in and I’d be the one with the laptop as it’d help me get to grips with the affected parts. This hilighted something I’ve always seen as a problem with pair programming, dealing with it when the a person with the keyboard has a totally different way of working to you. In particular, I’m an emacs user, whereas Mark is a die-hard vim user; but also right down to the working directory I would work from, how I test results, etc. When you’re the one without the laptop, this can be quite infuriating, as you know exactly how to solve a problem and the idiot with the keyboard is dithering about doing things you don’t understand just to get there. If you’re a Type-A personality, you generally snatch the laptop away at this point and do it yourself; or at least you would, if you could drive the strange editor the other person uses. I swear Mark was sulking as he gave the laptop back and let me do it. Anyway, there is some relevance to that and we’ll come to it in a moment. The third and final methodology is the concept of unit tests and more specifically test driven development. For anyone who hasn’t come across this before, I highly recommend it; I was suspicious too when Robert Collins took it upon himself to teach me the true way, but now I’m sold. Simply the idea is that if you have any code that doesn’t have another piece of code in a test suite that checks that it’s working correctly, that code is broken. More particularly unit testing involves checking just one feature or requirement at a time, and stacking them together to test all of the code paths. I’ve actually found that this improves the APIs I design, as I write the code in small blocks and functions to make them easier to test. Test driven development takes it even further, you write the unit tests first, before you write the code they’re supposed to test; usually one or two at a time. Obviously these tests will fail at first, it’s then your job to write as little code as possible to make them pass. If your code isn’t right, you add a unit test that will fail, and modify the code to make it pass. This really comes into its own when you’re fixing bugs later, once you’ve identified the bug you write a test case or two that cause the problem; these will of course fail. You can then modify the code to make them pass, and at the same time be sure you’ve not broken other functionality because of the existing test suite. I’ve also found it really useful for particularly complicated or tricky pieces of code, especially those intricate algorithms that do particularly heavy lifting. So onto the event itself, this was a sprint in our London office to indoctrinate Gustavo Niemeyer with the various projects he’d be working on. The goal for this sprint was decided to start the conversion of HCT to Bazaar-NG, and to do this Gustavo and I would pair program. Now my last experience of pair programming had been that story with Mark (see, it had some relevance) and I knew I’d be just as bad if I wasn’t allowed to have the keyboard. I was also acutely aware that if Gustavo was just sitting by and watching, he wouldn’t get much benefit from it either, so he needed to be actively involved in the coding so he could learn the things he’d be working on. I came to what I thought was a pretty neat, and obvious, solution to these problems. We set up my laptop with the code we’d be working on, and on my display were two side-by-side terminals both running screen sessions. Gustavo then set up two same-sized terminals on his, ssh’d into my laptop from both of them and joined the screen sessions. In the left-hand one he ran vi, and in the right-hand one I ran emacs. Thus we both had keyboards, and both had editors, yet could see each other working and even steal the keyboard without danger of violence. We didn’t just sit side-by-side and code on different things though, that’s not pair programming and is just ordinary programming with a bit of a voyeuristic twist. What we did was: in my terminal I started writing the test cases for the code we needed, it was Gustavo’s job to write the code that would make them pass. We actually added a third terminal in which we could run the test cases themselves; so I could run them when I’d added something that would fail, and Gustavo could run them when he thought he’d made them pass again. This turned out to be a rather fun way to work, and at one point I was almost able to try and convince myself that the code was writing itself to pass the test cases I was writing. It also got me thinking that it’d be really neat to use genetic algorithms to breed code to pass test cases.

Barely 3 weeks into my time in Edmonton, I am off to NYC, so that I can attend the following conference: OPEN MINDS: CULTURAL, CRITICAL, AND ACTIVIST PERSPECTIVES ON PSYCHIATRY. I resisted going, in part because I travelled plenty this summer, in part because traveling is such a pain now, but mostly because it somewhat preverse to hop over to NYC in a day for 3-4 days when it took me a week of non-stop driving to get out here in the first place! But the allure of the conference was too strong, so back I go, down south and out east.

For the project that I’m dealing with at work I wanted to be able to authenticate devices in a two stage process. Stage 1 should authenticate the device to the network (via an x509 certificate) and then Stage 2 should authenticate the user who possesses the device with a username and password. Unfortunately none of the methods shipped by default with FreeRadius will support this sort of configuration.

• EAP-TLS supports mutual certificate authentication between the server and the client but does not allow for a second stage to verify the username and password
• EAP-TTLS and EAP-PEAP both support mutual certificate authentication between the server and the client followed by a second stage password verification, however the current FreeRadius implementation doesn’t let you require a client certificate. If one is present it will use it, but it is just as happy to continue on without one, which is not ideal for my circumstances.
• The remaining EAP- methods don’t support certificates at all, so they’re not even an option.

A quick skim through the FreeRadius source code revealed that it wouldn’t be too hard to add support for requiring client certificates with EAP-TTLS and EAP-PEAP. The following patch adds a new configuration option to the tls section of eap.conf which if set to true will require the client to present a certificate before authentication will succeed.

• freeradius-require-tls-client-cert.diff

Example eap.conf

eap  
                default_eap_type = ttls
                timer_expire     = 60
                ignore_unknown_eap_types = no
                cisco_accounting_username_bug = no
                md5  
                 
                tls  
                        private_key_file = $ raddbdir /certs/radius.key
                        certificate_file = $ raddbdir /certs/radius.pem
                        CA_file = $ raddbdir /certs/cacert.pem
                        dh_file = $ raddbdir /certs/dh
                        random_file = $ raddbdir /certs/random
                        # This is the new option
                        # If set to no, or missing, client certificates are
                        # not required for EAP-TTLS or EAP-PEAP
                        require_client_cert = yes
                  
                 ttls  
                        default_eap_type = md5
                 
         

Joey Hess from Debian has come up with a pretty nifty analysis of thread patterns in email. This is a great example of the geek technique to transform what seems like chaos (and can surely be experienced as such) into something more categorical, segmentable, and thus manageable.

With the ever increasing number of trolls, the average developer has no more time to maintain his packages. Motivating him to work on his packages instead of reading debian-vote is a hard challenge.

Here comes BAD-tank, which is the way to:

• motivate developers to work on their packages in the best interest of users;


• motivate users to get interest in what developers do;


• find a way for possible sponsors to spend their money in a useful manner.

BAD stands for Bribe Assisted Development. The idea is to add a new field, named XC-Bribe, to the control file. The field must contain the amount of money the maintainer requires for an upload, with a ISO-4217 currency code. For example, the maintainer might add:

XC-Bribe: EUR 50.00

to require 50 to get the package uploaded.

Then, the package shall be uploaded to the BAD-tank FTP queue. The related sources and binaries will be automatically put into a private area, while the .changes remains public. Users can then read the .changes and decide whether they want to contribute to this upload. BAD-tank accepts Visa, Mastercard, AmEx and PayPal. When the amount of the bribe is reached, the package is automatically uploaded to the Debian ftp-master queue, and the developer is credited with the bribe.

I’m back from Japan. Time went too fast… Met a lot of people we hadn’t seen for a while, had a great time. Only problem is that I’m totally jetlagged. Woke up at 3am this morning, and decided to actually do something after trying to sleep some more for an hour. The downside of holidays is the 4000+ mails (excluding spam) that got in my mailbox. I’ve already gone through most of the new bug reports and have been able to vote for the last GR that was issued while I was away. One of the most annoying problem that was raised during my vacation is this Firefox trademark issue. We’ve been avoiding to change the package name for a while. We got agreements to use the Firefox name when we had to drop the “Mozilla” part of “Mozilla Firefox”, and now it seems they (again) changed their mind. I really think we should go with the iceweasel name because this name is already widespread as the “debian may rename firefox this way” name. It is even in wikipedia. On other news, we bought a MacBook and a Nikon Coolpix S6 for Miki, and a (already broken) PSP as well as some books for me. I’ll write more on these in coming posts.

We're having quite an interesting discussion on the Hecl mailing list about the semantics and syntax of the language (since it's in its early days, we get to do that!). I have some questions for those of you reading this, if you have time to comment. Hecl inherits a lot of things from Tcl, which is both a simple, and amazingly flexible language. One of the things that Tcl lets you do is modify the value of a variable. For instance, set x [list 1 2 3] ; lappend x 4;. However, what is at times confusing for people is that to get the value of a variable, you place a dollar sign in front of the variable: lindex $x 0 would return 1 for instance. To some people this is all quite intuitive, because commands that modify the variable take its name, which is then looked up in a hash table and fiddled with as needs be. So we have constructs like if $n == 10 incr n . To other people, especially those who come from the Perl and PHP worlds, variables always have a $ in front of them, so the Tcl way of doing things doesn't make as much sense. Even for experienced programmers, it can at times require a moment to recall how a particular command works. lappend foo ; lrange $foo ; lset foo ; lindex $foo and so on. On the plus side, this is all very easy to implement, and really isn't that difficult to figure out. One avenue I've pursued with Hecl is to be able to pass references around, instead of variable names, so that all commands just take "a variable": lappend $foo x instead of Tcl's lappend foo x, and so on. For the moment, I've also enabled proc's (user defined commands/functions) to receive references to variables passed to them, so they can modify them. This is both powerful, and dangerous:

set x 1
proc foo  v   
    set v [* $v 10]
 
foo $x
puts "foo is $x" ;# $x is now 10

(Well, not exactly, but that's the general idea). We are considering ideas like marking the variables that may be modified in the proc definition to reduce the danger of people shooting themselves in the foot. In any case, it increases consistency, because commands that modify values always take a $variable. That still leaves commands like set foo 10 (others are proc, foreach, and catch) that take variable names to be created, rather than $variable $references, so perhaps to someone who comes from the world of PHP, it's still not as clear as it could be. We're also considering some radical changes, for the sake of not excluding any particular direction: taking a more lisp/ruby/python like approach, and not using dollar signs for variables (except perhaps in string interpolations). This has some slight disadvantages in terms of requiring quotes around more things (in Tcl, you don't need to quote single world strings). This might result in code that looks something like this:

proc test  name code result   
   catch code res
   if   eq res result    
       ok name
     else  
       fail name res result
    
 
proc testfiles  files   
   global ok
   global failed
   # clear success/error list
   set ok [list]
   set failed [list]
   foreach f files  
   puts "Running $f"
       source f
    
 

Which isn't bad, although in some places, I think the added syntax of $ helps to make it clearer, at a glance, what's what. eq $res $result is just that much quicker to pick out. This is especially true if you stash a command name in a variable. In other words, set p puts ; $p "hello world" might be a bit a bit clearer than set p "puts"; p "hello world", but... it's definitely "eye of the beholder" territory, and depends on what you're used to! In any case, it's been a very interesting process, and I've greatly enjoyed the company of Alexandre Ferrieux and Wolfgang Kechel on the list, as well as my friend Salvatore Sanfilippo. They're all very bright guys with good ideas, which makes for lots of thought-provoking discussions. Where you come in, if you've made it this far... is what kind of syntax and semantics you prefer, and why? Call by name? By reference? Modifiable values? Tell me what you think!

Seen caveman. Much funny, much Ha, ha! Thanks to woman, bought tickets.

If everyone thought that accepting bogus obligations just to be allowed to name something by it&aposs name, take a look at Eric Dorland&aposs blog or directly into the new problems.

My vote for this: Call it firesomething or mffbrowser or some other free name once and forall. With some luck somebody will then also write a nice patch to have a common Debian ca-certificate handling. (I&aposm sick of having to do anything twice, especially if it includes writing mozilla extensions adding a ca-certificate every time a user loads its config as I&aposm ignorant in all this stuff to know any better way). Having things as similar as possible in different environments is a nice goal, but having working solutions and the right to implement working solutions is much more important...