It’s been over a year now since my last proper upload to Debian, and nine months since I announced my intent to put aside working on Debian for a while. With Matthew Garrett’s resignation from Debian, several people have compared it to my own “resignation” from Debian. It has got my thinking about whether I currently intend to ever end my “Sabbatical” and return as an active Debian Developer. I think that the end of my love-affair with Debian started at Debconf last year where several developers treated those of us who also worked on Ubuntu quite rudely. Someone was attacked for wearing an Ubuntu t-shirt at the conference, while someone else was applauded for wearing a “Fuck Ubuntu” t-shirt. That’s where I realised that maybe I didn’t have as much in common with these people as I thought I did. I still don’t understand why Debian singles Ubuntu out for this kind of treatment, we’re still the only derviative distribution that makes all of our patches to Debian available, yet Ubuntu is claimed to not do anything at all. Another example is that Ubuntu is being asked to change the Maintainer field of every package, something no other derivative is being asked to do or has ever been asked to do. Martin Krafft has had some interesting things to say about this strange relationship in the past. If that was the start of my falling out with Debian, I think that Debian considering removing documentation and firmware from the distribution, especially the documentation, was another point I started wondering whether I shared anything in common with the project anymore. Call me strange, but I think that one of the fundamental purposes of a Linux distribution is to be useful to its users. If nobody can use the distribution because it doesn’t support their hardware, and even if it did, all the documentation has been stripped out; I started to wonder what it’s aims are. It became increasingly apparent that the only users Debian was considering a priority were its own developer. And the third thing is simply a matter of Fun. Fun for me, at the moment at least, has been to build a system that fits together extremely nicely with each component doing the right thing. For Ubuntu this has meant being a driving force for it being Linux 2.6 only, with reliance on udev for hardware, etc. Upstart is just a continuation of these goals, getting a system which all “just works”, even if it means throwing out a few things people were previously fond of. All of these things would have been impossible to do in Debian itself. Getting upstart installable required changes to twelve different packages, including sysvinit itself; at a worst case, this would have required the agreement of twelve different maintainers in Debian. It’s often exhausting just persuading one of the reason for the change, persuading a dozen would have been a herculian effort. Perhaps Matthew is right, what Debian lacks is a single leadership. There’s nobody in Debian I could have gone to for approval over the changes I wanted to make, whose decision developers cannot overrule. Ubuntu has such people (the Technical Board and sabdfl), which gives the project an obvious direction instead of a couple of hundred people all pulling in different directions. In a way, I don’t feel that I’ve left Debian. I feel like I was happily going along, only to realise the mob had gone in a different direction and with no easy way of rejoining the group.

Today I reached another milestone in the development of upstart, the packages in universe can now replace the existing sysvinit package. Before trying this, make sure your installation is up to date as we’ve had to split out some parts of sysvinit into a new sysvutils package. If you’re up to date, and want to try it out, install the upstart and upstart-compat-sysv packages from universe. Note that the first reboot after you’ve installed the packages (from sysvinit to upstart) will be a little tricky… use reboot -f. If your system boots and shuts down normally, everything’s working just fine. Note that both will be somewhat more quiet than you’re used to, unless you have usplash running. Throughout the rest of this entry, I’ll try to answer some of the questions and comments that I’ve received since the last post. Events As I talked about previously, upstart is an event-based init daemon. Events are the primary force for having your services and tasks started and stopped at the appropriate time and in the appropriate order. So what are events and where do they come from? (Note that this part is under development, so may change in later releases). Events are just simple strings that may be sent by any process when something it is tracking the state of changes. They have no state or longevity, and if, when queued, they do not cause any job state changes, then they have no effect unless they are sent again. Jobs can list which events cause them to be started if they are not already running and which events cause them to be stopped if they are running. Multiple start and stop events may be listed, in which case the first to occur changes the job until the next one occurs. upstart itself generates the following system events:

• “startup”, on system boot.
• “shutdown”, when the system is about to be shut down.
• “stalled”, when there are no jobs running and no events in the queue.

The shutdown tool included in the package also causes one of the following events to be sent once the “shutdown” event has been handled:

• “reboot”,
• “halt”,
• “poweroff”,
• “maintenance” (aka. going into “single user” mode),
• any user-defined event with shutdown -e event.

Jobs also generate events whenever they change state, this is the primary source of events for ordering:

• “jobname/start”, when the job is first started.
• “jobname/started”, once the job is running.
• “jobname/stop”, when the job is first stopped.
• “jobname/stopped”, once the job has stopped.
• “jobname”, for services this is generated once it is running; for tasks this is generated once it has finished.

And as mentioned, any other process on the system may send events through the control socket or just by using initctl trigger EVENT. For now this is just the event string, however it’s intended that the event may include other details including environment variables and even file descriptors. Typical example To clarify how it all hangs together, here’s an example (using fictional names) of how the tasks and events can be arranged to provide race-free mounting of filesystems.

• “udev” service started on the “startup” event.
• udev daemon is configured to send a “new-block-device” event whenever a new block device is added to the system.
• “checkfs” task is started on the “new-block-device” event to check the filesystem.
• “mountfs” task is started when the “checkfs” task has finished and mounts the filesystem if listed in /etc/fstab
• “filesystem-mounted” event is generated whenever a filesystem is mounted.
• “fstab” task is started on the “filesystem-mounted” event, it checks the list of mounted filesystems against /etc/fstab and if all are mounted, generates the “writable-filesystem” event.
• other services and tasks would be started on the “writable-filesystem” event.

By breaking this job into these small tasks, we can see how the pieces fit together. Because everything is now done on events, there are no race conditions; we know that any filesystem listed in /etc/fstab will be checked and mounted. The only reason they wouldn’t be is if there’s an error of some kind, and that means you have larger problems anyway and the system administrator would have a shell to fix it. Of course, the moment they finish checking the filesystem and mount it, the boot process would carry on. There’s no reason that any of these events need to be generated by the upstart daemon itself, it can receive them from any other daemon on the system such as udev, acpid, etc. This keeps the focus of the init daemon narrow. A large part of the future development will be working out exactly what kinds of events we want init itself to generate, what kinds we want to come from elsewhere, and what the contents of an event can be. Getting Involved If you want to get involved with trying to nudge the direction of upstart development, you can join the upstart-devel mailing list at http://lists.netsplit.com/ Or if you just want to grab the source code, tarballs are published at http://people.ubuntu.com/~scott/software/upstart/ and the bzr archive is at http://bazaar.launchpad.net/~keybuk/upstart/main

Wikipedia has been in the news a lot lately. Some of the articles are quite insightful:

• Know It All: Can Wikipedia conquer expertise? in the July 31 New Yorker.
• The Hive in the September issue of the Atlantic Monthly

Each article examines a different aspect of Wikipedia knowledge and culture, and they fit together nicely (although presumably unintentionally). Both articles are proof that traditional print media can still be relevant and coexist semi-peacefully with peer production efforts like Wikipedia. They are also excellent examples of something other than a knee-jerk defensive reaction by incumbent media to a perceived threat (e.g., this FOX news story or Britannica’s response to Nature’s article comparing Wikipedia to Brittanica). Meanwhile, an interesting unrelated Wikipedia commentary in the blogosphere:

• Seven Wikipedia Fallacies by enfant terrible James Grimmelmann. It’s nice to see James blogging more, now that his judicial clerkship is over. (Is he still an enfant terrible now that he has finished his clerkship? Only time will tell.)

Queen Elizabeth II and the Debian project threaten religious warfare! D-Inq has today uncovered a conflict between the Debian Project and the queen of England and other places. The row is over the King James Version of the Bible. "It's all mine! Mine! Mine!" the monarch told D-Inq in an exclusive interview. "Keep yer dirty fingers off it or I'll sick the SAS on you." In the UK, the monarch owns the copyright for the KJV Bible, but the Debian project wishes to distribute it freely. The SAS, UK's elite fighting force and the Queen's personal mob, is already gearing up. Instead of their usual arsenal of guns and bombs, they're opting to use a new kind of weapon, the DD Attitude Readjustment Tool, or DART. The Debian Project is undaunted. "We've seen worse, this is a pedestrian transport in a big grassy place in comparison" says a Debian flame war veteran, Steve, 22, from Portland, Oregon. "The SAS is nothing compared to a vigorous discussion about firmware freeness. It's not even as bad as someone spoiling the next episode of B5." Things might get more difficult, though. There are rumors that MI5 has already infiltrated Debian's discussion forums. Worried analysts imply that some of the recent, divisive developments within the project are the result of agent provocateurs. "The new Python policy is one", says Lars W, an analyst who wishes to remain nameless. "No sensible person would have done things like that", he continues. While both sides ramp up their fighting forces, local constabulary in many English towns has already started to confiscate and deport machines running Debian to Libya. Time will tell when they will return.

This weekend I was at Steve's House in Cambridge for the 2006 Debian BBQ. Many thanks to Steve for hosting a great party! There was a spectacular turnout of people. It was nice to see people I've met at UKUUG conferences and such like, but also nice to meet some new people, including Europeans who had ventured over. There are some of pictures from the weekend on flickr, but they're not very good because my camera was broken (more on that another time). There are some (much better) pictures linked at the bottom of this wiki page. On a related note, I'm sad to see Matthew Garrett resign from Debian. He joins several others including Scott James Remnant (both of whom were at the BBQ) who have made the move to "full time" Ubuntu, having realised that Debian just wasn't fun for them anymore.

upstart is a replacement for the init daemon, the process spawned by the kernel that is responsible for starting, supervising and stopping all other processes on the system. The existing daemon is based on the one found in UNIX System V, and is thus known as sysvinit. It separates jobs into different “run levels” and can either run a job when particular run levels are entered (e.g. /etc/init.d/rc 2) or continually during a particular run level (e.g. /sbin/getty). The /etc/init.d/rc script is also based on the System V one (and is in the sysv-rc package), it simply executes the stop then start scripts found in /etc/rcN.d (where N is the run level) in numerical order. Why change it? Running a fixed set of scripts, one after the other, in a particular order has served us reasonably well until now. However as Linux has got better and better at dealing with modern computing (arguably Linux’s removable device support is better than Windows’ now) this approach has begun to have problems. The old approach works as long as you can guarantee when in the boot sequence things are available, so you can place your init script after that point and know that it will work. Typical ordering requirements are:

• Hard drive devices must have been discovered, initialised and partitions detected before we try and mount from /etc/fstab.

• Network devices must have been discovered and initialised before we try and start networking.

This worked ten years ago, why doesn’t it work now? The simple answer is that our computer has become far more flexible:

• Drives can be plugged in and removed at any point, e.g. USB drives.

• Storage buses allow more than a fixed number of drives, so they must be scanned for; this operation frequently does not block.

• To reduce power consumption, the drive may not actually be spun up until the bus scan so will not appear for an even longer time.

• Network devices can be plugged in and removed at any point.

• Firmware may need to be loaded after the device has been detected, but before it is usable by the system.

• Mounting a partition in /etc/fstab may require tools in /usr which is a network filesystem that cannot be mounted until after networking has been brought up.

We’ve been able to hack the existing system to make much of this possible, however the result is chock-full of race conditions and bugs. It was time to design a new system that can cope with all of these things without any problems. What we needed was an init system that could dynamically order the start up sequence based on the configuration and hardware found as it went along. Design of upstart upstart is an event-based init daemon; events generated by the system cause jobs to be started and running jobs to be stopped. Events can include things such as:

• the system has started,

• the root filesystem is now writable,

• a block device has been added to the system,

• a filesystem has been mounted,

• at a certain time or repeated time period,

• another job has begun running or has finished,

• a file on the disk has been modified,

• there are files in a queue directory,

• a network device has been detected,

• the default route has been added or removed.

In fact, any process on the system may send events to the init daemon over its control socket (subject to security restrictions, of course) so there is no limit. Each job has a life-cycle which is shown in the graph below: The two states shown in red (“waiting” and “running”) are rest states, normally we expect the job to remain in these states until an event comes in, at which point we need to take actual to get the job into the next state. The other states are temporary states; these allow a job to run shell script to prepare for the job itself to be run (“starting”) and clean up afterwards (“stopping”). For services that should be respawned if they terminate before an event that stops them is received, they may run shell script before the process is started again (“respawning”). Jobs leave a state because the process associated with them terminates (or gets killed) and move to the next appropriate state, following the green arrow if the job is to be started or the red arrow if it is to be stopped. When a script returns a non-zero exit status, or is killed, the job will always be stoped. When the main process terminates and the job should not be respawned, the job will also always be stopped. As already covered, events generated by the init daemon or received from other processes cause jobs to be started or stopped; also manual requests to start or stop a job may be received. The communication between the init daemon and other processes is bi-directional, so the status of jobs may be queries and even changes of state to all jobs be received. How does it differ from launchd? launchd is the replacement init system used in MacOS X developed as an “Open Source” project by Apple. For much of its life so far, the licence has actually been entirely non-free and thus it has only become recently interesting with the licence change. Much of the goal of both systems appears initially to be the same; they both start jobs based on system events, however the launchd system severly limits the events to only the following:

• system startup,

• file modified or placed in queue directory,

• particular time (cron replacement),

• connection on a particular port (inetd replacement).

Therefore it does not actually allow us to directly solve the problems we currently have; we couldn’t mount filesystems once the “filesystem checked” event has been recived, we couldn’t check filesystems when the block device is added and we certainly couldn’t start daemons once the complete filesystem (as described by /etc/fstab) is available and writable. The launchd model expects the job to “sit and wait” if it is unable to start, rather than provide a mechanism for the job to only be started when it doesn’t need to wait. Jobs that need /usr to be mounted would need to spin in a loop waiting for /usr to be available before continuing (or use a file in a tmpfs to indicate it’s available, and use that modification as the event). This is not especially surprising given that Apple have a high degree of control over both their hardware and the actual underlying operating system; they don’t need to deal with the wide array of different configurations that we have in the Linux world. Had the licence been sufficiently free at the point we began development of our own system, we would probably have extended launchd rather than implement our own. At the point Apple changed the licence, our own system was already more suitable for our purposes. How does it differ from initng? Initng by Jimmy Wennlund is another replacement init daemon intended to replace the sysvinit system used by Linux. It is a dependency-based system, where upstart is an event-based system. The notion of a dependency-based system is interesting to talk about at this point. Jobs declare dependencies on other jobs that need to happen before the job itself can be started. Starting the job causes its dependencies to be started first, and their dependencies, and so on. When jobs are stopped, if running jobs have no dependencies, they themselves can be stopped. It’s a neat solution to the problem of ordering a fixed boot sequence and the problem of keeping the number of running processes to a minimum needed. However this means that you need to have goals in mind when you boot the system, you need to have decided that you want gdm to be started in order for it, and its dependencies, to be started. Initng uses run levels to ensure this happens, where a run level is a list of goal jobs that should be running in that run level. It’s also not clear how the dependencies interact with the different types of job, a dependency on Apache would need the daemon to be running where a dependency on “checkroot” would need the script to have finished running. Upstart handles this by using different events (“apache running” vs. “checkroot stopping”). Again while interesting, Initng does not solve the problems that we wanted to solve. It can reorder a fixed set of jobs, but cannot dynamically determine the set of jobs needed for that particular boot. A typical example would be that if the only dependency on the job that configures networking is the mount network filesystems job, then should that job fail or notbe a goal (e.g. because there are no network filesystems to be mounted) the result is that network devices themselves will not be configured. You could make everything a goal, and just use the dependencies to determine the order, however this is less efficient than just ordering the existing sysv-rc scripts (which
can be done at install time). Another example is that often you simply don’t know whether something is a dependency or not without reading other configuration, for example the mount network filesystems may be a dependency of everything under /usr or may just be a dependency of anything allowing the user to login if it just mounts /home. The difference in model can be summed up as “initng starts with a list of goals and works out how to get there, upstart starts with nothing and finds out where it gets to.” How does it differ from Solaris SMF? SMF is another approach to replacing init developed by Sun for the Solaris operating system. Like initng it’s a dependency-based system, so see above for the differences between those systems and upstart. SMF’s main focus is serive management; making sure that once services are running, they stay running, and allowing the system administrator to query and modify the states of jobs on the system. Upstart provides the same set of functionality in this regard, services are respawned when they fail and system administrators can at any time query the state of running services and adjust the state to their liking. Will it replace cron, inetd, etc? The goal of upstart is to replace those daemons, so that there is only one place (/etc/event.d) where system administrators need to configure when and how jobs should be run. In fact, the goal is that upstart should also replace the “run event scripts” functionality of any daemon on the system. Daemons such as acpid, apmd and Network Manager would send events to init instead of running scripts themselves with their own perculiar configuration and semantics. A system administrator who only wanted a particular daemon to be run while the computer was on AC power would simply need to edit /etc/event.d/daemon and change “on startup” to “on ac power”. What about compatibility? There’s a lot of systems administrators out there who have learned how Linux works already and will not want to learn again immediately, there’s also a large number of books that cover the existing software and won’t cover upstart for at least a couple of years. For this reason, compatibility is very important. upstart will continue to run the existing scripts for the forseeable future so that packages will not need to be updated until the author wants. Compatibility command-line tools that behave like their existing equivalents will also be implemented, a system administrator would never need to know that crontab -e is actually changing upstart jobs. Does it use D-BUS?

“To D-BUS people, every problem seems like a D-BUS problem.”
—Erik Troan

The UNIX philosophy is that something should do just one job, and do it very well. upstart’s one job is starting, supervising and stopping other jobs; D-BUS’s one job is passing messages between other jobs. D-BUS does provide a mechanism for services to be activated when the first message is sent to them, thereby starting other jobs. Some people have taken this idea and extended it to suggest that all a replacement init system need do is register jobs with D-BUS and turn booting into a simple matter of message parsing. This seems wrong to me, D-BUS would need to be extended to supervise these services, provide means for them to be restarted and stopped; as well as deal with being process #1 which means cleaning up after children whose parent’s have died, etc. It seems far simpler to arrange for D-BUS to send an event to init when it needs a service to be started, and focus on being a very good message passing system. The IPC mechanism used by upstart is not currently D-BUS because of various problems, however it’s always been expected that even if init itself doesn’t communicate with D-BUS directly, there would be a D-BUS proxy that would ensure messages about all init jobs and events would be given to D-BUS and D-BUS clients could send messages to init to query and change the state of jobs. What is the implementation plan? Because this is process #1 we are changing, we want to make sure that we get it right. Therefore instead of releasing a fully-featured daemon and configuration to the world, we’re developing it in the following stages:

1. Principal development; at the end of this stage the daemon has been implemented and can manage jobs as described.
2. Replacement of /sbin/init while running the existing sysv-rc scripts. This is the shake-down test of the daemon, can it perform the same job as the existing sysvinit daemon without any regressions?
3. /etc/rcS.d scripts replaced by upstart jobs. These consitute the majority of tasks for booting the system into at least single-user mode, and contain many of the current ordering problems and race conditions. If the daemon solves the problems here, it will be a success.
4. Other daemon’s scripts replaced by upstart jobs on a package-by-package basis; this will be an ongoing effort during which upstart will continue running the existing sysv-rc scripts as well as its own jobs. During this time the event system may be tweaked to ensure it truly solves the problems we need.
5. Replcement of cron, atd, anacron and inetd. This will happen alongside the above and result in a single place to configure system jobs.
6. Modification of other daemons and processes to send events to init instead of trying to run things themselves.

The current plan is that we will be at least part of the way into stage #3 by the time edgy is released, with that release shipping with upstart as the init daemon and the most critical rcS scripts being run by it to correct the major problems For edgy+1 we hope to have completed stage #5 and be at least part of the way into the implementation of stage #6. From the start of development of edgy+2, no new packages will be accepted unless they provide upstart jobs instead of init scripts and init scripts will be considered deprecated. What state is it in now? The init daemon has been written and is able to manage jobs as described above, receiving events on the control socket to start and stop them. This has now been uploaded to the Ubuntu universe component in the upstart package for testing before it becomes the init daemon. We welcome any experienced users who want to help test this; install the package and follow the instructions in /usr/share/doc/upstart/README.Debian to add a boot option that will use upstart instead of init. If your system boots and shut downs normally (other than a slightly more verbose boot without usplash running) then it is working correctly. Other types of events will be added as required during development and testing. Currently only a basic client tool (initctl) has been written, compatibility tools such as shutdown will be written over the next week or two before it replaces our sysvinit package.

Peter Nuttall is my personal hero for running with my idea to limit N consecutive posts on a planetplanet installation - if you’d like test it and apply it to planet.debian.org, or elsewhere, find it here. (Note: Patch is reversed). Bad day at work today. I spent half the morning wrestling with xen on my laptop - it seems that linux-image-2.6.16-2-xen-686 or linux-modules-2.6.16-2-xen-686 panics on boot in some weird Xen way. (ie. before initrd.) I upgraded to these packages yesterday afternoon, and the problem only showed itself by a failure to boot this morning. Since the laptop is fairly new none of my install CDs were capable of booting it with networking. In the end I downloaded knoppix, broke a window whilst waiting for it to download, and finally installed a non-Xen kernel so I could work. I’ve reported a bug against the kernel package (with a bogus email address which I’ll need to fixup afterwards) but it hasn’t shown up in the bts yet. There isn’t anything obvious in the bootup message just a panic after “disabling trace buffers” so I’m not sure the report will be terribly useful. In other “worky” news four months after making the request I got a second phone line installed in my flat today, so I can get another network connection. Neat. Now I need to track down somebody to replace / fix my window. Thats going to be pricy, but it will have to be done before the weather turns or I’ll freeze in my house. The problem is frustratingly mechanical - the window “handle” is jammed in such a way that I can’t turn the handle and close the window - so it is permananetly open about four inches. The frustration comes primarily because if the window would open more, (it won’t), I could use a screwdrive to unscrew the locking site - and that would allow me to close it. (Albeit leaving it unlocked). Its half-open/half-closed state is just the worse way for it to be stuck.

quest upstart% ./initctl trigger test-event
ping (start) starting, process 5029 active
script (start) starting, process none
script (start) running, process 5031 active
test-event event triggered

quest upstart% ./initctl status ping
ping (start) running, process 5040 active
quest upstart% ./initctl jobs &
[1] 5442
quest upstart% kill 5040
ping (start) respawning, process none
ping (start) running, process 5450 active

quest upstart% ./initctl status ping
ping (start) running, process 5450 active
quest upstart% ./initctl stop ping
ping (stop) running, process 5450 killed

[This is the first post in the new “Cornell” Category]
Well, my room looks quite empty and the van is quite full. We weren’t sure if all the stuff we were planning on taking was going to fit, but it did. Now, the question is will it fit in the 10×10 dorm room that is typical of the building I’m in. We’ll soon find out. We are driving up to Ithaca Thursday, and I move in Friday. Now that it’s getting close, I am starting to get more and more excited.
If anyone recently out of college, or still in it (Ari, joshk, and others) wants to offer any advice for a rising freshman, feel free to leave a comment. In other news (those of you who were helping me in #d-d already know), my server was out of commision for a few days. Grub and lilo both refused to boot off the 200 hard drive that I had moved my data to using a live system, and sarge D-I saw it as the wrong size, and did not see the partitions (perhaps why grub and lilo both failed). This was apparently a kernel issue, and so I installed using the etch beta3 netinst. G-I was working fine, but after partitioning got in a loop where it would drop to console and complain about some fatal error then go back to graphics again (somewhere around setting time zone) so I eventually used the old text system, which worked flawlessly. The only thing that I found I forgot to backup was my wordpress theme directory, so not only do I have to find my theme again (whose name I forget), I have to modify it, since I had customized the CSS. Pity.
The last bit of this post is a thank you to Joerg for approving my NM application. Now all that’s left is for elmo to proccess his backlog.

I went up today to visit Kasper Souren, AKA Guaka, who's visiting wt:Montreal for a few weeks. Kasper and I have been in touch for several years -- he's been involved with Wikitravel almost as long as anyone -- but we just met in person a couple of weeks ago at Wikimania. He's just returned from South America, and will soon depart for the West Coast of North America and then off to Mexico. He's quite a traveller. Guaka's here to help out with the CouchSurfing Project, an excellent wt:hospitality exchange program that lots of people say good things about. There are apparently bunches of CouchSurfers in Montreal right now helping out with technical issues, content, support, and other great Website-errific stuff. Of course, the local CS community is practicing the ethic and sharing their homes with visitors. I'm not 100% sure how it works, CouchSurfing runs on a very collective model, and a lot of people are devoting their time to this get-together. I went to visit this afternoon, partly to catch up with Guaka, and partly to visit the collective. We talked a lot about the difference between Wikitravel and hospitality exchange -- how anonymous the sharing is for an information-oriented project like a wiki, versus the very intimate person-to-person nature of home sharing. I also talked for a while with Casey Fenton, the founder of CouchSurfing and a very nice guy. We all discussed how to let CouchSurfers copy and distribute Wikitravel guides, possibly customizing them for CS needs, and also for linking back to WT to update and improve the data on our guides. I'm pretty jammed on the whole CS thing -- I'm going to sign up quite soon. Should be fun. tags: couchsurfing wikitravel guaka casey fenton hospitality exchange

DekiWiki I'm interested to see how this OpenGarden thing works out for MindTouch. I saw a story about their Wiki product on Port 25, which I have to admit I read pretty regularly. Every once in a while there's something I should probably know. OpenGarden is MindTouch's Open Source initiative, including a wiki engine called DekiWiki, built on the .NET technology, and a wp:REST-based Web app framework called Dream. Apparently the whole thing runs fine on top of Mono, which means theoretically I could install it. But I haven't really done a lot of Mono web stuff before -- no idea how well it works. Also, have you noticed how people involved with Mono say "MOH-NOH", like the actual original Spanish pronunciation? Unlike everyone else, who calls it "MAH-NOH", like mononucleosis? That's a weird thing. OpenGarden has a MediaWiki conversion tool, which is almost a necessity for any wiki engine trying to compete in 2006. They might want to try lobbying those guys at the Mono Project... who appear to be using MW as their main site engine! Anyways, I'm going to give this DekiWiki a try. Who knows? It could be fun. I don't know about their overreaching claims for the GPL, but it can't hurt for a local install, right? I seem to remember there was a Scheme language plugin for .NET somewhere, too... By the way, I'm still bullish on JAMWiki, the MediaWiki-workalike using Java Web Services. But I like to experiment with new software, too. tags: wiki engine mono open garden mindtouch dekiwiki jamwiki wiki

SocialText Speaking of commercial Open Source Wiki software, I was really glad to see that SocialText has released a FLOSS version of their flagship product. SocialText has been such a good corporate citizen in the wiki and FLOSS world. Not only were they instrumental in the launch of BarCamp, but ST employs some of the smartest wiki minds in the world today (like Seb and Sunir). And, of course, Ross Mayfield's blog is a great clearinghouse for wiki business info. ST has made a business model in training companies in the WikiWay. I think that makes a lot of sense, and they show a lot of commitment to the idea by demonstrations like this. tags: socialtext open source wiki ross mayfield

Fired from Technorati for my UFO beliefs So, I was having some random and unpleasant 500 Server Error errors with this blog recently, and apparently Technorati has given up on my site in frustration. No matter how often I do an update, and then ping, it still doesn't refresh my recent articles. I'll have to figure out some tricks to get back in Technorati's good graces. I like the service a lot. tags: technorati 500 ufo beliefs

I’m confused by your blog, Lucas There seems to be no point to it; you’re comparing the stable releases of one distribution against the unstable archive of another one with a completely different release schedule. Of course the software is going to be out of date! In theory, there’s only one time when both Ubuntu and Debian are in sync (for main, at least); that’s Ubuntu’s Upstream Version Freeze. You won’t compare edgy because “not many people use Ubuntu development versions”, I would argue that this is not true and that the development community at least have an edgy machine around by now. And I would argue that the same is true of Debian, it’s the development community that use Debian testing or unstable. A sensible comparison can only be performed between the stable releases, and then you find it’s quite boring. There’s a linear map between the versions of software and the date of release (e.g. hoary has later software than sarge which has later software than warty).

The “foiled” terrorist attack looks suspiciously like an attack on the underlying system, rather than an actual attempt to blow planes up - it’s almost laughable that terrorists in this day and age could hope to get explosive devices onto planes in hand luggage. If we decide that the terrorists aren’t stupid, the attack begins to look decidedly different. They’ve succeeded in causing absolute chaos in one of the busiest weekends in the british airline calendar, the effects of which will probably drag on for a few days. If you wanted to be really nasty, they’ve also caused a large number of people to be trapped in one place, and extremely vulnerable to attack. The BBC report says Heathrow Terminal 1 is “jam packed”. I suspect that a terrorist organisation will look at something like this as relatively cheap, too - 18 peons arrested, sure, but we know they can afford the manpower and they’ve once again demonstrated that the air travel infrastructure our society is so reliant on is extremely fragile and vulnerable.

Today marks the 1 year anniversary of my application to become a Debian Developer. Here is an approximate break down of my time in the NM Queue: Waiting for advocate: 2.5 weeks (This should have been shorter, I had my advocate lined up, but then there was a question about me signing the key of the guy who signed mine, but I hadn’t checked his ID). Waiting for AM: 6 months Going through the checks: 2.5 months. (This would have been shorter had it not been right around the end of school, with the mad rush to cram everything in before exams)
Waiting for FD: 1.5 Months Waiting for DAM: almost 7 weeks so far… Total: 1 year and counting…

Serendipity is my blogging software of choice lately, but in the past I was a happy user of LiveJournal. Fortunately, now my friends at LJ can keep up to date with my current blog, because of a great event plugin which mirrors posts here on LiveJournal (configurable of course). On LJ people use markup like [lj user="jamuraa"] in order to make a handy link to the page of the livejournal user in question. I was missing that feature, so I coded up a small plugin for s9y which converts that into jamuraa. This is version 0.1, which only does the user tag. It also works in comments if you enable it. In the future, I hope to add code to detect the user type correctly (on LiveJournal there are other icons for different user types). I also probably should not hotlink to the .gifs on the livejournal servers.

To install, just unzip into the plugins/ directory of your s9y installation.

Download the LJ Markup Plugin 0.1 for Serendipity

Mark Jaroski asked if I had any recommendations for Web hosting. I thought I'd share them here. Mostly I find Web hosting really difficult to evaluate -- there are just too many options. It's a really competitive business, and the hosting companies have SEO-saturated the Web hosting keyword at Google. On top of that, there are a ton of fake "hosting comparison" sites -- really just fronts for the bottom-feeders of hosting. When you get to even a good hosting company's Web site, there'll be a lot of flashing lights and gewgaws. A lot of companies try to sell you add-on features you don't need, and they'll have abusive UI to get you to click on the wrong thing, like:

 You're eligible for a big discount on our
 blogging software! Only $3.95 a month! 

 [Continue] Cancel

... where the "Continue" button adds the add-on to your account and "Cancel" is what you need to go on with your order. I think this is borderline fraud, but as far as I know nobody's been shut down or fined for it. For contrast, there are a lot of really good community-oriented and Open Source-friendly hosting services out there, like Montreal's Koumbit. They'll usually be able to devote more time and energy to individuals' technical needs. Often, though, small companies and community groups can't get the same economy of scale and thus price level to compete with the big guys. Of course, what you usually trade away for that $0.75 a month is any customer service or friendliness or control of your own site. The big guys are organized around selling hard to inexperienced people on price, then ignoring their requests for help and treating them like idiots for asking until they go away. Getting help with real problems like server downtime or misbehaviour can be as humiliating as having someone explain what "Cut", "Copy", and "Paste" mean, and probably get you as many over-the-phone eye roll pauses. "My Web site is down!" <eye roll> "Did your AOL Chat window get in front of your browser window, sir?" I think the best option for people is to look around, find a group or company that has a price you can live with and business practices that don't turn your stomach, and give them a try. Make frequent local backups of your site, and be ready to leave if you need to. tags: web hosting community comparison customer service

Dumb Contest Julien has started a dumb bug reports contest (you be the judge of what "dumb" applies to here). What's the point? I guess if it catches on we can humiliate Debian users and keep them from filing bug reports at all. And as we all know, no bug reports means no bugs, right? Being abusive towards users has never helped a project improve the quality of its software. Our bug tracking system is one of the things that's made Debian a top-tier operating system. Making fun of Debian users for using it is a very bad idea. tags: bts debian contest

Yay Akron The Akron Beacon Journal has an article about suggested Akron-related topics to add to Wikipedia. That seems like a great idea. tags: akron ohio wikipedia

Wow Douglas Self has some cool items in his Museum of Retro Technology about old technology -- like this rocket bicycle and this gyro-stabilized monorail. Thanks to Nick Moffitt for the links, even though he probably cribbed them from somewhere lame like Slashdot or Boing Boing. tags: douglas self museum retro rocket bicycle monorail nick moffitt

Supportworthy So, I have two blog items about technical support people looking down on those who are reporting problems. And I was thinking how much I work to gain the respect of technical support people so they'll listen to my problems when I have to make a phone call. As little as I like it, I find myself working to qualify to get my questions answered and my problems fixed. So I wonder: what if there were some kind of standardized computer user certification? You could take a series of increasingly-harder tests at some training centre, and you'd give your certification level (or -- better -- your standardized identification number) when you register software or hardware or when you report a bug. People will then know that when you say, "I think postfix isn't accepting connections!", you might actually have a point. tags: certification computer user technical support

JAM on it Maj, of course, was the one who pointed me to the page for a new Wiki engine in development. Ho-hum, I know... there's already too many wiki engines. But this one is interesting in that it aims to be MediaWiki compatible and run in a Java Servlet container. That seems like a killer combination for those people who want "the wiki that Wikipedia uses" but with an "enterprise-strength" Web technology. I've only tried the software a little bit, but it definitely looks like it's already more featureful than 70% of Wiki engines. Taking MW as a model is a great idea for a new Wiki engine -- it's an extremely featureful wiki engine that "fits well in the hand" of writers and users. Already the use of images and the special support pages look good on JAMWiki. And the integration of lucene means an excellent search engine -- something MediaWiki lacks out of the box. The Roadmap looks to have some good plans in the future. Anyways, Maj likes JAMWiki since JAM is MAJ backwards. I can't tell you what to make of that, though. What I do know is that the project needs a logo real bad. tags: wiki wiki engine java jamwiki jam servlet logo

Ajax in 10 weeks So, wp:AJAX is no longer as bonerific a word as it was six months ago, but it's still pretty hot. I mean, we've all stopped nattering about how fantabulous Ajaxical UI is, and just started writing code to make it work. It seems like nobody even uses "Ajax" and "Web 2.0" as synonyms anymore (well... I guess only the people who still call it "CGI/PERL".) But I was interested to see when I was looking for the Java Servlets link above that Sun is part of the cryptic Open Ajax Alliance. They've got an Ajax project incubating called jMaki to integrate Javascript programming on the server and on the client. Neato, say I. Most interesting was an external link to a 10-Week Free AJAX Programming Online Course. Starts August 4th, seems like a nice self-training tool. tags: ajax sun java javascript

Stuck in ol' CGI again So, I've been getting some non-deterministic 500 errors from WiLiKi, and I'm starting to think they're concurrency problems with searching the database the WiLiKi pages are stored in. I also think there may be a FastCGI tie-in, so I've rolled-back to CGI for a while to see if it cuts down on the errors. Annoying, indeed. tags: wiliki wiki cgi fastcgi

Yesterday was such an insanely long day -- we woke up in a tent, had breakfast over a Treo, then released a lot of software and went to sleep in the middle of the night. It seemed like 4 days, not 1. And this morning I had to take our new mini-SUV into the shop for its first maintenance checkup. For the last 6 years, I've driven a beautiful old wp:Citro n DS station wagon, but around the time Amita June was born I decided I wanted her to learn to drive on it. Maj and I decided that that meant it had to be taken care of a little better, so I put it in storage and we bought at 2005 Hyundai Tucson. (The Hyundai people here in Quebec pronounce it like "Tux-awn", to our endless amusement.) It's weird having a new car -- like, newer than 1972. Parts are not only plentiful but inexpensive. There is a selection of multiple mechanics in my major metropolitan area that can service the vehicle. And people don't point and wave when you're driving, or stop you on the street to tell you stories about how their grandfather used to drive one on their family farm. We'll be driving the car to wt:Boston next week for Wikimania 2006, so I'm glad it went to the shop. It should be in good shape for the drive -- I hope the rest of us are. tags: hyundai tucson citro n ds quebec new car

Slides Speaking of Wikimania -- I'm nowhere near ready for my presentation. Jeez, how'd it get to be almost-August already? Thankfully, no one else I've talked to is really ready to talk, either... so at least I'm not the last one to get my homework done. tags: wikimania presentation slides

Software The OpenID software I did for MediaWiki went out yesterday, and I got a lot of good feedback on it. OpenID requires that some state data is stored between participating servers, and the code I wrote uses the memcached client built into MediaWiki to store it. Not optimal but it works. Well, better today than yesterday, that is. There were some pretty gruesome bugs in the implementation that were only showing up for some exceptional circumstances. So I adapted the JanRain unit tests for the other storage classes to my Memcached class, and that helped a whole hell of a lot for debugging. Yay, unit testing! Some people are still having problems with the software, though. The main thing I've found is that the key-value pair output required by OpenID for its inter-server communications is sensitive to extra whitespace thrown in. Why would there be extra whitespace? I've found that a lot of PHP libraries have bits of whitespace at the end, after the closing ?>, and that will show up in the output. Whether it matters depends on how sensitive the output format is to whitespace. In the case of OpenID, the answer is: pretty sensitive. But I'm looking forward to having some more people try the OpenID MediaWiki extension out. tags: openid mediawiki extension unit test php whitespace

Wikipedia's 25th of July Spot-on satire from The Onion: Wikipedia Celebrates 750 Years Of American Independence. I quote:

• "On July 25, 1256, delegates gathered at Comerica Park to sign the Declaration Of Independence, which rejected the rule of the British over its 15 coastal North American colonies," reads an excerpt from the entry. "Little did such founding fathers as George Washington, George Jefferson, and ***ERIC IS A FAG*** know that their small, querulous republic would later become the most powerful and prosperous nation in history, the Unified States Of America."

Does the fact that I find an Onion article funny say more about my declining humor standards, or about their recent improvements? I don't know. tags: the onion wikipedia satire

Four ideas I was very interested to see Tim O'Reilly's Four Big Ideas About Open Source. I think that there are some fascinating developments over the last few years that have fallen under the rubric of wp:Web 2.0, and I think they affect those of us who put our hearts and minds into Open Source software. I think this was part of his keynote at OSCON 2006, which, y'know, I'm not techically at or anything. It briefly crossed my mind, but I've got too much to do for Thermidor and Fructidor to flit off to yet another conference -- this one only tangentially involved with what I do for a living. Still, it's interesting that Open Source software and Web services are crossing paths. FLOSS has been critical for making Web services possible, but taking that up to the meta-level and seeking the same freedoms from Web apps that you expect from "regular" software... that's a Moebius strip of rights right up in there. tags: floss open source oscon tim oreilly

My life as a toolbar So, sometime in the last year I started installing Firefox toolbars. The main one that's useful to me is Chris Pederick's excellent Web Developer Extension, jam-packed with little tools that make my job as a developer that much easier. But I also install, well, pernicious toolbars. Judgemental toolbars. Toolbars that make me forget what really matters and turn the Web into a horserace or a highschool popularity contest. They're horrible, and they're addicting, and I can't stop using them. The Google Toolbar has some diddly little useless gadgets that I don't give a shit about, like searching my Gmail account for the given search terms (whoopee). What I loooove, though, is the wp:PageRank indicator. PageRank, of course, is the patented algorithm developed by Google founder Larry Page to estimate the importance of a page on the Web. It's incredibly accurate and valuable and though no longer the only criteria Google uses for ranking pages, it's still critical as a measure of the intrinsic value of a page. So, there's this little wp:progress bar on the Google Toolbar that shows the PageRank of any page you're looking at. It's an apparently logarithmic scale from 1-10, with the number showing up in a popup tooltip. Creative Commons? 9/10. Ted's story about the great worldwide race? 2/10. Technorati home page? 8/10. Male Secretary? 6/10. Wikitravel's France discussion? 3/10. And so on and so on. Even worse is the A9 Toolbar, which (again) has a whole bunch of bogus boring features, but also shows the Alexa ranking of any site you're looking at. Alexa ranking isn't some egghead hooty-tooty algorithm about the importance of a page -- it's a ranking on the Big Board of who's the most visited site in the world and who doesn't matter in the slightest. So, I can see that Flickr is ranked #41 today, le Sans Fil is #347,296, and wikiHow is #2248. Alexa is a lot more coarse-grained than Google's toolbar -- it measures the traffic to a given second-level-domain, not a particular URL. So any LiveJournal or Blogger site gets LJ's entire Alexa ranking -- not very useful. Still, I've taken to thinking almost entirely in terms of PageRank and Alexa rank. It's unreasonable. Every site I go to, my eye drifts up to my toolbars to confirm my idea about it. Do I really like Achewood? Sure, but it's only got an Alexa rank around 20,000... Doc Searls's blog has a PageRank of 8, which is really impressive; but... my page for 22 Prairial (Journal/22 Prairial CCXIV) apparently has the same PageRank. Holy moly! Why could that be? (I think it has something to do with the wp:Law of 22 Prairial, but I could be wrong.) Really, putting these numbers on everything is ruining the Web for me. It's turning me into a total jerk, too. I kinda want to go back to the time when I just found stuff on the Web and didn't think, "How do I measure up?" Thankfully the Yahoo Toolbar doesn't include anything remotely like rankings... just as well. Of course, that's the reason I uninstalled it, too. I can find my way to Yahoo! Canada Greetings just fine on my own if I need to. tags: google a9 alexa toolbar pagerank

One of my former teachers told a reporter from the local paper that I had just gotten back from Israel and he called me to set up an interview to get my views on the recent conflict. I met with him monday, and the story is now in print. I haven’t seen where in the physical paper it is, but it’s position on the webpage suggests it’s a front-page story. It’s available online here: The Pilot: War Zone: Local Residents Flee Mideast.

One of these days I'd like to understand why my life goes in waves. This round was contacts from Vancouver.

Katy and Ryan were here this weekend.
Frances just called to tell me about working at an Ottawa church this fall.
I sat down at my email and Mulan emailed me.
Ted, Tam, Heather and Rachel emailed to arrange visiting at the end of August.
Sandy (Angie's mom) is coming here on Saturday and will house-sit through Opus.
Jamu was in Montreal last week (even though I wasn't).
Heather (Angie's sister) was here the week before.

Weird. I don't mind at all, though. =) I miss all y'all!

So I’m planning a new box for college, and have come up with a tentative parts list. I am soliciting any advice and comments that people would be willing to offer. CPU: AMD Athlon 64 X2 3800+ Manchester 2000MHz HT 2 x 512KB L2 Cache Socket 939 Dual Core Processor - Retail
http://www.newegg.com/Product/Product.asp?Item=N82E16819103562
$169 (This was $289 2 days ago when I was pricing it - sweet) Motherboard: ASUS A8N5X Socket 939 NVIDIA nForce4 ATX AMD Motherboard - Retail
http://www.newegg.com/Product/Product.asp?Item=N82E16813131569
$84.99 RAM: CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail
http://www.newegg.com/Product/Product.asp?Item=N82E16820145587
$207.00 Video: eVGA 256-P2-N553-AX Geforce 7600GT CO 256MB 128-bit GDDR3 PCI Express x16 Video Card - Retail
http://www.newegg.com/Product/Product.asp?Item=N82E16814130283
$169.99 (Before $30 mail-in rebate) Hard Drives: I was originally planning on doing 3 Western Digital Caviar SE16 WD2500KS 250GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM (http://www.newegg.com/Product/Product.asp?Item=N82E16822144701, $82.99) in a RAID-5 (software) configuration but there is currently a sale on the Seagate Barracuda 7200.10 ST3320620AS (Perpendicular Recording Technology) 320GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM (http://www.newegg.com/Product/Product.asp?Item=N82E16822148140) for $99.99 and I might just order three today. Enclosure/Power: Antec LifeStyle SONATA II Piano Black Steel ATX Mid Tower Computer Case 450Watt SmartPower 2.0 ATX 12V V2.0 for AMD & Intel systems Power Supply - Retail
http://www.newegg.com/Product/Product.asp?Item=N82E16811129155
$104.99 (But that’s a one-day sale price, I think it’s like $109.99 normally, though this is the 2nd day in a row it’s had a one-day sale. Both before $15 mail-in rebate). Optical Drive: Reusing my Sony DVD Burner from my current box. Not including shipping or rebates, the total price comes out as: $1035.94 Advice, comments, etc. are most definitely welcome as I don’t follow the hardware market.