For a couple of weeks, one of the harddrives in my server has started to behave strangely. The RAID controller has started first reporting some drive timeouts (daily, when the drive runs a short or long smart test), and now it even kicks it out of the (RAID1) array. And yet, during normal operation, there's no read error or write error that I can trigger, and there are no reallocated sectors (according to smart). The only time when I can reproduce the error is:

• the drive is running a SMART short or long self-test
• a SMART query for the drive is being done (e.g. -c, -l selftest, etc.)

What happens when both these conditions are met? The the SMART query is taking ages (as in ~20-30 seconds). This might cause some such high delays that the drive itself will report a timeout error (if any I/O takes place at the same time), and log an error in its internal error log. Another harddrive (identical brand): ~3s for SMART query during selftest and I/O load, no issues whatsoever. For this harddrive, smartctl -a reads a while, and then:

Error SMART Error Self-Test Log Read failed: Input/output error
Smartctl: SMART Self Test Log Read Failed
 
real    0m39.029s

The timeout above also has generated lots of errors in the drive's error log. I don't know how to read these properly, but in any case they don't seem too scary:

Error 144 occurred at disk power-on lifetime: 13552 hours (564 days + 16 hours)
  When the command that caused the error occurred, the device was doing SMART Offline or Self-test.
  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  10 51 00 80 ae 39 40
  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  61 80 70 80 ae 39 1c 08  41d+08:07:45.052  WRITE FPDMA QUEUED
  b0 d0 01 00 4f c2 00 08  41d+08:07:45.038  SMART READ DATA
  ec 00 01 00 00 00 00 08  41d+08:07:44.958  IDENTIFY DEVICE
  2f 00 01 10 00 00 00 08  41d+08:07:44.957  READ LOG EXT
  61 80 70 80 ae 39 1c 08  41d+08:07:37.960  WRITE FPDMA QUEUED

For some of the errors, all preceding commands are WRITE FPDMA QUEUED, but all are during a "SMART Offline or Self-test" phase. When a self-test is not being done, reading all the SMART data (smartctl -a) is very very quick, taking half a second. The only thing I can think of is that the drive's own area for storing SMART data is unhealthy, and reading it takes time, and a concurrent SMART test and I/O load makes it hard for the drive to do so. But again, I can't trigger any real I/O error, nor at the beginning of the drive neither at the end, so This also happens when the drives is connected to a plain SATA port, skipping the RAID controller, so it's not just the controller playing games on me. I'm really confused now. Given my previous experience, this drive will die, should already have died, and yet, no I/O errors, just some timeouts. Do I just need to wait a couple more weeks?

Debian and Ubuntu have a set of official membership roles that can be granted to regular contributors. Those roles come with rights that enable the contributors to do their work and to participate in the project governance (elections and other official decision-making processes). It s also a way for the distributions to acknowledge the work done: most contributors are proud of the status they reached. The membership structure plays an important role in the development of a distribution: it defines the kind of contributors that are welcome in the project, it sets expectations of the project towards its contributors and defines their rights. In the end, this shapes the project s ability to recruit new contributors to keep the project alive and kicking. This article introduces the existing statuses in Debian and Ubuntu, and defines the sometimes confusing jargon associated with them. The Debian Case Debian only has two types of official members: Debian Developers (DD) and Debian Maintainers (DM). The rights of the developers are codified in the Debian Constitution while those of the maintainers have been defined in a general resolution of 2007. The Debian Maintainer status is still mostly documented in a wiki page. The integration of this new status in Debian s official processes has been slow to come largely because it was introduced at that time without enough negotiation with the involved parties. Nowadays, it is preferred that people get the DM status before applying for DD. DM is a very limited role: maintainers can only upload packages that already have their name on them (either in the Maintainer or Uploaders field) and a specific flag (DM-Upload-Allowed: yes) that only Debian Developers can add. They have no other rights and limited access to Debian s resources. Besides those official roles, there are also maintainers of packages that have no official status within Debian except that they are listed in the Maintainer field of the package. They are doing the maintenance work but all uploads are done by a Debian Developer after verification of the work done (this is called sponsorship and is the only way to start with official packaging work). Once the DD trusts the maintainer, the developer will typically ask the maintainer to apply for DM status in order to be relieved from the sponsorship work. In the end, that makes three different kind of package maintainers and a lot of confusion when you discuss membership issues in particular when the New Maintainer process is the path that you follow to become a Debian Developer. Don t be fooled by the names when reading Debian s documentation! The Ubuntu Case Ubuntu had, from the start, an official Ubuntu Member status that includes all contributors: developers of course, but also documentation writers, artists, translators, etc. This status notably grants the right to vote in elections of the Community Council, the right to participate on Planet Ubuntu, and the @ubuntu.com email alias. For developers, the situation is more complicated: the wiki page lists no less than five different statuses. Initially, developers were split between Ubuntu Core Developers and the MOTU (Masters Of The Universe). The latter were responsible of the universe/multiverse sections of the archive while the former also had upload rights for the main/restricted sections. But, inspired by the Debian Maintainers concept and facing concrete problems in terms of archive management, they changed their infrastructure to offer more fine-grained control on package uploads. Ubuntu can now grant upload permissions on a package-per-package basis, but it can also delegate the right to grant upload permissions with the same granularity. This lead to the new Per-Package Uploader status which is simply an Ubuntu Member with upload rights on a limited set of packages where they have a specific expertise. The more generic Ubuntu Developers status now encompasses members of various development teams that have been delegated the right to manage upload permissions on a (usually large) package set (the current teams are Ubuntu Desktop, Mythubuntu, Kubuntu, and Edubuntu). Those teams can define their own policy to add new members provided they follow the basic rules defined by the Developer Membership Board (see this wiki page). Ubuntu Contributing Developer is an intermediate status for someone who is not yet ready for one of the other developer statuses but who has still shown enough commitment to be an Ubuntu Member. All those statuses can be obtained in a similar way: you prepare a wiki page listing your past contributions, you collect testimonials from existing members that you have worked with, you add yourself in the agenda of the next meeting of the board (or council) that grants the status that you seek, and you attend the meeting. The members of the board will decide whether you are ready for the status (or not) based on what you provided in the wiki, based on your answers during the meeting (and on a mailing-list for developers), and based on what others have to say about you. The most important boards are usually elected by the community while others are commonly appointed by the community council. Those governance bodies include Canonical employees but not as many as one would expect: two out of eight in the Developer Membership Board, two out of eight in the Community Council, but all six members of the Technical Board. The last figure, while not intended, is not surprising given the high expectations set on potential members of the technical board. Mark, as the founder, is the only person to have a permanent seat on both the Community Council and the Technical Board. Comparison of the Statuses Between Debian and Ubuntu The following table summarizes the rights given to each developer role in the two projects (Put the mouse over the abbreviations to know what they are referring to).

Rights	Debian		Ubuntu
	DM	DD	UM	PPU/UD	MOTU	UCD
Package maintenance via sponsorship	Y	N/A	Y	Y	Y	N/A
Official email alias	-	Y	Y	Y	Y	Y
Participate in votes for members	-	Y	Y	Y	Y	Y
Participate in votes for developers	-	Y	-	Y	Y	Y
Upload rights restricted to pre-approved packages	Y	-	-	Y	-	-
Upload rights restricted to a section of the archive	-	-	-	-	Y	-
Unlimited upload rights	-	Y	-	-	-	Y
Number of contributors (as of 2010-07-27)	117	904	462	27	85	63

Please note that the number of contributors are not 100% accurate for Ubuntu. A contributor can have multiple statuses (direct membership to a launchpad group) granted over time (while gaining experience). The problem has been mostly avoided by calculating differences between number of members of the various groups but it s not perfect and it can t be: some MOTU are also PPU for packages in main and it s legitimate (but I only counted them as MOTU and not as PPU). Another limitation is that members of some administrative teams are included indirectly in many teams and thus appear in the count while they should not. Anyway, this simple table makes it obvious that Ubuntu s structure offers a broader choice of statuses. They acknowledge the work of all contributors from the start while still giving the most critical rights only to those who have proven that they deserve them. Despite this difference, Debian still has a significant advantage in terms of number of developers. That number does not tell the whole story though: the Ubuntu contributors include many Canonical employees (e.g. 36 out of 63 core developers have a @canonical.com email registered on their launchpad account) that are likely to spend more time working on the distribution than the average Debian member. But even if comparing person-hours would be a challenging thought experiment, in practice it s of not much interest if both projects continue to cooperate and if more and more of the contributions flow in both directions. Debian is aware of the shortcomings of its structure. Changes to better accommodate non-packagers have been discussed several times already. The last efforts in that direction were unfortunately perceived as a solution ready to go rather than a proposal to be discussed, and the project got quickly buried by a general resolution (GR). Even if that resolution invited for further discussion and a new proposal, the truth is that when someone s initiative is corrected by way of GR, it usually kills any motivation to go forward. Possible Evolution? On the Ubuntu side, the infrastructural changes were completed recently and they don t expect any further change in the near future. They do plan, however, to expand usage of those new features so that more teams benefit from the possibility to control upload rights on packages that are relevant to them, and so that more individuals developers apply to become Per-Package Uploaders on packages that they know very well. On the Debian side, a recent discussion on the debian-project list brought back the topic of the bad terminology and it was agreed that the New Maintainer process should be renamed into something else ( New Developer process has been suggested). But Christoph Berg Debian Account Manager and hence heavily involved in the New Maintainer Team suggested that Debian would be better off implementing the long-awaited membership changes before trying to update all the documentation. It would certainly imply some more vocabulary updates. Later in the discussion, he confirmed that membership reform is on the top of the TODO list of the new maintainer team (just after the rewrite of the nm.debian.org website). What can be expected from this reform? The following answers are my own guesses based on my experience of Debian, but the project hasn t decided anything yet.

• First of all: a new status for contributors that are not packagers. The tricky part will be defining the process to follow and the rights granted.
• Changes to the technical implementation of the DM status. The current implementation does not allow to give upload rights to a single DM if two are listed in the Uploaders field of a package (and both might not have the same experience for that package). Furthermore, it suffers from annoying restrictions like the inability to upload new binary packages.
• A change of the Debian constitution to integrate those new statuses is almost unavoidable.
• Other more invasive changes have been proposed like replacing the NM process by a simple designation by other DD, but it s unlikely to happen. The NM process can already be greatly simplified by the application manager if the applicant can show good testimonials from other developers and if he has a track record of real contributions (e.g. as witnessed by changelog entries in Debian packages).

Almost two years have elapsed since the previous efforts in that direction, the new maintainer team has recruited new members and is in a general better shape. Hopefully, the next episode of this saga will have a better outcome.

This article was first published in Linux Weekly News. In a comment, Mark Shuttleworth tried to explain how the Ubuntu community is being setup.

10 comments Support my work

The DebConf registration team has the hard job of coordinating accommodation, food, and payments for hundreds of people.There are two major dates here: the sponsored registration deadline, and the reconfirmation deadline. There is also the date of opening registration, but that's usually more a matter of "how soon can we?" than "when should we?"The sponsored registration deadline (April 15 for DC10) is the deadline for requesting sponsored accommodation, food, or travel. This allows us to figure out how much we'll have to pay for attendees - attendees who register after this point should pay enough to cover their own costs.The reconfirmation deadline (June 10th for DC10) is used as a ping for attendees, particularly the sponsored ones. Attendees must reconfirm they will attend, so that we don't waste money reserving more food/rooms than we will have people using them. This deadline usually depends on the date we must give firm numbers to our accommodations.Historically, registration hasn't ever closed, anyone can come, just if you register too late you won't necessarily get any free stuff (and definitely no free food/accommodation). The deadline for food and accommodation themselves has been based on the logistics of the conference itself - DC9, when we ran out of physical rooms for accommodation, and DC10, when we had to give firm housing numbers to Columbia. Food has tended to be easier as caterers could take updates on shorter notice, usually the day before.DebConf8 began a system where attendees could pay a registration fee if they desired. This was encouraged, for example, for attendees whom could get this fee reimbursed by their company, but was not required nor encouraged of sponsored attendees. DC8-9 had two classes of fees: Professional, covering the actual costs, and Corporate, covering about double of professional and for companies or registrants whom could provide extra financial support for DebConf. DebConf10 added an extra layer of "pay per day" on top of this.It's up to the registration team to pin down attendee numbers as early as possible, as this is usually the biggest cost. DebConf provides food and accommodations to a huge number of people, but the numbers are constantly changing. It's hard to know how many people will drop out at the last minute. It's hard to know how many people will actually pay a corporate or professional fee, as opposed to mistakingly selecting it when they'd rather something else. Our registration system can't support all the little intricacies of how much an attendee will cost us, or pay us. Without knowing how many people will attend, how can we accurately get the right number of meals and right number of rooms? Sponsored people may always choose to not eat some meals with us, possibly saving us money there. How many people with travel sponsorship will actually request their full reimbursement?The registration system, Pentabarf, is used to track attendance numbers, but it isn't completely reliable. Not everyone in it will attend all days, and most people who aren't staying with us don't have accurate dates of attendance. Thus, it can only be used as estimates, not for all decisions.
The difficulties above are challenges for the registration and budgeting teams, but are not too bad - they just require thought, time, and answering many questions from attendees.Tags: debconf

Shereen El Feki gave an inspiring TED talk about Islamic youth culture [1]. She shows some interesting exerpts from the 4SHBAB TV network which is known as Islamic MTV , the music video from the US was of particularly high quality while I expect high quality videos to be made in the US I don t generally expect quality Islamic videos from the US (or anywhere else really). She also notes that the videos show a kinder gentler face of Islam . She contrasts that with a clip by Haifa Wehbe [2] a Lebanese pop star who appears to have a lot in common with Britney Spears (Shereen describes her as a pan-Arab pinup-girl ). She cites the comic The 99 which has Islamic super-heros who represent the 99 attributes of Allah. One thing that I found very interesting was that the 99 character Jemi is shown using what is obviously an OLPC. This seems to indicate some very positive trends for the interaction of Islamic culture with the European and American culture which is Christian and Atheist dominated. Kavita Ramdas gave an interesting TED talk about radical women embracing tradition [3]. She highlights a woman who teaches girls to read in Afghanistan based on the religious edict that every Muslim should read the Koran and a Croatian Lesbian choir that sings traditional folk songs. Naif al-Mutawa gave an interesting TED talk about the creation of The 99 [4]. He starts by comparing some of the characters in the Justice League of America to Christian traditions and then describes the back story behind his Islamic super-heroes. His major aim is to provide positive role models for Muslim children. Apparently a cross-over production involving characters from The 99 and the Justice League of America is being developed at the moment Wonder Woman wears clothing that is less revealing than usual though. It s worth noting that Naif is a practicing psychologist who s clients include victims of political torture, so he seems to have some insight into the problems that most people will never have.

• [1] http://www.ted.com/talks/lang/eng/shereen_el_feki_pop_culture_in_the_arab_world.html
• [2] http://www.haifafans.net/
• [3] http://www.ted.com/talks/kavita_ramdas_radical_women_embracing_tradition.html
• [4] http://www.ted.com/talks/naif_al_mutawa_superheroes_inspired_by_islam.html

I don't like repeating myself, but I'm very tempted to past my mini-review of the Roomba Vacuum Cleaner robot into this blog. Instead I will practise restraint and summerise:

• It works. It works well.
• It is a little noisy, but despite this it is great fun to watch.
• It takes a long time to clean a few rooms, due to the "random walk" it performs. Despite this it is still fun to watch and actually useful.
• Have I mentioned I grin like a child when it doesn't crash into things, and hums away past me on the floor?

250. Worth. Every. Penny. In more Debian-friendly news I've been fighting HTTP proxies today. I've noticed a lot of visitors to the various websites I host are logged as 127.0.0.1 - which is an irritation. My personal machine looks like this:

Internet -> Apache listening on *:80 -> thttpd on 127.0.0.1:xxxx

(This has been documented previously - primarily it is a security restriction. It means I can run per-UID web-servers.) I had previous added a patch to thttpd to honour the X-Forwarded-For: header - so that it would receive the correct remote address passed on from Apache. However the fact that so many visitors are logged as coming from 127.0.0.1 meant it wasn't working 100% correctly, and I wanted to understand why. Today I used ngrep to capture the incoming headers and the source of the problem became apparent:

skx:~# ngrep  -d lo  X-For ' port 1007'
..
T 127.0.0.1:41886 -> 127.0.0.1:1007 [AP]
  GET /about/ HTTP/1.1..Host: images.steve.org.uk..If-Modified-Since: Mon, 07
   Jun 2010 15:24:33 GMT..User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-U
  S; rv:1.9.1.10) Gecko/20100701 Iceweasel/3.5.10 (like Firefox/3.5.10)..Acce
  pt: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8..Accept
  -Language: en-us,en;q=0.5..Accept-Encoding: gzip,deflate..Accept-Charset: I
  SO-8859-1,utf-8;q=0.7,*;q=0.7..Referer: http://images.steve.org.uk/2009/11/
  20/img_0471.html..X-Forwarded-For: 127.0.0.1, 11.22.33.123..Cache-Control:
  max-age=0..X-Forwarded-Host: images.steve.org.uk..X-Forwarded-Server: image
  s.steve.org.uk..Connection: Keep-Alive....

I bolded the important input; just in case that didn't jump out it was:

X-Forwarded-For: 127.0.0.1, 11.22.33.123

My patch to thttpd was making it read the first address, rather than the second - which meant that requests were being logged as coming from 127.0.0.1 and avoiding my efforts to track sources. Now I understand the problem - The X-Forwarded-Host header is being tweaked by a proxy server, such as Squid, upstream of my server. For the moment I've updated the thttpd patch to read:

        else if ( strncasecmp( buf, "X-Forwarded-For:", 16 ) == 0 )
            char *tmp = NULL;
            /* Jump to the header-value  */
            cp = &buf[16];
            cp += strspn( cp, " \t" );
            /*
             * If the first change is a 127.0.0.1, then we'll
             * jump over it.  Cope with Squid, et al.
             */
            if (  ( tmp = strstr( cp, "127.0.0.1, " ) ) != NULL )
              cp = tmp + strlen( "127.0.0.1, " );
            /* Parse the IP */
           inet_aton( cp, &(hc->client_addr.sa_in.sin_addr) );
         

That's not perfect, but the alternative would be:

• Install a patched version of libapache2-mod-rpaf to add a X-HONEST-REMOTE-IP
• Update thttpd to use that header.

Or something equally hacky and security-by-obscurity-alike. Really I just want a simple way of always getting the correct remote IP. Shouldn't be so hard, should it? *pout*. ObQuote: "You don't mess with fate, Peanut. People die when they are meant to die. There's no discussion. There's no negotiation. When life's done, it's done." - Dead Like Me.

After a long time, I decided to write again. I decided to start with software I wrote for my own usage that could be usefull for other people. I'll start with python-dvdvideo, a DVD-Video reader written in Python 3, and the reference tool dvd-video-backup-image, a generic DVD-Video dumper. Lets see, if this blog will see more postings in the future.

A number of things have happened in and around forked-daapd in the last month or so, so let s review the new features and the plan for Squeeze. On the features front, Kai Elwert has done a lot of work since July to implement missing features at the DAAP and DACP level to get better support for Remote:

• playlists can now be played from Remote;
• listings are sorted in a way that s closer to what you get from iTunes;
• sort headers are now generated when requested so Remote can display the A-Z quick access list.

Also, forked-daapd can now remember which speakers were selected before shutdown and attempts to automatically reselect these speakers the next time around. Speakers that were selected will be reselected if they appear at most 5 minutes after startup and the player isn t running at that time. On the Squeeze front, a viable snapshot of forked-daapd will release with Squeeze. This snapshot doesn t have all the Remote improvements listed above, although it s got the playlist support, can remember the speaker selection and includes bug fixes. My plan is to provide backports through backports.org once Squeeze is released and for as long as possible after that. There s always more in the works, so stay tuned =)

mksh was merged into Android (both AOSP and Google s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread. mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling. There s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT s LFN or ISO 9660 s Joliet, or one that only allows Unicode (canonically decomposed u out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance s small of getting that in the BMP). Still. Oh well, the torrents. I ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe f.scarywater.net will return. Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable the device sadly doesn t have a hardware keyboard, but it comes with SiRFstar GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. Any chance to get MirWorldDomination onto that device as well?

If you find that running Windows makes a GRUB 2-based system unbootable (Debian bug, Ubuntu bug), then I'd like to hear from you. This is a bug in which some proprietary Windows-based software overwrites particular sectors in the gap between the master boot record and the first partition, sometimes called the "embedding area". GRUB Legacy and GRUB 2 both normally use this part of the disk to store one of their key components: GRUB Legacy calls this component Stage 1.5, while GRUB 2 calls it the core image (comparison). However, Stage 1.5 is less useful than the core image (for example, the latter provides a rescue shell which can be used to recover from some problems), and is therefore rather smaller: somewhere around 10KB vs. 24KB for the common case of ext[234] on plain block devices. It seems that the Windows-based software writes to a sector which is after the end of Stage 1.5, but before the end of the core image. This is why the problem appears to be new with GRUB 2. At least some occurrences of this are with software which writes a signature to the embedding area which hangs around even after uninstallation (even with one of those tools that tracks everything the installation process did and reverses it, I gather), so that you cannot uninstall and reinstall the application to defeat a trial period. This seems like a fine example of an antifeature, especially given its destructive consequences for free software, and is in general a poor piece of engineering; what happens if multiple such programs want to use the same sector, I wonder? They clearly aren't doing much checking that the sector is unused, not that that's really possible anyway. While I do not normally think that GRUB should go to any great lengths to accommodate proprietary software, this is a case where we need to defend ourselves against the predatory practices of some companies making us look bad: a relatively small number of people do enough detective work to realise that it's the fault of a particular Windows application, but many more simply blame our operating system because it won't start any more. I believe that it may be possible to assemble a collection of signatures of such software, and arrange to avoid the disk sectors they have stolen. Indeed, I have a first draft of the necessary code. This is not a particularly pleasant solution, but it seems to be the most practical way around the problem; I'm hoping that several of the programs at fault are using common "licence manager" code or something like that, so that we can address most of the problems with a relatively small number of signatures. In order to do this, I need to hear from as many people as possible who are affected by this problem. If you suffer from this problem, then please do the following:

• Save the output of fdisk -lu to a file. In this output, take note of the start sector of the first partition (usually 63, but might also be 2048 on recent installations, or occasionally something else). If this is something other than 63, then replace 63 in the following items with your number.
• Save the contents of the embedding area to a file (replace /dev/sda with your disk device if it's something else): dd if=/dev/sda of=sda.1 count=63
• Do whatever you do to make GRUB unbootable (presumably starting Windows), then boot into a recovery environment. Before you reinstall GRUB, save the new contents of the embedding area to a different file: dd if=/dev/sda of=sda.2 count=63
• Follow up to either the Debian or the Ubuntu bug with these three files (the output of fdisk -lu, and the embedding area before and after making GRUB unbootable.

I hope that this will help me to assemble enough information to fix this bug at least for most people, and of course if you provide this information then I can make sure to fix your particular version of this problem. Thanks in advance!

My last post discussed the year-to-year budgeting of DebConf. This post discusses what makes budgeting difficult within one particular DebConf. Since every DebConf is unique in its arrangements, the particular difficulties always vary year-to-year. For example, DebConf10 was easy in that Software in the Public Interest was in the same country so that we had easy access to bank accounts, but more difficult venue payment logistics. I'll use DebConf10 as an example below.
For DebConf10, we get our venue "for free" thanks to the support of the Columbia Computer Science department. However, we had to pay for guard costs after building hours, and custodial costs. For custodial costs at least, we had to go through many layers of bureaucracy, and people kept insisting it would be more than they quoted. (It ended up being about half, since DebConf attendees were so responsible!)DebConf sponsors many attendees. This means that all their food and accommodation costs are paid by us. We have sponsored registration and reconfirmation deadlines to set a maximum of the number of people we have to sponsor, but people are continually dropping out and it's hard to know just how many people to reserve for. Sponsored people won't always eat every meal with us, and usually won't decide until the day before - so figuring out food amounts is hard, too. Furthermore, we have unsponsored people who will want food and accommodations (paying themselves) which we must somehow plan for.Different venues operate on different payment rules. This year, CU Housing was fairly strict on asking for attendee counts and room assignments two weeks in advance. Then, as time gets closer, they got more flexible, and in the end told us that they will, in fact, only charge based on number of people actually attending - a huge win for us. Similarly with CU Dining, we had to go through great lengths to make sure that we would pay only for meals we ate. At CU, food had to be paid in advance, but accommodation and other costs are paid afterwards.We have to guess how much will be spent on miscellaneous supplies once DebConf starts. We have to then decide how much to spend on extras such as the day trip or conference dinner, and how to sponsor those, if possible.Debian (and thus DebConf) has money held in trust in many currencies in different accounts. Different sponsors will pay to different currencies, depending on what is convenient for them. This means that we need to consider fluctuating currency conversion rates, and decide if and when to transfer money. In general, DebConf hasn't transfer money just to make sure the DebConf-tagged money gets spent first. Some surpluses are best left in other countries to help pay future travel sponsorship since it is expensive to move money out of the country. Some conferences may result in a net movement of money from USA to Europe (more donors in one, more expenses in the other), some the other way around.Travel sponsorship is in the form of reimbursement. These can take many months to have the documentation received from attendees and the reimbursements all sent out, and thus finalizing the budget and surplus for next year.The overall principle we've tried to stick to is plan for a balanced budget in the worst case scenario, and then be happy when it ends up better than that.
At the DebConf&Debian BOF, someone mentioned how budgeting for any (free software) conference is hard. I think that's certainly true. Some of the registration fluctuations could be made easier by being less flexible, but DebConf wants to serve our attendees even if it is a bit of extra work for us. Each DebConf will have its own ways budgeting is difficult, but things have always work out, and usually with a bigger surplus than we were expecting.Tags: debconf

Urban Honking has an insightful article about the Arduino and suggests that it is one of the most important factors for the development of the computer industry in the near future [1]. It compares the Arduino to the Altair. Wired has an interesting article about a company that provides a satellite kit and a launch into low Earth orbit for $8000 [2]. Arduino in space? Linux Journal has an interesting article by David Rowe about the Mesh Potato which is a Wifi mesh router that also runs VOIP [3]. One particularly interesting aspect of this article is the explanation of the way they designed and tested it. Susan Shaw gave an informative TED talk about the toxic effects of the attempts to clean up the oil spill in the Gulf of Mexico [4]. It seems that trying to disperse the oil just makes it worse, and the chemical companies are refusing to disclose the chemicals that are being used. The New York Times has an interesting article by David Leonhardt about the value of pre-school teachers [5]. Some research on the difference that good teachers can make in economic terms suggests that the make an economic difference to the children to the value of $320,000 per annum (IE a class of 16 children who were taught for a year would on average each receive a benefit of $20,000 over their lifetime). Also there are social benefits which aren t counted by that study. While I can t imagine pre-school teachers getting paid $320,000 any time soon, it does seem obvious that good teachers deserve significantly better pay. Of course one problem is how to determine which teachers are good, better test results are not a reliable indication. Paul Krugman describes America as being on the unlit, unpaved road to nowhere due to the policies of saving money by cutting funding for schools, street-lights, and roads [6]. The Chive has an amusing post about how to quit a bad job [7]. It would be good if someone really did this, I m sure that there are enough creative people who don t like their job. Hell Pizza in New Zealand published a zombie themed choose your own adventure on Youtube [8]. Unfortunately the options to choose the next segment don t work on HTML5 with Chromium so if you don t have flash you miss out. AskThePilot.com has an informative essay about airline security written by a commercial pilot [9]. The anecdote about the pilot not being allowed to take the type of knife that is issued to first and business class passengers is rather amusing. FredOnEverything.net has an interesting analysis of Wikileaks and why the Pentagon and Fox News hate it [10]. Fred is a very skillful writer, while he s not the first person to say some of these things he may have said it best. The Wikipedia page on Borosilicate glass (which is best known under the trademark Pyrex) is really interesting [11]. Borsilicate glass was formerly known as Duran and it s main characteristic that makes it suitable for lab use is resistance to Thermal Shock, but it s also harder and has a higher melting point. Apparently you can get Pyrex drinking glasses, I want some! Eben Moglen gave an interesting talk Freedom in the Cloud about the development of free servers to manage personal data and replace Facebook etc (among many other things) [12]. The Debian Wiki has an articla about designing such a system [13]. The APNIC published an interesting paper on IPv4 background radiation [14]. Apparently some /24 s receive so much random traffic (from broken applications and viruses) that they can t be delegated. IPv6 will solve this problem by making it infeasible to scan all IP addresses. Also it s interesting to note the excessive amounts of traffic to 1.0.168.192 which is from applications too broken to correctly send data to 192.168.0.1 which have been installed by sysadmins who are too incompetent to watch what is being sent out of their network.

• [1] http://www.urbanhonking.com/ideasfordozens/2009/05/why_the_arduino_matters.html
• [2] http://www.wired.com/gadgetlab/2010/07/tubesat-personal-satellite/
• [3] http://www.linuxjournal.com/article/10604
• [4] http://www.ted.com/talks/susan_shaw_the_oil_spill_s_toxic_trade_off.html
• [5] http://www.nytimes.com/2010/07/28/business/economy/28leonhardt.html
• [6] http://www.nytimes.com/2010/08/09/opinion/09krugman.html
• [7] http://thechive.com/2010/08/10/girl-quits-her-job-on-dry-erase-board-emails-entire-office-33-photos/
• [8] http://www.youtube.com/watch?v=9p1yBlV7Ges
• [9] http://www.askthepilot.com/essays-and-stories/terrorism-tweezers-and-terminal-madness-an-essay-on-security/
• [10] http://www.fredoneverything.net/Wikileaks.shtml
• [11] http://en.wikipedia.org/wiki/Borosilicate_glass
• [12] http://www.isoc-ny.org/?p=1338
• [13] http://wiki.debian.org/FreedomBox
• [14] http://meetings.apnic.net/__data/assets/pdf_file/0007/23398/ipv4-background-radiation.pdf

OCaml compiles native executables in static mode. It allows to have a minimal set of dependencies when delivering an executable. It has also disadvantages like the size of the executable and problems arising when considering libraries update -- but this is another topic. There is still one strong dependency that you should not forget when you want to deliver a product for most of the Linux distributions: dependency on the glibc version. Trying to run OASIS compiled with Debian Lenny, on CentOS 5.5:

$ OASIS
.../OASIS: /lib64/libc.so.6: version  GLIBC_2.7' not found (required by .../OASIS)

So when compiling for delivery, one should choose the oldest distribution he targets. In my case, I choose CentOS 5 which comes with glibc v2.5. I usually choose Debian stable at the moment of writing Debian Lenny. But for now, the Debian Lenny's glibc is newer (v2.7) than the one coming from the CentOS 5.5 stable release. CentOS is a Red Hat like Linux distribution. I use a Debian Lenny amd64 host system and I decided to setup a chroot of CentOS 5 i386 and amd64. I also setup schroot to use my CentOS chroot. CentOS 5 amd64 setup First of all we use rinse, which can setup a RPM based distribution in a chroot. The version v1.3 shipped with Debian Lenny has some bugs: it doesn't install nss and other mandatory packages. So I downloaded v1.7 directly from Debian Sid. There is no dependencies problems and the package is arch:all, so it is straightforward to install:

$ wget http://ftp.de.debian.org/debian/pool/main/r/rinse/rinse_1.7-1_all.deb # Replace ftp.de.debian.org by your preferred Debian mirror
$ dpkg -i rinse_1.7-1_all.deb

Then I create the chroot directory and launch rinse:

$ mkdir /srv/chroot/centos5-amd64
$ rinse --arch amd64 --distribution centos-5 --directory /srv/chroot/centos5-amd64 # N.B. you must use --arch, the default is i386

Once installation is complete, you can add an entry for this distribution in /etc/schroot/schroot.conf:

[centos5-amd64]
description=Centos 5 (amd64)
location=/srv/chroot/centos5-amd64
priority=3
users=XXX
groups=
root-groups=root
type=directory
run-setup-scripts=true
run-exec-scripts=true

Replace XXX by your login. If you try to login directly, you will get warnings:

$ schroot -c centos5-i386
I : [chroot centos5-i386-a952de23-7f4b-4bae-a9b9-752ecee4a185] Ex cution de l'interpr teur de commandes initial :  /bin/bash 
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied

This is a bit misleading because the real problem is that nothing is created in /dev/. CentOS delegates creating char/block devices to udev. You have two solutions to solve this issue:

• login and call MAKEDEV to create missing devices:

$ MAKEDEV random
$ MAKEDEV console
$ MAKEDEV zero
$ MAKEDEV null
$ MAKEDEV stdout
$ MAKEDEV stdin
$ MAKEDEV stderr

• use an already setup Debian chroot to copy the missing devices:

$ rsync -av /srv/chroot/lenny-amd64/dev/* /srv/chroot/centos5-amd64/dev/

That's it, you now have a functional chrooted CentOS 5 environment:

$ schroot -c centos5-amd64 cat /etc/redhat-release
I : [chroot centos5-amd64-b9bae264-285b-4d17-a046-13386736cecd] Ex cution de la commande :   cat /etc/redhat-release  
CentOS release 5.5 (Final)

CentOS 5 i386 setup To setup an i386 environment, we follow almost the same scheme, except we need to fix a bug in rinse v1.7: we need to call linux32 before executing chroot. The problem is that the first stage installation of rinse install an i386/686 environment but as soon as you call chroot yum install ..., it will guess that the system is amd64 and will install missing packages. See the Debian bug report and the example patch attached to correct this behavior. WARNING: this patch is just an example, you can apply it for creating CentOS i386 chroot on Lenny amd64 host but you should remove the patch as soon as the installation is complete.

$ mkdir /srv/chroot/centos5-i386/
$ rinse --arch i386 --distribution centos-5 --directory /srv/chroot/centos5-i386 # With /usr/lib/rinse/centos-5/post-install.sh patched 
$ rsync -av /srv/chroot/lenny-i386/dev/* /srv/chroot/centos5-i386/dev/

Add this distribution to /etc/schroot/schroot.conf:

[centos5-i386]
description=Centos 5 (i386)
location=/srv/chroot/centos5-i386
priority=3
users=XXX
groups=
root-groups=root
type=directory
run-setup-scripts=true
run-exec-scripts=true
personality=linux32

You now have a schroot of CentOS 5 i386:

$ schroot -c centos5-i386 cat /etc/redhat-release
I : [chroot centos5-i386-9acafa91-9862-4488-aaef-4ab2a482771e] Ex cution de la commande :  cat /etc/redhat-release 
CentOS release 5.5 (Final)

Happy schroot hacking!

Apparently this statements also holds true for frozen yogurt. An Austrian A1 spokes person has confirmed that the HTC Magic will not receive a 2.1 or 2.2 Android update. One can only wonder about the sugar HTC puts on top of regular Android that hinders themself to update their products. The Austrian A1 carrier sells you the device for a 18 month contract, but actively only supported it for 6 month. I must revise my previous positive review of the HTC Magic. Some HTC speaker had promised earlier this year an upgrade to at least 2.1: HTC Magic 2.1 upgrade. A SFR speaker had promised an update to current Android: HTC Magic and Nexus One 2.2 upgrade. The Webkit Android Browser can be easily tricked into leaking your user and passwords: Android Luecke. Beside the obvious that as a user one shouldn't give out to much data to untrusted third party this opens lots of Google accounts for criminal activity. The inability of the carrier to provide a secure and uptodate device is massively deceiving and certainly not appropriate handling of their defects liability.

On the other hand I have managed to take a family holiday in sunny Brittany...

One of the most debated points at the DebConf&Debian BOF was money management. Let's talk about how it works in an ideal world, and then how it tends to work in practice. This post only talks about year-to-year management, I hope for another post talking about the complexities of budgeting within a single year.In an ideal world, DebConf fundraises all the money it needs to go on. There is a DebConf sub-team for fundraising, since Debian doesn't have its own fundraising team. This team make brochures and solicits sponsors - there will be another post about this by someone on the fundraising team.We do our best to raise enough money to cover the conference, but it's long and hard work. In the months leading up to the conference, we are almost always running short. However, in the end, somehow, it comes through by a combination of cutting costs and continued fundraising work.The money is held by various nonprofits like SPI in the USA and FFIS in Europe. They hold it in trust for Debian. I think most people on the DebConf team agree that this money belongs to Debian, but is earmarked for DebConf since it was fundraised for DebConf. In the past, the DPL has not been asked for permission to use DebConf-fundraised money to put on a DebConf, since that's always been kind of obvious that it's for DebConf.Planning how much it will cost is hard. Venue costs aren't always known in advance, and we don't know our number of attendees until late. There are many uncertainties, which will be described later. These questions are hard to answer, but not impossible. Most of they time things work out in our favor.At the end of the conference, we tally everything up. Usually(=always, as far as I know), there is a surplus. This surplus is important, since DebConfN+1 usually needs a little seed money to get started before the new funds start rolling in. Historically, this seed money has been used for DebConfN+1, since it was raised earmarked for DebConf.In practice, it works out mostly like this. We keep track of the money, and try to stay within our budget, and cut things when we need to.For the last two years (coinciding with the recession), we have expected a deficit in the months and weeks leading up to DebConf. In these cases, we asked the DPL for some money to be given from Debian general funds for DebConf, and we provide background on our fundraising and budget status for use in making that decision. Both times, the Debian funds went to ensure that we could get travel sponsorship out. Debian general funds are not used without DPL approval.Also, keeping all the paperwork straight is a hard task. We have limited people, and the budget-trackers are always doing many other jobs, too. Personally, I'm not convinced that we always know how much the exact post-DebConf surplus is and where it is (SPI, FFIS, ...) each year. It sort of gets morphed into the general Debian money, and then we know "we have (about) XXX surplus" and then use that amount as our starting seed for the next year, taken from wherever is convenient. If Debian gives money to help make DebConf happen, it's usually as a loan. However, this loan isn't tracked carefully year to year. Someone looked up numbers during the BOF and reported that Debian has given/loaned 50,000 USD to DebConf over the years, but it's hard to know just how accurate that is. For comparison, the total DebConf budget over those years has been close to 1,000,000 USD, as best I can figure.We've tried various types of accounting software with mixed success. They are a lot of overhead, and we usually revert to keeping track of things in spreadsheets.
Our goals for budgeting (being self-sustaining) are good and well agreed on, but the implementation is hard.However, it's a place where it is easy for dedicated volunteers to get involved. If you wanted to help to make this better, you could do a lot of good by watching #debconf-team and lists, making careful records of our cash flow and where it comes from and goes to. All money we spend is announced and documented on the lists, so the information is there. To really do a complete job, this person would need to remain motivated in the months after DebConf and wrap up all of the final payments. Preferably, we'd have the same person year after year.There are many more suggestions that both others and I could make, but I'll save those for later.Tags: debconf

In a great post a few days ago, Jono asks why you are doing free software and reminds us to remember it. It s quite easy to lose sight of our goals and motivations while working on the day-to-day tasks. This reminds me of an interesting point made in the course I m currently studying on Applying Our Co-operative Values and Principles. As well as our shared values and principles, each co-op should have a statement of its goals. For our co-op, that s to provide computer-related services . For the Co-operative Group, it s currently inspiring young people; tackling global poverty; and combatting climate change it s a big co-op, so it has big goals. But while goals are a shared commitment, your motivation for them is often a personal thing. Often it will be rooted in a negative, a dissatisfaction about how the world was before you started, then making that into an excitement about what you could do, the new possibilities that are opened up. I think you can see this a little in Jono s motivation of how excited they were at exploring their new system and many of the comments there. They re positives and often the silver lining around some cloud. So, please, head over to Jono s page and share your motivations. Then pop back to my site and tell me your goals (or how I ve misinterpreted this ) in a comment.

I was finding myself to be running the same povray command lines over and over again. And then ffmpeg2theora. And occasionally forget to update one or two .png files in the mean time, because I was trying and trying and trying some stuff all the time, and forgot that one frame in the final render. That's silly; that's why we have make. So I wrote me a makefile.

START := $(shell grep 'Initial_Frame' $(INIFILE)   cut -d'=' -f 2)
STOP := $(shell grep 'Final_Frame' $(INIFILE)   cut -d'=' -f 2)
NUMBERS := $(shell seq -w $(START) $(STOP) )
POVFILE := $(shell grep 'Input_File_Name' $(INIFILE)   cut -d'"' -f 2)
ifeq ($(OUTPUT),)
OUTPUT := $(basename $(shell grep 'Output_File_Name' $(INIFILE)   cut -d'"' -f 2))
endif
ifeq ($(OUTPUT),)
OUTPUT := $(basename $(INIFILE))_$(basename $(POVFILE))
endif
PNGFILES := $(foreach nr, $(NUMBERS), $(OUTPUT)$(nr).png)
LEN := $(shell echo "$(STOP)"   wc -m)
$(OUTPUT).ogv : PATTERN := $(OUTPUT)%$(LEN)d
$(OUTPUT).ogv : $(PNGFILES)
	ffmpeg2theora -f image2 -i '$(PATTERN)'.png -o $@
$(OUTPUT)%.png : ONAME := $(OUTPUT)
$(OUTPUT)%.png : INAME := $(INIFILE)
$(OUTPUT)%.png: $(POVFILE) $(INIFILE)
	povray $(INAME) -D +SF'$*' +EF'$*' +O$(ONAME)
ifeq ($(CLEANFILES),)
clean:
	$(RM) $(CLEANFILES)
endif
CLEANFILES := $(CLEANFILES) $(PNGFILES)

The first part, to just write the makefile so that it would work, was easy. The second bit, write it so that you can include it in the same Makefile several times and have it generate patterns for the files you need, without having them conflict with eachother, was... slightly harder. But not impossible, apparently, with GNU make. Use like this:

INIFILE=foo.ini
include povanim.mk
INIFILE=bar.ini
OUTPUT=bar
include povanim.mk

with povanim.mk containing the above Makefile Now just run 'make' or 'make bar.ogv', wait for the rendering and encoding to complete, and you have an animation film. You can even use make's '-j' option to speed up processing by using multiple processors. Which will no longer be useful when povray 3.7 comes out (since that uses multithreading to use all cores with a single povray instance, anyway), but it is for now.

Recently I was surprised to find that many of the rules I took for granted when dealing with an OpenPGP keyring, and the other activities associated with a web of trust, were new to some people I spoke to. These rules are born from a mixture of experience, accepted good practice, and wider reading, with a small dose of paranoia thrown in for good measure.

• Protect your secret keyring with file permissions. For GPG, your ~/.gnupg directory should not be readable by anybody else, including root (even though he could break in if he wanted). Mine is octal 700, and the secret keyring itself is 600.
• Create a revocation certificate, and keep online and offline copies (gpg --gen-revoke <keyid>). The revocation certificate is published to key servers by you in the event that you detect key compromise or loss. It s important that this is also well-protected, because the secret key is not required to upload it. Print a copy in bold and on good quality paper, so it will last a long time, and preferably keep them both off-site in a trusted location. Typing all those characters back in is a bore, but if your online copy fails it s better than being impersonated.
• Use a good pass-phrase, not a password. A pass-phrase is actually easier to remember than a string of random characters, without risk. It can be long enough to make brute-force attacks pointless, and if you re using the key a lot and can t be bothered to type it all the time, gpg-agent is a handy and safe tool. Your login password is not an acceptable pass-phrase for your secret key!
• Regularly refresh your public keyring from a well-connected keyserver. I detailed this in an earlier post; it s a case of calling gpg --refresh-keys once a week or so. This is the first line of defence against other people s keys being compromised, since without it you ll never know that your messages to them are now vulnerable. A well-connected server, like those in pool.sks-keyservers.net, matters since not all servers synchronise predictably.
• Rotate encrypting sub-keys regularly. This reduces the effect of a compromised key: if you rotate keys annually, a compromise only opens one year of mail. Small comfort, but every little helps. Don t delete the old keys though, just revoke them otherwise you won t be able to open mail from those senders who don t refresh their public keyrings.
• Keep your key offline and make a separated signing sub-key for day-to-day use. There isn t room to go into detail here, but you can export a signing sub-key with a relationship to your primary key material, so if it is compromised you can revoke it without losing all those carefully-collected signatures. The same goes for encrypting sub-keys. Adrian von Bidder has a guide to this, but he warns that it should be taken with a pinch of salt, so this isn t so much a rule as an interesting idea. I have not experimented with this yet.

And some rules about signing and signatures:

• Don t place too much store by signing parties. Parties are great for rapidly building a web of trust, but a part of that trust is to do with how the other party checks identities. I don t believe it s possible to establish that with just a few minutes chat and an identity check, so I never make a sig2 or sig3 on a key I met at a party. I m usually happy to do that later when I have an established relationship with the key holder, of course.
• Never upload your own signature to a key server. Apart from the fact that the key holder might not want their key published, you also have no way to prove that the key holder has access to the email address on their UID. It s much better to send the signature to them encrypted, which demonstrates that they the both key and can read mail at the address (the caff program is ideal for this).
• Give personal trust (the trustdb) carefully. Getting that message Good signature from in your mail client is really nice, but not at the expense of giving away trust where it is not deserved. In the same vein as not placing too much trust at signing parties, you should avoid the temptation to over-trust keys in your personal trustdb. Reputation on a mailing list is absolutely no substitute for observing the party taking care over their verifications when doing face-to-face signings.

Do you have other rules or guidelines that you use when dealing with keys and signatures?

A discussion the other night suggested that I m quite unusual among geeks in doing vanishingly little customisation of my system most of my systems have something very close to their default configuration, individual programs are usually at most a very few tweaks away from their upstream settings. For Emacs all I tend to do is install revbufs, set Linux mode by default for C and turn on syntax highlighting and line number mode (sometimes the last two only get done when I start Emacs). For vim it s just configuring a 72 column word wrap if my main use for vi on the system will be writing e-mail. For GNOME it s putting a terminal launcher in one of the panels and setting focus follows mouse. All very trivial stuff done in five minutes. The theory here is partly that I just can t be bothered to spend more time on tweaking things (and the subsequent carrying around of those tweaks) and partly that it ensures that I m not confused when I end up using a new system or someone else s for whatever reason. I find it somewhat odd that this would be something you d want to spend any time on.

DebConf continued until Saturday, but Friday the 6th was my last day as I left New York that evening. I m a bit late in getting this summary written up. Making Debian Rule, Again (Margarita Manterola) Marga took a bold look at the challenges facing Debian today. She says that Debian is perceived to be less innovative, out of date, difficult to use, and shrinking as a community. She called out Ubuntu as the elephant in the room , which is taking away from Debian. She insists that she is not opposed to Ubuntu, but that nonetheless Ubuntu is to some extent displacing Debian as a focal point for newcomers (both users and contributors). Marga points out that Debian s work is still meaningful, because many users still prefer Debian, and it is perceived to be of higher quality, as well as being the essential basis for derivatives like Ubuntu. She conducted a survey (about 40 respondents) to ask what Debian s problems are, and grouped them into categories like motivation and communication (tied for the #1 spot), visibility (#3, meaning public awareness and perception of Debian) and so on. She went on to make some suggestions about how to address these problems. On the topic of communication, she proposed changing Debian culture by:

• Spreading positive messages, celebrating success
• Thanking contributors for their work
• Avoiding escalation by staying away from email and IRC when angry
• Treating every contributor with respect, no matter how wrong they are

This stimulated a lot of discussion, and most of the remaining time was taken up by comments from the audience. The video has been published, and offers a lot of insight into how Debian developers perceive each other and the project. She also made suggestions for the problems of visibility and motivation. These are crucial issues for Debian devotees to be considering, and I applaud Marga for her fortitude in drawing attention to them. This session was one of the highlights of this DebConf, and catalyzed a lot of discussion of vital issues in Debian. Following her talk, there was a further discussion in the hallway which included many of the people who commented during the session, mostly about how to deal with problematic behavior in Debian. Although I agreed with much of what was said, I found it a bit painful to watch, because (ironically) this discussion displayed several of the characteristic people problems that Debian seems to have:

• Many people had opinions, and although they agreed on many things, agreement was rarely expressed openly. Sometimes it helps a lot to simply say I agree with you and leave it at that. Lending support, rather than adding a new voice, helps to build consensus.
• People waited for their turn to talk rather than listening to the person speaking, so the discussion didn t build momentum toward a conclusion.
• The conversation got louder and more dense over time, making it difficult to enter. It wasn t argumentative; it was simply loud and fast-paced. This drowned out people who weren t as vocal or willful.
• Even where agreement was apparent, there was often no clear action agreed. No one had responsibility for changing the situation.

These same patterns are easily observed on Debian mailing lists for the past 10+ years. I exhibited them myself when I was active on these lists. This kind of cultural norm, once established, is difficult to intentionally change. It requires a fairly radical approach, which will inevitably mean coping with loss. In the case of a community, this can mean losing volunteer contributors cannot let go of this norm, and that is an emotionally difficult experience. However, it is nonetheless necessary to move forward, and I think that Debian as a community is capable of moving beyond it. Juxtaposition Given my history with both Debian and Ubuntu, I couldn t help but take a comparative view of some of this. These problems are not new to Debian, and indeed they inspired many of the key decisions we made when founding the Ubuntu project in 2004. We particularly wanted to foster a culture which was supportive, encouraging and welcoming to potential contributors, something Debian has struggled with. Ubuntu has been, quite deliberately, an experiment in finding solutions to problems such as these. We ve learned a lot from this experiment, and I ve always hoped that this would help to find solutions for Debian as well. Unfortunately, I don t think Debian has benefited from these Ubuntu experiments as much as we might have hoped. A common example of this is the Ubuntu Code of Conduct. The idea of a project code of conduct predates Ubuntu, of course, but we did help to popularize it within the free software community, and this is now a common (and successful) practice used by many free software projects. The idea of behavioral standards for Debian has been raised in various forms for years now, but never seems to get traction. Hearing people talk about it at DebConf, it sometimes seemed almost as if the idea was dismissed out of hand because it was too closely associated with Ubuntu. I learned from Marga s talk that Enrico Zini drafted a set of Debian Community Guidelines over four years ago in 2006. It is perhaps a bit longand structured, but is basically excellent. Enrico has done a great job of compiling best practices for participating in an open community project. However, his document seems to be purely informational, without any official standing in the Debian project, and Debian community leaders have hesitated to make it something more. Perhaps Ubuntu leaders (myself included) could have done more to nurture these ideas in Debian. At least in my experience, though, I found that my affiliation with Ubuntu almost immediately labeled me an outsider in Debian, even when I was still active as a developer, and this made it very difficult to make such proposals. Perhaps this is because Debian is proud of its independence, and does not want to be unduly influenced by external forces. Perhaps the initial growing pains of the Debian/Ubuntu relationship got in the way. Nonetheless, I think that Debian could be stronger by learning from Ubuntu, just as Ubuntu has learned so much from Debian. Closing thoughts I enjoyed this DebConf very much. This was the first DebConf to be hosted in the US, and there were many familiar faces that I hadn t seen in some time. Columbia University offered an excellent location, and the presentation content was thought-provoking. There seemed to be a positive attitude toward Ubuntu, which was very good to see. Although there is always more work to do, it feels like we re making progress in improving cooperation between Debian and Ubuntu. I was a bit sad to leave, but was fortunate enough to meet up with Debian folk during my subsequent stay in the Boston area as well. It felt good to reconnect with this circle of friends again, and I hope to see you again soon. Looking forward to next year s DebConf in Bosnia