GSoC 2010 is over for some time and I should write some summary how students projects ended up. The very short summary is that all five students were successful and their work got merged. Follows description of the project in no particular order. Thanks to Martynas Mickevicius you can now get charts out of various parts of phpMyAdmin. They are used on server status pages or you can get query results in form of several charts. It seems to work pretty well at least what I've tried so far. Ankit Gupta was working on Visual Query Builder for phpMyAdmin. Unfortunately this is only project which is not yet merged to master branch, mostly because some UI things were not yet finished. But hopefully it will be merged soon. Adnan Mughal was converting our schema export feature from PDF to support multiple formats. You can now get the schema as SVG, DIA or even Visio formats, some of them will require a little bit of tuning still, because the scaling is not perfect. Ninad Pundalik did a lot of work on AJAXifying phpMyAdmin. His changes touched quite a lot of places and there are still some rough edges (as you can see in our bug tracker), but this is definitely welcome improvement and I hope it will get stable soon. Piotr Przybylski basically continued in his effort two years ago when he had reimplemented setup script and now he had focused on user configuration. It can be stored in session, browser local storage (HTML 5 feature) or in separate table in phpMyAdmin configuration storage (that's new name for pmadb). This is something what people were requesting for very long time and I'm happy we can finally bring this feature. And last but not least is Lorikeet Lee, who spend lot of time on tuning user interface of phpMyAdmin. The most visible changes are on export and import pages, which now should be less scaring for new users, but there are other changes in lot of other places, for example the main page. Generally it was a great summer and I hope we will be so successful also next year. PS: You can try all these features on http://demo.phpmyadmin.net/.

GSoC 2010 is over for some time and I should write some summary how students projects ended up. The very short summary is that all five students were successful and their work got merged. Follows description of the project in no particular order. Thanks to Martynas Mickevicius you can now get charts out of various parts of phpMyAdmin. They are used on server status pages or you can get query results in form of several charts. It seems to work pretty well at least what I've tried so far. Ankit Gupta was working on Visual Query Builder for phpMyAdmin. Unfortunately this is only project which is not yet merged to master branch, mostly because some UI things were not yet finished. But hopefully it will be merged soon. Adnan Mughal was converting our schema export feature from PDF to support multiple formats. You can now get the schema as SVG, DIA or even Visio formats, some of them will require a little bit of tuning still, because the scaling is not perfect. Ninad Pundalik did a lot of work on AJAXifying phpMyAdmin. His changes touched quite a lot of places and there are still some rough edges (as you can see in our bug tracker), but this is definitely welcome improvement and I hope it will get stable soon. Piotr Przybylski basically continued in his effort two years ago when he had reimplemented setup script and now he had focused on user configuration. It can be stored in session, browser local storage (HTML 5 feature) or in separate table in phpMyAdmin configuration storage (that's new name for pmadb). This is something what people were requesting for very long time and I'm happy we can finally bring this feature. And last but not least is Lorikeet Lee, who spend lot of time on tuning user interface of phpMyAdmin. The most visible changes are on export and import pages, which now should be less scaring for new users, but there are other changes in lot of other places, for example the main page. Generally it was a great summer and I hope we will be so successful also next year. PS: You can try all these features on http://demo.phpmyadmin.net/.

This summer of 2010 has been memorable. It started with a difficult period following the hospitalization and death of my mother, a series of very intense and equally memorable conferences catapulting me out of my funk and ending with a trip to Ireland, perhaps one of my most pleasant trips ever. I have always wanted to go there, as I have some good Irish friends and I was also quite attracted to the place due to its history, so when the opportunity came for me to go, I did not hesitate to book my ticket. I was not left disappointed in any way, shape or form, although since I barely experienced the gray, misty, and rainy weather Irish is famous for, my experience may admittedly be a bit skewed. These are some of things I did and some of my fragmented thoughts about Ireland and some photos, proof that the weather was UnIrish Ireland and The Irish: Well I can t as no one can speak of The Irish as if they were some unitary group but I did learn a lot about Irish history and managed hang out with a number Irish folks (even a family) and one thing that seems to mark Ireland as distinct, what makes it stand out from the rest of its Western European brothers and sisters, is the pervasive sense of history bleeding into the ambiance, perhaps because it is so tragic. The short version of the history, if you don t know it, is that the Irish, especially the Catholics, got repeatedly screwed by the British monarchs/rulers/planters/government/ for nearly a thousand years, the last five hundred of those being particularly harsh and ugly, a cycle of slight gains crushed by various forms of tyranny and violence at least until part of the country achieved independence (Northern Ireland is a bit of a different story). I may have gone a bit out of my way to learn about Irish history, more than I have done for any other place, but this historical consciousness seemed to be inescapable, precipitating into all sorts of conversations and places. To take one example, I went to see Gaelic Football, one of the two beloved national sports (the other being hurling), and the minute you learn anything about this sport, you learn that it is intimately bound with the Irish fight for independence and nationalism. The Irish are also very warm, kind, and outgoing. They also seem to curse an awful lot as well, so much so cursing is a bit of a national pastime, which yes, I (f*cking) loved as I tend to have a bit of a foul mouth myself, curbed I will admit, in recent years and in the classroom. It crept up in a lot of places but was most pronounced during the All Ireland Semi-Final Gaelic football game when the ladies (not lads, mind you) behind me were constantly yelling at the referee, hurling the c-word (rhymes with trunk) whenever they made a call they disagreed with. EASA/Maynooth.: I went to Ireland to attend the largest Anthropological meetings in Europe, and in specific an all day panel on digital anthropology, which seemed like a great opportunity given we are a a bit of a minority. The conference was impressively large with roughly 1200 attendees (can you believe there are that many anthropologists?), smoothly run, and the all-day panel on digital media was quite lively and I got to meet some really interesting folks. I was a tad sad to find out Maynooth is the only university in Ireland with an Anthropology department (for crying shame lads!!!) but at least it is located in a darn stunning university: the old quarters of the campus are strikingly beautiful. Anonymous: I have done some work on Anonymous and well when I found out there was going to be a raid/protest at the Church of Scientology (a pretty dismal, and run down church), I got in contact with Irish anon to let them know I was coming. Although someone first decapitated me (at least in character with their norms, right?), when I showed up in person, they were not only civil but really quite hospitable (greeting me with one of my favorite songs). Overall it was a great day. I was reminded of important differences among Anons (Irish Anon s take their anonymity pretty seriously, the New York Anons do not) and also good to experience the social life and metabolism of a protest, especially one attended by folks who have lost family to the church. Dublin: Since I stayed with my friend and his family in Dublin, this is where I spent most of my time. I was able to hook up with various friends, including one from graduate school who just got back from years of fieldwork in Rwanda and hearing about his experiences and stunning but stunningly sad project made me feel like mine in comparison was Child s Play (in fact, it really was). I got to see the Debian crew (many who work at Google) and I finally paid a visit to the office, which was exactly how I imagined it to be (good and abundant food, good lighting, lots of toys and bikes, lots of Star Wars posters.. Yep, it could have been in Silicon Valley). But I was surprised at the young age of the marketing and sales folks who were hanging in their lounge when I ran into them. In fact when I saw them I thought like I was looking at my freshman class or something! It was great to see the Debian folks (though no one I met was actually Irish), as well, one of my favorite things to do whenever I visit a foreign city. I walked my heart out in the city getting a blister in a shoe that I thought was blister proof and while not as picturesque as some other European cities, it has a ton of character and no shortage of Guinness and pubs (no surprise there). My favorite places/things were: The National Library (great exhibit on Yeats, but make sure to use the multi-media as that is where all the information is stuffed), St. Stephen s Park (overflowing with chubby ducks and lovely flowers), the prison Kilmainham Gaol (would not advise a visit if you are feeling in any way down, there is some heavy shit you learn during the tour), the simple stained glass that seemed pretty common, and finally the Long Room in Trinity Church, which you enter after the Book of Kells (I realized just how much I adore books when I visited this old library stuffed from floor to ceiling with old old old books). The West Coast: I did not think I was going to head out west but after hurricane Earl started its burst along the eastern Seaboard and I was able to change my ticket for free so I stayed a few more days. I went to the Burren and the Cliffs of Moher both totally stunning, really majestic. As is often the case with these type of these natural wonders, I am often left elated and awed but such strikingly wondrous places also seem to subsequently spur a more melancholic state of mind and heart. Friends, Family, and Dogs: While in Dublin I stayed with my friend A. and his extremely hospitable family, which included, a brother, a father, and three Irish mutts, one of which, Buster (pitt bull/lab mix), pretty much stole my heart. Buster s true love, is food, so much so he almost poised himself to death a little while back snorting down something he shouldn t have costing the family a pretty penny to save him. My friend no longer lives there but came from Berlin and it was a real treat to not only spend days layered upon each other with a friend (it has been an awful long since I have done that outside of conferences) but also meet his family. You learn a lot about your friends that way and in this case, there is some serious and I mean serious intellectual jousting that happens, sometimes bordering on warfare but generally it plays out in more contained, civil and fascinating fashion. Now I understand why my friend is armed with seemingly endless knowledge: it was needed for purposes of defense at home. So in essence, a great, great trip and a fantastic way to end a memorable summer and transition into what I hope will be a bit of a monkish (I call it monk mode) period for this academic year. I am (so so so) fortunate enough to have a fellowship at the Institute for Advanced Study and am going to try my darnest to take advantage of the fact I am not teaching (Hell Yes!) and hide away and accomplish all that I have set out to do.

Two updates for Debian today.

I'm the upstream for psmisc and 22.13 finally got released, which also meant 22.13-1 Debian package got released too. There was some delay to it (see below why) but it is now out. Unless you run a mips or superH architecture, there is not really any exciting changes, but should make it compile for those two architectures and then at least get the mips buildd underway so the versions all line up.

Secondly, gjay had two minor bug fixes and was updated. If the analysis daemon kept playing music instead of looking at it or the vorbis files were not being recognised, then this new version will help there. The mpg123 command line patch will be rolled into the upstream too.

I'm also working on getting gjay to work with the Exaile player. If you want it to work with your player the easiest thing is to send me patches or code snippets that do the following:

• Detect your player is running
• Can give me the currently played song, preferably the filename of it. Exaile doesn't so I'll put it a kludge to guess it from title and artist
• Remotely send a playlist to your player and get the player to start

Last of all, my second son was born last week. It's back to those night feeds again and is probably why I've not written as much code as I normally would; but I wouldn't change that either.

I recently decided to use OfflineIMAP to synchronize my mails on my laptop. It's a great piece of software, and allows me to read my mail while I'm offline. I use it with Gnus, of course. But I lacked a proper way to integrate OfflineIMAP with it, so I decided to write a little Emacs extension to run and monitor OfflineIMAP directly from Emacs. Here comes offlineimap.el, an Emacs extension to run OfflineIMAP directly within Emacs. It will display OfflineIMAP output in a buffer, and optionally shows the current OfflineIMAP operation in the mode line. By default the status is in the mode line only if you are in the Gnus group buffer. But that's customizable, of course, since this is Emacs! If you are using el-get, there's already a recipe to install it!

The unofficial rc bug tracker currently knows about the following bugs:

In Total:	379
Affecting Squeeze:	291
Squeeze only:	65
Remaining to be fixed in Squeeze:	226

Of these 226 bugs, the following tags are set:

Pending in Squeeze:	14
Patched in Squeeze:	43
Duplicates in Squeeze:	21
Contrib or non-free in Squeeze:	13
Claimed in Squeeze:	2
Delayed in Squeeze:	8
Can fixed in a security Update:	14
Otherwise fixed in Squeeze:	16

Ignoring all the above (multiple tags possible), 129 bugs need to be fixed by Debian Contributers to get Debian 6.0 Squeeze released. However, with the view of the Release Managers, 244 need to be dealt with for the release to happen. Please see my older post for an explanation of the different numbers.

Sarcasm and satire usually don t work on the Internet. One cause of this is the lack of out of band signalling via facial expression or tone of voice. Another issue is the fact that in real life people usually know something about the person who they listen to while on the Internet it s most common to read articles without knowing much about the author. So the reader can t use I know that the author isn t an asshole as a starting point to determine whether a message should be interpreted literally. This is really nothing new. The standard in printed communication for a long time has been to use Emoticons (Wikipedia) to indicate emotion and other interpretation that might not be deduced from a direct reading of the text. The Wikipedia page cites examples of emoticon use dating back to 1857 although the combinations of characters used for different emotions has changed significantly many times. The common uses that we now know on the Internet date back to 1982. In my experience the symbol :-# is commonly used to note sarcasm or satire. Unfortunately it seems that none of the Internet search engines allow searching for such strings so I couldn t find an early example of this being used. While I haven t found a reference describing this practice, I regularly receive messages annotated with it and find that people generally understand what I mean when I use it in my own email. But that is usually applied to a sentence or two. For a larger section of text a pseudo-HTML tag such as </satire> can be used to signal the end of satire. It seems that a matching start tag is optional as recognising the start of satire is a lot easier once the reader knows that some of the content is satirical. In spoken English a phrase such as but seriously may be used for the same purpose, but such a subtle signal may be missed on the Internet particularly by readers who don t use English as their first language. Another way of signaling a non-literal interpretation is by using Scare Quotes the deliberate usage of quotation symbols to indicate that the writer disagrees with the content that is written. That is common for the case of referencing a phrase or sentence that you disagree with, but doesn t work for a larger section of text. A final option is to make the satire or sarcasm so extreme that no-one can possibly mistake it for being literal. This is not always possible, Poe s Law holds that Without a winking smiley or other blatant display of humor, it is impossible to create a parody of Fundamentalism that SOMEONE won t mistake for the real thing [1]. I think that Poe was understating the case, it is impossible to create a parody of religion that most people won t mistake for the real thing without signals or context. For an example read LandOverBaptist.org and Chick.com, of course if you know those sites then you will know whether they are satirical or serious but I expect that most readers of my blog won t invest enough effort into either of those religious sites to determine whether they are serious or satire. But satire and sarcasm without signals or a reputation usually fails. One example of success is The Onion which is a long running and well known satirical news site [2]. But even The Onion it is regularly mistaken for being serious the number of occasions when people forward me Onion articles for amusement are vastly outnumbered by the number of occasions when I see people taking it seriously. Even when material is known to be satirical it can still fail grossly. An example is the Chaser s satire of the Make A Wish Foundation [3]. Even material that is well known to be satirical seems to fail when it attacks bad targets or attacks in a bad way. One difficulty is in satirising bigoted people, to effectively satirise them without attacking the minority groups that they dislike can be a difficult challenge. Finally, when you write some satire and members of your audience don t recognise it you should consider the possibility that you failed to do it properly. If you can t get a hit rate close to 100% for people with the same background as you then it s probably a serious failure.

• [1] http://rationalwiki.org/wiki/Poe's_Law
• [2] http://www.theonion.com/
• [3] http://en.wikipedia.org/wiki/The_chaser

You will be missed so much. You were kind, you were fun to be around.
It is a privilege to have met you. Debian is privileged for the effort and time you put in it.
Your contribution will remain with us and will inspire others for a long time.
You made a difference in this world, one that will last and outlive you. I can only thank you.

Rest in peace, my brother. See you at the other side of the Firewall, and thanks for all the FLOSS ;)

Just got an email from Tobias Gruetzmacher as a followup on my previous post about sshfs. He reported another problem with sshfs. It fail to handle hard links properly. A simple way to spot this is to look at the . and .. entries in the directory tree. These should have a link count >1, but on sshfs the count is 1. I just tested to see what happen when trying to hardlink, and this fail as well:

% ln foo bar
ln: creating hard link  bar' =>  foo': Function not implemented
%

I have not yet found time to implement a test for this in my file system test code, but believe having working hard links is useful to avoid surprised unix programs. Not as useful as working file locking and symlinks, which are required to get a working desktop, but useful nevertheless. :) The latest version of the file system test code is available via git from http://github.com/gebi/fs-test

Debian and Ubuntu have a set of official membership roles that can be granted to regular contributors. Those roles come with rights that enable the contributors to do their work and to participate in the project governance (elections and other official decision-making processes). It s also a way for the distributions to acknowledge the work done: most contributors are proud of the status they reached. The membership structure plays an important role in the development of a distribution: it defines the kind of contributors that are welcome in the project, it sets expectations of the project towards its contributors and defines their rights. In the end, this shapes the project s ability to recruit new contributors to keep the project alive and kicking. This article introduces the existing statuses in Debian and Ubuntu, and defines the sometimes confusing jargon associated with them. The Debian Case Debian only has two types of official members: Debian Developers (DD) and Debian Maintainers (DM). The rights of the developers are codified in the Debian Constitution while those of the maintainers have been defined in a general resolution of 2007. The Debian Maintainer status is still mostly documented in a wiki page. The integration of this new status in Debian s official processes has been slow to come largely because it was introduced at that time without enough negotiation with the involved parties. Nowadays, it is preferred that people get the DM status before applying for DD. DM is a very limited role: maintainers can only upload packages that already have their name on them (either in the Maintainer or Uploaders field) and a specific flag (DM-Upload-Allowed: yes) that only Debian Developers can add. They have no other rights and limited access to Debian s resources. Besides those official roles, there are also maintainers of packages that have no official status within Debian except that they are listed in the Maintainer field of the package. They are doing the maintenance work but all uploads are done by a Debian Developer after verification of the work done (this is called sponsorship and is the only way to start with official packaging work). Once the DD trusts the maintainer, the developer will typically ask the maintainer to apply for DM status in order to be relieved from the sponsorship work. In the end, that makes three different kind of package maintainers and a lot of confusion when you discuss membership issues in particular when the New Maintainer process is the path that you follow to become a Debian Developer. Don t be fooled by the names when reading Debian s documentation! The Ubuntu Case Ubuntu had, from the start, an official Ubuntu Member status that includes all contributors: developers of course, but also documentation writers, artists, translators, etc. This status notably grants the right to vote in elections of the Community Council, the right to participate on Planet Ubuntu, and the @ubuntu.com email alias. For developers, the situation is more complicated: the wiki page lists no less than five different statuses. Initially, developers were split between Ubuntu Core Developers and the MOTU (Masters Of The Universe). The latter were responsible of the universe/multiverse sections of the archive while the former also had upload rights for the main/restricted sections. But, inspired by the Debian Maintainers concept and facing concrete problems in terms of archive management, they changed their infrastructure to offer more fine-grained control on package uploads. Ubuntu can now grant upload permissions on a package-per-package basis, but it can also delegate the right to grant upload permissions with the same granularity. This lead to the new Per-Package Uploader status which is simply an Ubuntu Member with upload rights on a limited set of packages where they have a specific expertise. The more generic Ubuntu Developers status now encompasses members of various development teams that have been delegated the right to manage upload permissions on a (usually large) package set (the current teams are Ubuntu Desktop, Mythubuntu, Kubuntu, and Edubuntu). Those teams can define their own policy to add new members provided they follow the basic rules defined by the Developer Membership Board (see this wiki page). Ubuntu Contributing Developer is an intermediate status for someone who is not yet ready for one of the other developer statuses but who has still shown enough commitment to be an Ubuntu Member. All those statuses can be obtained in a similar way: you prepare a wiki page listing your past contributions, you collect testimonials from existing members that you have worked with, you add yourself in the agenda of the next meeting of the board (or council) that grants the status that you seek, and you attend the meeting. The members of the board will decide whether you are ready for the status (or not) based on what you provided in the wiki, based on your answers during the meeting (and on a mailing-list for developers), and based on what others have to say about you. The most important boards are usually elected by the community while others are commonly appointed by the community council. Those governance bodies include Canonical employees but not as many as one would expect: two out of eight in the Developer Membership Board, two out of eight in the Community Council, but all six members of the Technical Board. The last figure, while not intended, is not surprising given the high expectations set on potential members of the technical board. Mark, as the founder, is the only person to have a permanent seat on both the Community Council and the Technical Board. Comparison of the Statuses Between Debian and Ubuntu The following table summarizes the rights given to each developer role in the two projects (Put the mouse over the abbreviations to know what they are referring to).

Rights	Debian		Ubuntu
	DM	DD	UM	PPU/UD	MOTU	UCD
Package maintenance via sponsorship	Y	N/A	Y	Y	Y	N/A
Official email alias	-	Y	Y	Y	Y	Y
Participate in votes for members	-	Y	Y	Y	Y	Y
Participate in votes for developers	-	Y	-	Y	Y	Y
Upload rights restricted to pre-approved packages	Y	-	-	Y	-	-
Upload rights restricted to a section of the archive	-	-	-	-	Y	-
Unlimited upload rights	-	Y	-	-	-	Y
Number of contributors (as of 2010-07-27)	117	904	462	27	85	63

Please note that the number of contributors are not 100% accurate for Ubuntu. A contributor can have multiple statuses (direct membership to a launchpad group) granted over time (while gaining experience). The problem has been mostly avoided by calculating differences between number of members of the various groups but it s not perfect and it can t be: some MOTU are also PPU for packages in main and it s legitimate (but I only counted them as MOTU and not as PPU). Another limitation is that members of some administrative teams are included indirectly in many teams and thus appear in the count while they should not. Anyway, this simple table makes it obvious that Ubuntu s structure offers a broader choice of statuses. They acknowledge the work of all contributors from the start while still giving the most critical rights only to those who have proven that they deserve them. Despite this difference, Debian still has a significant advantage in terms of number of developers. That number does not tell the whole story though: the Ubuntu contributors include many Canonical employees (e.g. 36 out of 63 core developers have a @canonical.com email registered on their launchpad account) that are likely to spend more time working on the distribution than the average Debian member. But even if comparing person-hours would be a challenging thought experiment, in practice it s of not much interest if both projects continue to cooperate and if more and more of the contributions flow in both directions. Debian is aware of the shortcomings of its structure. Changes to better accommodate non-packagers have been discussed several times already. The last efforts in that direction were unfortunately perceived as a solution ready to go rather than a proposal to be discussed, and the project got quickly buried by a general resolution (GR). Even if that resolution invited for further discussion and a new proposal, the truth is that when someone s initiative is corrected by way of GR, it usually kills any motivation to go forward. Possible Evolution? On the Ubuntu side, the infrastructural changes were completed recently and they don t expect any further change in the near future. They do plan, however, to expand usage of those new features so that more teams benefit from the possibility to control upload rights on packages that are relevant to them, and so that more individuals developers apply to become Per-Package Uploaders on packages that they know very well. On the Debian side, a recent discussion on the debian-project list brought back the topic of the bad terminology and it was agreed that the New Maintainer process should be renamed into something else ( New Developer process has been suggested). But Christoph Berg Debian Account Manager and hence heavily involved in the New Maintainer Team suggested that Debian would be better off implementing the long-awaited membership changes before trying to update all the documentation. It would certainly imply some more vocabulary updates. Later in the discussion, he confirmed that membership reform is on the top of the TODO list of the new maintainer team (just after the rewrite of the nm.debian.org website). What can be expected from this reform? The following answers are my own guesses based on my experience of Debian, but the project hasn t decided anything yet.

• First of all: a new status for contributors that are not packagers. The tricky part will be defining the process to follow and the rights granted.
• Changes to the technical implementation of the DM status. The current implementation does not allow to give upload rights to a single DM if two are listed in the Uploaders field of a package (and both might not have the same experience for that package). Furthermore, it suffers from annoying restrictions like the inability to upload new binary packages.
• A change of the Debian constitution to integrate those new statuses is almost unavoidable.
• Other more invasive changes have been proposed like replacing the NM process by a simple designation by other DD, but it s unlikely to happen. The NM process can already be greatly simplified by the application manager if the applicant can show good testimonials from other developers and if he has a track record of real contributions (e.g. as witnessed by changelog entries in Debian packages).

Almost two years have elapsed since the previous efforts in that direction, the new maintainer team has recruited new members and is in a general better shape. Hopefully, the next episode of this saga will have a better outcome.

This article was first published in Linux Weekly News. In a comment, Mark Shuttleworth tried to explain how the Ubuntu community is being setup.

10 comments Support my work

After a long time, I decided to write again. I decided to start with software I wrote for my own usage that could be usefull for other people. I'll start with python-dvdvideo, a DVD-Video reader written in Python 3, and the reference tool dvd-video-backup-image, a generic DVD-Video dumper. Lets see, if this blog will see more postings in the future.

mksh was merged into Android (both AOSP and Google s internal master tree) in the night 24/25th August, and is expected to be the one shell to rule them all, for Gingerbread. mksh(1) now also has a cat builtin, for here documents mostly. It calls the cat(1) command if it receives any options. The shell is nevertheless smaller than yesterday because of improved string pooling. There s another reason to use the MirOS OPTU-16 encoding instead of PEP 383, on which I already wrote: try passing a wide-char filename to a function such as MessageBoxW, or create a filename on a system using wide chars, such as FAT s LFN or ISO 9660 s Joliet, or one that only allows Unicode (canonically decomposed u out of all things) like HFS+. OPTU-8 at least maps to somewhat reserved codepoints (would, of course, be better to get an official 128 codepoint block, but the chance s small of getting that in the BMP). Still. Oh well, the torrents. I ve remade them all, using one DHT seed node and OpenBitTorrent as tracker and put them on a very rudimentary BT page that will be completely redone soonish. Please re-download them. I currently do not believe f.scarywater.net will return. Finally, I fell victim to a selling-out and may have just bought a Windows Mobile 6 based phone (Glofiish X650) and an SDHC card and an extra battery with double capacity. Well, at least it s said to run CacheWolf well. I still would like to have something like Interix, Cygwin, UWIN, coLinux, or maybe some qemu-for-WinCE variant that runs Android, Maemo, Debian/armhf (or armel or arm) at near-native speed (and is usable the device sadly doesn t have a hardware keyboard, but it comes with SiRFstar GPSr). It only has 64 MiB RAM, like the Zaurus SL-C3200 and the jesusPhone, though. Any chance to get MirWorldDomination onto that device as well?

The unofficial rc bug tracker currently knows about the following bugs:

In Total:	390
Affecting Squeeze:	299
Squeeze only:	82
Remaining to be fixed in Squeeze:	217

Of these 217 bugs, the following tags are set:

Pending in Squeeze:	18
Patched in Squeeze:	35
Duplicates in Squeeze:	21
Contrib or non-free in Squeeze:	14
Claimed in Squeeze:	3
Delayed in Squeeze:	10
Can fixed in a security Update:	13
Otherwise fixed in Squeeze:	17

Ignoring all the above (multiple tags possible), 126 bugs need to be fixed by Debian Contributers to get Debian 6.0 Squeeze released. However, with the view of the Release Managers, 245 need to be dealt with for the release to happen. Please see my older post for an explanation of the different numbers.

Urban Honking has an insightful article about the Arduino and suggests that it is one of the most important factors for the development of the computer industry in the near future [1]. It compares the Arduino to the Altair. Wired has an interesting article about a company that provides a satellite kit and a launch into low Earth orbit for $8000 [2]. Arduino in space? Linux Journal has an interesting article by David Rowe about the Mesh Potato which is a Wifi mesh router that also runs VOIP [3]. One particularly interesting aspect of this article is the explanation of the way they designed and tested it. Susan Shaw gave an informative TED talk about the toxic effects of the attempts to clean up the oil spill in the Gulf of Mexico [4]. It seems that trying to disperse the oil just makes it worse, and the chemical companies are refusing to disclose the chemicals that are being used. The New York Times has an interesting article by David Leonhardt about the value of pre-school teachers [5]. Some research on the difference that good teachers can make in economic terms suggests that the make an economic difference to the children to the value of $320,000 per annum (IE a class of 16 children who were taught for a year would on average each receive a benefit of $20,000 over their lifetime). Also there are social benefits which aren t counted by that study. While I can t imagine pre-school teachers getting paid $320,000 any time soon, it does seem obvious that good teachers deserve significantly better pay. Of course one problem is how to determine which teachers are good, better test results are not a reliable indication. Paul Krugman describes America as being on the unlit, unpaved road to nowhere due to the policies of saving money by cutting funding for schools, street-lights, and roads [6]. The Chive has an amusing post about how to quit a bad job [7]. It would be good if someone really did this, I m sure that there are enough creative people who don t like their job. Hell Pizza in New Zealand published a zombie themed choose your own adventure on Youtube [8]. Unfortunately the options to choose the next segment don t work on HTML5 with Chromium so if you don t have flash you miss out. AskThePilot.com has an informative essay about airline security written by a commercial pilot [9]. The anecdote about the pilot not being allowed to take the type of knife that is issued to first and business class passengers is rather amusing. FredOnEverything.net has an interesting analysis of Wikileaks and why the Pentagon and Fox News hate it [10]. Fred is a very skillful writer, while he s not the first person to say some of these things he may have said it best. The Wikipedia page on Borosilicate glass (which is best known under the trademark Pyrex) is really interesting [11]. Borsilicate glass was formerly known as Duran and it s main characteristic that makes it suitable for lab use is resistance to Thermal Shock, but it s also harder and has a higher melting point. Apparently you can get Pyrex drinking glasses, I want some! Eben Moglen gave an interesting talk Freedom in the Cloud about the development of free servers to manage personal data and replace Facebook etc (among many other things) [12]. The Debian Wiki has an articla about designing such a system [13]. The APNIC published an interesting paper on IPv4 background radiation [14]. Apparently some /24 s receive so much random traffic (from broken applications and viruses) that they can t be delegated. IPv6 will solve this problem by making it infeasible to scan all IP addresses. Also it s interesting to note the excessive amounts of traffic to 1.0.168.192 which is from applications too broken to correctly send data to 192.168.0.1 which have been installed by sysadmins who are too incompetent to watch what is being sent out of their network.

• [1] http://www.urbanhonking.com/ideasfordozens/2009/05/why_the_arduino_matters.html
• [2] http://www.wired.com/gadgetlab/2010/07/tubesat-personal-satellite/
• [3] http://www.linuxjournal.com/article/10604
• [4] http://www.ted.com/talks/susan_shaw_the_oil_spill_s_toxic_trade_off.html
• [5] http://www.nytimes.com/2010/07/28/business/economy/28leonhardt.html
• [6] http://www.nytimes.com/2010/08/09/opinion/09krugman.html
• [7] http://thechive.com/2010/08/10/girl-quits-her-job-on-dry-erase-board-emails-entire-office-33-photos/
• [8] http://www.youtube.com/watch?v=9p1yBlV7Ges
• [9] http://www.askthepilot.com/essays-and-stories/terrorism-tweezers-and-terminal-madness-an-essay-on-security/
• [10] http://www.fredoneverything.net/Wikileaks.shtml
• [11] http://en.wikipedia.org/wiki/Borosilicate_glass
• [12] http://www.isoc-ny.org/?p=1338
• [13] http://wiki.debian.org/FreedomBox
• [14] http://meetings.apnic.net/__data/assets/pdf_file/0007/23398/ipv4-background-radiation.pdf

Apparently this statements also holds true for frozen yogurt. An Austrian A1 spokes person has confirmed that the HTC Magic will not receive a 2.1 or 2.2 Android update. One can only wonder about the sugar HTC puts on top of regular Android that hinders themself to update their products. The Austrian A1 carrier sells you the device for a 18 month contract, but actively only supported it for 6 month. I must revise my previous positive review of the HTC Magic. Some HTC speaker had promised earlier this year an upgrade to at least 2.1: HTC Magic 2.1 upgrade. A SFR speaker had promised an update to current Android: HTC Magic and Nexus One 2.2 upgrade. The Webkit Android Browser can be easily tricked into leaking your user and passwords: Android Luecke. Beside the obvious that as a user one shouldn't give out to much data to untrusted third party this opens lots of Google accounts for criminal activity. The inability of the carrier to provide a secure and uptodate device is massively deceiving and certainly not appropriate handling of their defects liability.

My file system sematics program presented a few days ago is very useful to verify that a file system can work as a unix home directory,and today I had to extend it a bit. I'm looking into alternatives for home directory access here at the University of Oslo, and one of the options is sshfs. My friend Finn-Arne mentioned a while back that they had used sshfs with Debian Edu, but stopped because of problems. I asked today what the problems where, and he mentioned that sshfs failed to handle umask properly. Trying to detect the problem I wrote this addition to my fs testing script:

mode_t touch_get_mode(const char *name, mode_t mode)  
  mode_t retval = 0;
  int fd = open(name, O_RDWR O_CREAT O_LARGEFILE, mode);
  if (-1 != fd)  
    unlink(name);
    struct stat statbuf;
    if (-1 != fstat(fd, &statbuf))  
      retval = statbuf.st_mode & 0x1ff;
     
    close(fd);
   
  return retval;
 
/* Try to detect problem discovered using sshfs */
int test_umask(void)  
  printf("info: testing umask effect on file creation\n");
  mode_t orig_umask = umask(000);
  mode_t newmode;
  if (0666 != (newmode = touch_get_mode("foobar", 0666)))  
    printf("  error: Wrong file mode %o when creating using mode 666 and umask 000\n",
           newmode);
   
  umask(007);
  if (0660 != (newmode = touch_get_mode("foobar", 0666)))  
    printf("  error: Wrong file mode %o when creating using mode 666 and umask 007\n",
           newmode);
   
  umask (orig_umask);
  return 0;
 
int main(int argc, char **argv)  
  [...]
  test_umask();
  return 0;
 

Sure enough. On NFS to a netapp, I get this result:

Testing POSIX/Unix sematics on file system
info: testing symlink creation
info: testing subdirectory creation
info: testing fcntl locking
  Read-locking 1 byte from 1073741824
  Read-locking 510 byte from 1073741826
  Unlocking 1 byte from 1073741824
  Write-locking 1 byte from 1073741824
  Write-locking 510 byte from 1073741826
  Unlocking 2 byte from 1073741824
info: testing umask effect on file creation

When mounting the same directory using sshfs, I get this result:

Testing POSIX/Unix sematics on file system
info: testing symlink creation
info: testing subdirectory creation
info: testing fcntl locking
  Read-locking 1 byte from 1073741824
  Read-locking 510 byte from 1073741826
  Unlocking 1 byte from 1073741824
  Write-locking 1 byte from 1073741824
  Write-locking 510 byte from 1073741826
  Unlocking 2 byte from 1073741824
info: testing umask effect on file creation
  error: Wrong file mode 644 when creating using mode 666 and umask 000
  error: Wrong file mode 640 when creating using mode 666 and umask 007

So, I can conclude that sshfs is better than smb to a Netapp or a Windows server, but not good enough to be used as a home directory. Update 2010-08-26: Reported the issue in BTS report #594498 Update 2010-08-27: Michael Gebetsroither report that he found the script so useful that he created a GIT repository and stored it in http://github.com/gebi/fs-test.

If you're following me on identi.ca you probably already know that I've been designing a small PCB for a USB-based SPI chip programmer named openbiosprog-spi. The main use-case of the device is to help you recover easily from a failed BIOS upgrade (either due to using an incorrect BIOS image, due to power outages during the flashing progress, or whatever). The device only supports SPI chips, as used in recent mainboards (in DIP-8 form factor, or via manual wiring possibly also soldered-in SO-8 variants). It can identify, read, erase, or write the chips. Of course the whole "toolchain" of software tools I used for creating the hardware is open-source, and the hardware itself (schematics and PCB layouts) are freely released under a Creative Commons license (i.e., it's an "Open Hardware" device). The user-space source code is part of flashrom (GPL, version 2), the schematics and PCB layouts are licensed under the CC-BY-SA 3.0 license and were created using the open-source Kicad EDA suite (GPL, version 2).
The schematics, PCB layouts, and other material is available from gitorious:

  $ git clone git://gitorious.org/openbiosprog/openbiosprog-spi.git

You can also download the final Gerber files (ZIP) for viewing them, or sending them to a PCB manufacturer. Some more design notes:

• The device uses the FTDI FT2232H chip as basis for USB as well as for handling the actual SPI protocol in hardware (MPSSE engine of the FT2232H).
• Attaching the SPI chip:
  • There's a DIP-8 socket on the device so you can easily insert the SPI chip you want to read/erase/program.
  • Optionally, if you don't want a DIP-8 socket, you can solder in a pin-header with 8 pins, which allows you to connect the individual pins to the SPI chip via jumper wires or grippers/probes.
• The PCB board dimensions are 44mm x 20mm, and it's a 2-layer board using mostly 0603 SMD components.

Basic usage example of the device on Linux (or other OSes supported by flashrom):

  $ flashrom -p ft2232_spi:type=2232H,port=A -r backup.bin (reads the current chip contents into a file)

Over at the main projects page of openbiosprog-spi at http://randomprojects.org/wiki/Openbiosprog-spi I have put up a lot more photos and information such as the bill of materials, the Kicad settings I used for creating the PCBs, the Gerber files and the Excellon drill files and so on. The first few prototype boards I ordered at PCB-POOL.COM (but you can use any other PCB manufacturer of course), the bill of materials (BOM) lists the Mouser and CSD electronics part numbers and prices, but you can also buy the stuff elsewhere, of course (Digikey, Farnell, whatever). I already hand-soldered one or two prototypes and tested the device. Both hardware and software worked fine basically, you just need a small one-liner patch to fix an issue in flashrom, but that should be merged upstream soonish. In order to make it easy for interested users to get the PCBs I'll probably make them available in the BatchPCB Market Place soonish, so you can easily order them from there (you do still need to solder the components though). Note: I'm not making any money off of this, this is a pure hobby project. All in all I have to say that this was a really fun little project, and a useful one too. This was my first hardware project using Kicad (I used gEDA/PCB, also an open-source EDA toolsuite, for another small project) and I must say it worked very nicely. I didn't even have to read any manual really, it was all pretty intuitive. Please consider not using Eagle (or other closed-source PCB software) for your next Open Hardware project, there are at least two viable open-source options (Kicad, gEDA/PCB) which both work just fine.

One of the most debated points at the DebConf&Debian BOF was money management. Let's talk about how it works in an ideal world, and then how it tends to work in practice. This post only talks about year-to-year management, I hope for another post talking about the complexities of budgeting within a single year.In an ideal world, DebConf fundraises all the money it needs to go on. There is a DebConf sub-team for fundraising, since Debian doesn't have its own fundraising team. This team make brochures and solicits sponsors - there will be another post about this by someone on the fundraising team.We do our best to raise enough money to cover the conference, but it's long and hard work. In the months leading up to the conference, we are almost always running short. However, in the end, somehow, it comes through by a combination of cutting costs and continued fundraising work.The money is held by various nonprofits like SPI in the USA and FFIS in Europe. They hold it in trust for Debian. I think most people on the DebConf team agree that this money belongs to Debian, but is earmarked for DebConf since it was fundraised for DebConf. In the past, the DPL has not been asked for permission to use DebConf-fundraised money to put on a DebConf, since that's always been kind of obvious that it's for DebConf.Planning how much it will cost is hard. Venue costs aren't always known in advance, and we don't know our number of attendees until late. There are many uncertainties, which will be described later. These questions are hard to answer, but not impossible. Most of they time things work out in our favor.At the end of the conference, we tally everything up. Usually(=always, as far as I know), there is a surplus. This surplus is important, since DebConfN+1 usually needs a little seed money to get started before the new funds start rolling in. Historically, this seed money has been used for DebConfN+1, since it was raised earmarked for DebConf.In practice, it works out mostly like this. We keep track of the money, and try to stay within our budget, and cut things when we need to.For the last two years (coinciding with the recession), we have expected a deficit in the months and weeks leading up to DebConf. In these cases, we asked the DPL for some money to be given from Debian general funds for DebConf, and we provide background on our fundraising and budget status for use in making that decision. Both times, the Debian funds went to ensure that we could get travel sponsorship out. Debian general funds are not used without DPL approval.Also, keeping all the paperwork straight is a hard task. We have limited people, and the budget-trackers are always doing many other jobs, too. Personally, I'm not convinced that we always know how much the exact post-DebConf surplus is and where it is (SPI, FFIS, ...) each year. It sort of gets morphed into the general Debian money, and then we know "we have (about) XXX surplus" and then use that amount as our starting seed for the next year, taken from wherever is convenient. If Debian gives money to help make DebConf happen, it's usually as a loan. However, this loan isn't tracked carefully year to year. Someone looked up numbers during the BOF and reported that Debian has given/loaned 50,000 USD to DebConf over the years, but it's hard to know just how accurate that is. For comparison, the total DebConf budget over those years has been close to 1,000,000 USD, as best I can figure.We've tried various types of accounting software with mixed success. They are a lot of overhead, and we usually revert to keeping track of things in spreadsheets.
Our goals for budgeting (being self-sustaining) are good and well agreed on, but the implementation is hard.However, it's a place where it is easy for dedicated volunteers to get involved. If you wanted to help to make this better, you could do a lot of good by watching #debconf-team and lists, making careful records of our cash flow and where it comes from and goes to. All money we spend is announced and documented on the lists, so the information is there. To really do a complete job, this person would need to remain motivated in the months after DebConf and wrap up all of the final payments. Preferably, we'd have the same person year after year.There are many more suggestions that both others and I could make, but I'll save those for later.Tags: debconf

Recently I was surprised to find that many of the rules I took for granted when dealing with an OpenPGP keyring, and the other activities associated with a web of trust, were new to some people I spoke to. These rules are born from a mixture of experience, accepted good practice, and wider reading, with a small dose of paranoia thrown in for good measure.

• Protect your secret keyring with file permissions. For GPG, your ~/.gnupg directory should not be readable by anybody else, including root (even though he could break in if he wanted). Mine is octal 700, and the secret keyring itself is 600.
• Create a revocation certificate, and keep online and offline copies (gpg --gen-revoke <keyid>). The revocation certificate is published to key servers by you in the event that you detect key compromise or loss. It s important that this is also well-protected, because the secret key is not required to upload it. Print a copy in bold and on good quality paper, so it will last a long time, and preferably keep them both off-site in a trusted location. Typing all those characters back in is a bore, but if your online copy fails it s better than being impersonated.
• Use a good pass-phrase, not a password. A pass-phrase is actually easier to remember than a string of random characters, without risk. It can be long enough to make brute-force attacks pointless, and if you re using the key a lot and can t be bothered to type it all the time, gpg-agent is a handy and safe tool. Your login password is not an acceptable pass-phrase for your secret key!
• Regularly refresh your public keyring from a well-connected keyserver. I detailed this in an earlier post; it s a case of calling gpg --refresh-keys once a week or so. This is the first line of defence against other people s keys being compromised, since without it you ll never know that your messages to them are now vulnerable. A well-connected server, like those in pool.sks-keyservers.net, matters since not all servers synchronise predictably.
• Rotate encrypting sub-keys regularly. This reduces the effect of a compromised key: if you rotate keys annually, a compromise only opens one year of mail. Small comfort, but every little helps. Don t delete the old keys though, just revoke them otherwise you won t be able to open mail from those senders who don t refresh their public keyrings.
• Keep your key offline and make a separated signing sub-key for day-to-day use. There isn t room to go into detail here, but you can export a signing sub-key with a relationship to your primary key material, so if it is compromised you can revoke it without losing all those carefully-collected signatures. The same goes for encrypting sub-keys. Adrian von Bidder has a guide to this, but he warns that it should be taken with a pinch of salt, so this isn t so much a rule as an interesting idea. I have not experimented with this yet.

And some rules about signing and signatures:

• Don t place too much store by signing parties. Parties are great for rapidly building a web of trust, but a part of that trust is to do with how the other party checks identities. I don t believe it s possible to establish that with just a few minutes chat and an identity check, so I never make a sig2 or sig3 on a key I met at a party. I m usually happy to do that later when I have an established relationship with the key holder, of course.
• Never upload your own signature to a key server. Apart from the fact that the key holder might not want their key published, you also have no way to prove that the key holder has access to the email address on their UID. It s much better to send the signature to them encrypted, which demonstrates that they the both key and can read mail at the address (the caff program is ideal for this).
• Give personal trust (the trustdb) carefully. Getting that message Good signature from in your mail client is really nice, but not at the expense of giving away trust where it is not deserved. In the same vein as not placing too much trust at signing parties, you should avoid the temptation to over-trust keys in your personal trustdb. Reputation on a mailing list is absolutely no substitute for observing the party taking care over their verifications when doing face-to-face signings.

Do you have other rules or guidelines that you use when dealing with keys and signatures?