I’m currently in Xen hell. My Thinkpad (which I won’t replace any time soon) has a Pentium-M CPU without PAE support. I think that Debian might re-introduce Xen support for CPUs without PAE in Lenny, but at the moment I have the choice of running without Xen or running an ancient kernel on my laptop. Due to this I’ve removed Xen from my laptop (I’m doing most of my development which needs Xen on servers anyway). Now I’ve just replaced my main home server. It was a Pentium-D 2.8GHz machine with 1.5G of RAM and a couple of 300G SATA disks in a RAID-1. Now it’s a Pentium E2160 1.8Ghz machine with 3G of RAM with the same disks. Incidentally Intel suck badly, they are producing CPUs with names that have no meaning, and most of their chipsets don’t support more than 4G of physical address space [1]. I wanted 4G of RAM but the machine I was offered only supported addressing 4G and 700M of that was used for PCI devices. For computation tasks it’s about the same speed as the old Pentium-D, but it has faster RAM access, more RAM, uses less power, and makes less noise. If I was going to a shop to buy something I probably would have chosen something different to get support for more than 4G of RAM, but as I got the replacement machine for free as a favor I’m not complaining! I expected that I could just install the new server and have things just work. There were some minor issues such as configuring X for the different video hardware (and installing the 915resolution package (which is only needed in Etch) to get the desired 1650×1400 resolution. But for the core server tasks I expected that I could just move the hard drives across and have it work. After the initial install the system crashed whenever I did any serious hard drive access from Dom0, the Dom0 kernel Oopsed and network access was cut off from the DomU’s (I’m not sure whether the DomU’s died but without any way of accessing them it doesn’t really matter much). As a test I installed the version of the Xen hypervisor from Unstable and it worked. But the Xen hypervisor from Unstable required the Xen tools from Unstable which also required the latest libc6, and therefore the entire Dom0 had to be upgraded. Then in an unfortunate accident unrelated to Xen (cryptsetup in Debian/Unstable warns you if you try to use a non-LUKS option on a device which has been used for LUKS and would have saved me) I lost the root filesystem before I finished the upgrade. So I did a fresh install of Debian/Unstable, this time it didn’t crash on heavy disk IO, instead it would lock up randomly when under no load. I’ve now booted a non-Xen kernel and it’s working well. But this situation is not acceptable long-term, a large part of the purpose of the machine is to run virtualisation so that I can test various programs under multiple distributions. I think that I will have to try some other virtualisation technologies. The idea of running KVM on real servers (ones that serve data to the Internet) doesn’t thrill me, Tavis Ormandy’s paper about potential ways of exploiting virtual machine technologies [2] is a compelling argument for para-virtualisation. Fortunately however my old Pentium-3 machines running Xen seem quite reliable (replacing both software and hardware is a lot of pain that I don’t want). In the near future I will rename the Xen category on my blog to Virtualisation. For older machines Xen is still working reasonably well, but for all new machines I expect that I will have to use something else - and I’ll be blogging about the new machines not the old. I expect that an increasing number of people will be moving away from Xen in the near future. It doesn’t seem to have the potential to give systems that are reliable when running on common hardware. Ulrich Drepper doesn’t have a high opinion of Xen [3], the more I learn about it the more I agree with Ulrich.

• [1] http://en.wikipedia.org/wiki/List_of_Intel_chipsets


• [2] http://taviso.decsystem.org/virtsec.pdf


• [3] http://udrepper.livejournal.com/tag/virtualization

Share This

Here is a story people are bugging me to tell here: Since 2003, every year in fisl s last day we, Debian Brasil, hold a party to celebrate Debian s anniversary (I know it s on August, but it s probably the only opportunity we ll have to gather all the gang together so we do it in advance anyway). It s always something that draws everybody s attention in the conference I wonder if the pieces of cake we distribute have anything to do with it Anyway, this anecdote happened during fisl9.0 s party. I was there, helping by distributing cake and blowing our whistles when Jon Maddog Hall got there to check what s going on. I met Jon around 2001, in OpenBeach, an event that happens in Florian polis every year (and that Jon likes to attend)... he s the most pleasant guy, with lots of stories to tell. Since this years fisl was so intense, I barely had time to talk to him in fact, that was the first time we saw each other this year. We hug each other and were asking how s each other life s going and so, when Jon got his camera out of his pocket and asked some guy in the crowd to get our picture. I did the same. We exchanged some compliments and he left saying that he still had to work in his talk. Jon is quite a character. In fisl, every time he wanders around his picture is taken over one hundred times (I actually saw some father taking pictures of him holding his child like he were running for Senate or something, one time). So he left with some people around him and I think he d not seen what happened next. I turn towards Debian s booth, to resume the cake delivery when some guy in the crowd asked me to take a picture with him. And then another one and another. I believe my picture was taken another two or three times before I got to the booth. I can t believe! I was about to tell people Hey! I am nobody! Stop taking pictures with me What were they thinking? I imagine something like I don t know who this guy is, but if Maddog took a picture with him, he must be some one! was crossing their minds. When the party was over I went back to the Organization Committee room and told this story LTSP s Jim McQuillan (another good friend) told me I was Important by Association , and everybody just kept laughing at me because of that. I haven t got the time to tell Jon about it I hope he s reading. I think I am going to check what pictures people are uploading about fisl, to see if I can find myself on any ;-)

May 1st is a holiday in many countries around the world - It is, at least, here in Mexico. So, what's a man to do when faced with really-crappy network connectivity at home?
Yesterday I had dinner with Gigio, and among many other things, we talked about the Ciclot n, which I've only done twice. And on my way back home, I crossed (twice) the path of a group of ~100 cyclist going over Colonia Roma.
Anyway, whatever the reason, I woke up very well in the mood for a nice 20km ride:

Now, lets get the day started!
[update] I was told about a worldwide nudist cycling activity, the World Naked Bike Ride. Their page states their main motivator:

We face automobile traffic with our naked bodies as the best way of defending our dignit and exposing the unique dangers faced by cyclists and pedestrians as well as the negative consequences we all face due to dependence on oil, and other forms of non-renewable energy.

Of course, Mexican groups take part in this interesting activity - Ciclomarcha nudista.
Will I be there? I don't know - Quite probably, yes. I first thought of it as a joke, and as a dangerous activity. But, come think of it, I am a move-by-bicycle-in-the-largest-city-in-the-world activist. I have been knocked over or shit-scared (and fortunately not more than that) by careless drivers, but than won't force me back into driving my car when not needed. And this kind of activities, which do get their good share of exposure, need all the participants - I hope to be there, June 7 12:00PM, at the beginning of Mexico City Ciclopista (FFCC de Cuernavaca esq. Ej rcito nacional, Polanco) Chapultepec's Lions' entrance, at the crossing of Lieja and Reforma.

Share the awareness. There are very good promotional pictures. And collective nudist activities are quite fun, experiences to remember for life.

These last two weeks most of my time has been sucked into getting Python 2.5 as default into testing. That’s done now. I made use of the block uploads thingie ftpmaster implemented for the release team to use. Basically, if your package could disrupt an “almost there” transition, the upload will be rejected. The blocks were in place for 5 days, which I think it’s acceptable. As long as we don’t end blocking stuff for very long, I think we should be fine. See the end of this post for more about this. Though it was quite a bit of work, I’m very glad I took care it myself, since now I really feel I’m 100% back to Debian, after the time I spent off for health reasons. Update: Oh, and I forgot to say: having control over britney has really really helped. Thanks a lot Joerg for that.

---

Dudesconf Tomorrow I’m leaving to Coru a for Dudesconf, which is a kind of Debconf-ES. I’m giving an introductory talk to Git, a semi-lighting talk about grep-dctrl (30 min.), and (gasp) a talk about Debian packaging with a VCS. We may have a Debian Quiz as well. I’m so looking forward to it, since many people who’ll attend are amongst my most loved ones, and I already missed last year’s since I wasn’t fully recovered yet. See you there!

---

Minirok and Sevilla One of the reasons I wasn’t fully back to release management during the past 5 months or so is because I spent as much time as I had doing development for Minirok. I don’t think I mentioned here before, but I was participating in a Free Software Contest for college students organized by the University of Sevilla, Spain. Such effort finally paid off, since Minirok was elected as one of the finalists. This means next week I’ll go to Sevilla, to make a presentation of the project, and who knows what more. ;-) I’m very excited.

---

Finally, more on blocking uploads This Python 2.5 transition was the first time the block uploads feature was used, and there were a couple bumps along the way. In particular, a couple packages were blocked, when they shouldn’t have been (libqt4-ruby and evolution-sharp), and one needed package was not blocked, though Rene Engelhard thankfully spotted it very quickly (mono). The problem is it’s not completely straightforward to generate a list of all the stuff that could possibly affect the transition. What I did was to make a run of britney on an arch that had all the needed bits in place, and block all the packages that migrated together as a result of the hint. This fails in two ways:

• it can include stuff that is not needed for the transition, namely unrelated packages that happen to become candidates for testing by one of the their dependencies having migrated as part of the hint. This is what happened with libqt4-ruby and evolution-sharp.
• it can miss stuff that is needed, like packages that can migrate by themselves, so the main run prior to the hint migrates them, and hence do not show up as part of the hint, but that some package of the transition depend on. This is what happened with mono.

The second problem, though, can be fixed by parsing the excuses list and blocking stuff that some bits of the transition depends on. Easy enough. Yet, there are more cases when things can go wrong, for example a shlibs-bumping upload of a package (say, sqlite3) linked against by some package still needing a couple of builds (say, qt4-x11). For that, blocking uploads is not an option, since that’d be an insane amounts of packages that, furthermore, can be uploaded if they don’t bump shlibs, so I guess we’ll have to ask in the next release update that shlibs-bumping uploads are coordinated in -release too, at least when close to finishing a transition.

A few weeks ago, NYU hosted an interesting event about the future of the Internet, appropriately tittled The Futures of the Internet, the video of which is now available here. One of the panelists was Jonathan Zittrain (who recently wrote an important new book bearing the same name as the event) and during the talk he provided a few ideas about how geeks and developers can help secure the Future of the Internet. While I agree with a lot in fact most of his assessments about the state and fate of the Internet as he lays out in his book and his talks, his characterization of geek/hacker/developer politics is not one of them. Basically, one of Zittrain’s claims is that developers are not doing enough to save the Future of the Internet and it is their rampant, Atlas-like libertarianism, which is, in part, to blame (first made 37:20 minutes into the video for those who want to listen to the actual comments). They have little-to-no political consciousness, are too cool to care about the fine print and they don’t care about the broader politics of the the Internet because they assume that they can just hack around any sort of barrier and impediment. While we can, without a doubt, identify a strain of libertarianism among hackers, it is by no means representative of all of geekdom and in fact, is becoming more and more a worn out 1990s stereotype/clich as time passes than an accurate representation of what is a far more variegated set of ethics and practices among hackers (and I will soon publish an article on this topic). It also completely fails to capture the ethical spirit as well as sociological, and political workings of one of the most important strains of hacking free and open source software which not only powers most of our (open) Internet but which in fact has provided a pretty hefty ethical backbone by which to conceptualize one of the ways we should think about the fight for the future of the Internet. Ok, time for a rant now Geeks not only designed the Internet, an indisputably revolutionary medium, but also implemented, and continue to maintain it, and then in their copious spare time, also engage in fighting back the political, legal and corporate encroachment which threatens to limit the very revolutionary nature of the Internet (as Chris Kelty’s new book on Free Software argues). If these acts by geeks are not enough political action, then maybe the development of not just one, but multiple entirely open and free alternatives to the only two proprietary operating systems that exist today might be a political act that would satisfy? Many would agree that even simply using a free operating system is a political act. It would be better to claim that individuals, lawyers and other political actors are not doing enough to save the Future of the Internet, rather than imploring the already overtaxed geeks to set aside everything that they are already doing to do something even more. (end rant) It also seems that when it comes to political questions related to the Internet, net neutrality being the hot topic now, or fighting restrictive and problematic laws like the DMCA, one of the only groups of people (outside of lawyers and librarians) to actually understand and dissect the fine print (and geeks actually are pretty attuned to and like to dissect the fine legal print), to protest these unsavory laws, and to support the organizations who are doing something about it (like the EFF), are geeks and hackers. While many geeks are not necessarily keen on conceptualizing their labor in traditional political terms, or aligning their technical projects with a political affiliation, and yes would rather just be writing good code, they do fight for their productive freedom and this productive freedom just happens to relate to most questions and concerns related to an open, accessible, and tweakable Internet, built by the geeks, lest we forget What was perhaps most surprising was that he also seemed to think that geeks and developers have not turned to “apprenticeship,” nor policies and procedure to coordinate their development projects, unlike Wikipedia, which he considers a shining example that geeks should look towards as a beacon of policy that geeks should consider emulating in their projects (comments made answering my question). He clearly has not been hanging out with any Debian developers in the last 10 years nor has he gone through their New Maintainer Process In other words, he seems to think they are allergic to regulation due to their accentuated libertarianism, or are against structure because of their anarchism, neither which is remotely true. I think I found this characterization most ironic and problematic for before Wikipedia was even an entry on a Wiki, projects like Debian (and most other F/OSS projects) were transforming and changing to integrate normative procedures and policies that allowed a group of people to work together, scale, grow and deal with crises’. No, they don’t have the Wikipedia badge system, but that system is emblematic of Wikipedia’s own transformation into integrating its own normative procedures and policies for working together, not an example of an idealized policy system that other projects are too primitive to have evolved into yet. About one hour into the talk when questions opened up, I objected to his characterization, but given his answer back to me, I did not make much of a dent in his thinking. Another lawyer Tim Wu (who also wrote a wonderful book on the Internet) chimed in to give me some props and also made a good point that even if geeks are the only groups of people who would storm AT&T and know intimately about the importance of net neutrality, there is a lot of room for thinking about how to strengthen and improve the tactics and politics among geeks and developers so that we can ensure the type of open and generative Internet and set of technologies we value. As part of thinking and rethinking new strategies, it is as key to acknowledge and honor the past. In this regard, free software development has been pivotal both in terms of providing software (and making it is an important political act as is choosing to use free software over propriety software) and a set of important set of ideas that a lot of lawyers like Yochai Benkler and Lawrence Lessig have run with to make some important political claims of their own. So despite my rant above, which was a rant and thus exaggerates things to some degree, I do think there is much more that geeks and non-geeks can do, such as help translate these uber-geeky issues into less geeky terms (and actually this is already being done by some geeks as the work of Jelena Karanovic has shown, or translate the technical issues into new domains as the uber-geek Karl Fogel is doing with question copyright but first lets give credit where credit it due and recognize that labor is political

In the last week, Nicholas Negroponte gave this unfortunate interview decrying "open source fundamentalism" and hinting the possibility of a warmer relationship with Microsoft. Predictably, this has elicited an ongoing response by OLPC News and on the OLPC development mailing lists. Just a few days before Negroponte's statements hit the press, I gave a talk at Penguicon called Laptop Liberation where I talked about why I thought that OLPC's use of a free software operating system and embrace of free software principles was essential for the initiative's success and its own goals of education reform and empowerment. I've been saying similar things for some time. My main point boiled down to something that, appropriately enough, Nicholas Negroponte was fond of saying back when the project was still called the $100 laptop: an extremely cheap laptop is not a matter of if, but of when and how. This technology will define the terms on which students communicate, collaborate, create, and learn. These terms are dictated by those with the ability to change the software -- by those with access to computers, the source necessary to make changes, and the freedom to share and collaborate. Constructionism -- OLPC's educational philosophy -- is about putting powerful tools and control over those tools into the hands of learners. It is about learning through exploration and creation -- about shaping one's own educational environment. Constructionist principles bear no small similarity to free software principles. Indeed, OLPC's stated commitment to free software did not happen by accident. OLPC convincingly argued that a free system was essential for creating a learning environment that could be used, tweaked, reinvented, and reapplied by its young users. Through these processes, the XO becomes a force for learning about computation and an environment through which children and their communities can use technology on their terms and in ways that are appropriate and self-directed. We know that laptop recipients will benefit from being able to fix, improve, and translate the software on their laptops into their own languages and contexts. Much more importantly, however, are all of the uses for the laptops that OLPC has not -- and can not -- think up. OLPC is a powerful tool for learning, but ultimate power is only in the hands of those that can freely use, change, and collaborate in defining the terms of their learning environments. In its commitment to software freedom, OLPC chose not to be arrogant by assuming that it knows how its users will use their laptops. Flexible environments designed for constructionist learning and a free software platform protect against this arrogance. Constructionism and free software, implemented and taught in a classroom, offer a profound potential for exploration, creation, and learning. If you don't like something, change it. If something doesn't work right, fix it. Free software and constructionism put learners in charge of their educational environment in the most explicit and important way possible. They create a culture of empowerment. Creation, collaboration, and critical engagement becomes the norm. OLPC does not get to choose if educational technology happens. If we work hard at it though we might get to influence the "how" and the "who." Proprietary software vendors like Microsoft want the "who" to be them. With free software, users can be in power. What's at stake is nothing less than autonomy. We can help foster a world where technology is under the control of its users, and where learning is under the terms of its students -- a world where every laptop owner has freedom through control over the technology they use to communicate, collaborate, create, and learn. This, to me, is the promise of OLPC and its mission. It is the reason I've been involved and in support of the project since nearly day one. It is the reason I left Canonical and Ubuntu to come back to school at MIT to be closer to the then nascent unincorporated project. It is the reason that OLPC's embrace of constructionist philosophy is so deeply important to its mission and the reason that its mission needs to continue to be executed with free and open source software. It is why OLPC needs to be uncompromising about software freedom. As an adviser and sometimes contractor to OLPC, OLPC does not need to listen to me. But I hope, for all our sake, that they do.

Update: Richard Stallman and the FSF have published another essay on the same topic focused more on pure free software (i.e., less education specific) objections.

There was a discussion today on the LUV-main mailing list about apache virtual hosting. Someone suggested mod_vhost_alias which is OK for a simple solution, but pretty limited. IMO, there are much better ways of doing it. Specifically, generating config fragments for each virtual host. I wrote such a system years ago, and have always thought “I should package it up for debian”. There’s a moderate amount of work involved in doing so, and probably an even larger amount of support work afterwards. Worst of all, I’d have to write - or con someone else into writing - documentation for it (it’s easy enough to use that you can be SHOWN how to use it in 10 minutes, but that isn’t good enough for a distributed package). I guess the point of this blog post is to find out if there’s sufficient interest to make it worth the effort. Feel free to comment or email me about it. If anyone wants to look at the code, i can tar it up easily enough, and even write some simple/minimalist instructions. anyway, here’s (a slightly edited version of) what i posted to LUV-main: Preamble I looked into apache’s mod_vhost stuff years ago, but IMO it’s just too inflexible and limited to be of much use. It’s fine if ALL your vhosts are identical…but not if you need different settings on some or all of them. As soon as you need a custom setting for one vhost, you have to step outside mod_vhost_alias and roll-your-own anyway. which means you now have two different methods of configuring your vhosts. Instead, I’ve used my own vhosting system for years (long before mod_vhost), on dozens of servers…the largest of which had over 500 vhosts on it - the biggest problem was the enormous number of file handles the apache processes needed for log files. After tweaking the kernel file-max etc limits, it actually worked without problem but I always intended to rewrite that bit of it to use a log splitter daemon, perhaps vlogger or similar. Unfortunately, I a) left that job before I had time to implement it, and b) never did figure out a good way to handle the error logs because apache doesn’t prefix the vhost name to ErrorLog output like it does for CustomLog outout (best compromise was to just have one combined error log rather than one per vhost….most web programmers/website operators never look at the error logs anyway. Personally, I find it invaluable, most people don’t even seem to think of looking at it) Synopsis of Operation 1. Creating a new vhost is a 3-step operation:

1. create a user (with adduser), optionally with a postgres or mysql account of the same name.
2. edit /etc/virtuals/virtual-hosts.conf, copy/paste/edit one line
3. run “make”

In step 2, hardly anything needs to be changed from one vhost to another - in the most common case, just the account’s login name, and the domain name. There are a number of different “flags” that can be set for each vhost in this file - ssl, cgi, mason, htdig, webcheck, aliases, group, samba, canonify (use mod_rewrite to redirect back to the site’s canonical name), and more. The Makefile runs perl scripts to generate the apache config fragment (and other config fragments - e.g. for htdig and samba), add/remove any IP addresses if needed (usually not, with name-based vhosts), set up the home directory structure, copy in any sample files (e.g. html forms and images for htdig), and reloads apache. It also checks in any changes to svn (or RCS. easily modified to use other revision control systems). On machines which are part of a load-balanced setup, it even ssh-es into other machines in the array, runs “svn update” and “make” to ensure that the vhost is operational on the other machines too. Finally, it concatenates all the active config fragments into one large file because apache reloads a LOT faster if it doesn’t have to load hundreds of little config files. 2. Removing a vhost is as simple as editing virtual-hosts.conf and commenting out or deleting the line, then running make again. virtual-hosts.conf The virtual-hosts.conf file is extremely useful. It gives me a single authoritative source of all the relevant details about EVERY vhost running on a machine. These details can be used by other scripts (e.g. cron jobs to run webalizer, analog, htdig indexing, webcheck, etc; cgi scripts to display a list of vhosts on the machine with clickable links, and more). This single config file makes the whole system easily extensible: write a new script to parse the config file, and loop through each entry, performing some relevant operation. e.g. run webalizer for every vhost, dumping the output under ~/public_html/LOGS/ (with optional .htaccess file to control access). Adding optional generation of samba shares for each vhost was quite easy. In theory, it wouldn’t be hard to put a GUI or web front-end on to it. Oddly enough, i’ve never bothered. And I’m not inclined to bother now (but wouldn’t object if anyone else did) Database? no, thanks I’ve occasionally thought about putting this config data into a database, but the benefits just aren’t anywhere near good enough to outweigh vi and all the benefits of text file configuration (especially comments!) Packaging? And, of course, one of these days, i should package it all up for debian. I’ve been saying that for years, and just haven’t got around to it. It wouldn’t take a huge amount of work to do so, there are only a few small things i need to do to automate the installation, things that i just do by hand in a few minutes when installing on a new box. Before packaging it, I also want to automate the adduser part of the process, as well as creation of postgres or mysql accounts. Most importantly, I ought to separate the code from the templates in the config fragment generator scripts, so that the local sysadmin can edit them without having to edit the scripts….OK, all up not a tiny job but not particularly big either….just big enough to be ignored on my TODO list for years. I’d definitely have to change the config file format from the current one-line (colon separated fields) to using a .ini style file. I’ve wanted to do that for a while but have inertially-resisted it so far because the one-line format is so convenient…unfortunately, I’ve added so many flags and features over the years that it’s seriously ugly - probably scary to someone who doesn’t know it already (i.e. anyone outside of the set of “me + a handful of other people”). I estimate this would take about a days worth of hacking I’d also take the opportunity to clean up some of the cruft :-) Essential Design Criteria BTW, one crucial design criteria for my system is that it should *NEVER* under any circumstances overwrite any hand-edited changes. This is implemented in a number of ways. First, it takes an md5sum of each vhost’s config fragment when it generates it. If/when the script is run again, it only re-generates the config fragment if the md5sum hasn’t changed. If it has changed, it generates the updated fragment as “.new” for manual merging if required. Secondly, when generating conf fragments it looks for specially named files in a directory for extra stuff to add before the <VirtualHost> line, between the <VirtualHost>…</VirtualHost> lines, and/or after the </VirtualHost> line. This is to provide another location to put custom changes for each vhost while still retaining the ability to have improvements/new features to the script automatically applied to existing vhosts. The reason for this is that I utterly loathe and despise allegedly “easy” configuration tools that blow away custom hand-edited changes. I hate them so much that this was THE major incentive for developing my own vhosting system. In Use I’ve been using my system for over 10 years now. It grew out of earlier bits and pieces, but the current incarnation has been working since 1999, with the occasional new feature or update (e.g. to add support for apache2 as well as apache1, and minor changes to adapt to changes in debian’s apache packaging). There are also a handful of ISPs and several small businesses around Australia running it because they’ve hired me to set up their web servers. Finally… My vhosting system isn’t a system for people who are scared of or dislike command-lines or text-file configs. It’s a system for people who love them, but want an easy way to automate a dull, repetitive task. Such automation is easy for CLI tools, and next to impossible for web/GUI tools. Syndicated from Craig Sanders' Errata: Miscellaneous Thoughts poll: package my vhosting system for debian?

Guess I got your attention with that title? :) Anyways, britney is no longer. And with her testing died too. Ok, not really, but todays work was hand over all of testing and put it where it should have been for years - in the hands of the release team. For those not following the “inner” workings of stuff:

• Testing is the place where our next stable release gets prepared. It is managed by a software called britney, which was written by Anthony Towns.
• Britney was, until today, run by the ftpmasters. The release team had no way to change anything, besides their hint files. They couldn’t change code, couldn’t rerun it themselve if needed, nothing. (Somewhere in the past, for some period, some of the release team was able to do extra britney runs, AFAIK, but thats long ago).

As the above is surely not nice, especially as Anthony (and as such the one in ftpmaster who really fully knew britney) left the ftpmaster group, all of the code managing testing now lives with the release team. They now just prepare a file in a special format and then tell us (via a ssh trigger) to read that in. Which simply sets testing to the new state. This new way of doing things has multiple advantages:

• The release team can run britney whenever they want to. It no longer needs to wait for a cronjob at some externally defined time or ping an ftpmaster to do an extra run. That should be pretty helpful, especially when we are near to a release again.
• They can do code changes or even replace the current britney with a totally new one. (There seems to be a new one available that does the same as the current one, just way faster. Or even more. Which never seems to have got any comment from our side. Badbad. Well, now they can play with it if they want to.). As long as they output valid data for us to import, it is now completly their decision how it gets generated.

The above action did need surprisingly few code changes in dak. It basically was

• disable the cronjob for britney as user dak
• Add a script that the release team now pushes via ssh. This script reads in the above mentioned special file and sets the database.
• Change process-accepted to write the urgencies to a different directory. And no longer try to mkdir this directory, but warn if it doesn’t exist (or is not writeable by dak) and then use a temporary one for the run.
• Move the transitions.yaml to a different location and add a redirect in apache, so everyone can still access it from outside. (It has to live in a ftpmaster controlled directory, thats all).
• Eliminate a britney call from our daily cronjob.

Comments: 8

Since my DAM rejection earlier this year I’ve been trying to cut down on Debian work. Unsurprisingly (and perhaps surprisingly to some) I fealt a bit emotionally drained and demotivated from defending myself and working with debian security. Lately I have been working with new maintainer Andrea De Iacovo and I am very pleased with his work on Wordpress. Wordpress takes most of my time, like the beautiful day of Saturday yesterday. :) Unfortunately Andrea has had problems finding a sponsor, though a Wordpress security release expedited matters and got Nico Golde’s upload powers into action. :) So if you’re a DD and you use Wordpress, please help out Andrea. Before I get unsubscribed from Debian Planet, I thought I should pimp my:

• Debian tips and tricks site – the current 400-odd tips are in the public domain so please rip them, update them or add some!
• The fantastic Web kiosk distribution Webconverger based on Debian Live, which also has a blog

Both based on ikiwiki. ;) Anyway, I am still around of course. Met some great people in the Debian community and sadly some pretty pedantic unfriendly ones too. Final suggestion for DDs: Revoke your GPG keys more often and go to less key signing parties. :P

When I got in to my hotel room last night, it was brightly illuminated and the TV was on with music and a message to greet me. The first thing I did was turn it off, along with most of the lights. Later, I asked at the reception when they turn on the lights. I was expecting them to have a central switch for each room that the receptionist could flip when the guest finished checking in and has started to make way towards the room. Not so. The guy behind the desk told me that housekeeping sets it all up. So my lights have been on and the TV blasting for the entire afternoon and evening. That did not make me feel at home. It made me cross. NP: The Pineapple Thief: 8 Days Later

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

It seems existing versions of gcc don't warn when you use an assignment as a truth value as the first operand to the conditional operator. For example: Presumably you meant ==, but unfortunately that does not warn with gcc (maybe it will one day). We can use the tree dumping mechanisms (previously mentioned here) to confirm our problem. For the following function gcc creates: As you can see, this isn't the result we wanted. This can be quite nasty if you use the conditional operator in a hidden fashion; for example behind an assert. A common idiom is: You'll currently get no warning you've slipped up and used the assert wrong. Moral of the story: a quick audit of your code might turn up some surprising uses! However, in this case, the Intel compiler picks this one up: Although it's often not trivial to move to another compiler, as a rule I would recommend trying alternative compilers for your code as it's a great sanity check.

Sometimes, companies come up with the most stupid restrictions. And unfortunately, other companies enable them to do that.In this particular case, the DHL parcel service and Acrobat teamed up: You can buy and print shipping labels online with DHL. I thought I'd give that a try, especially since there is a 24h dropbox nearby.The label can be printed either using a Java applet or a PDF file. It thought PDF - sounds great. But it isn't. It's a very special PDF file. It has a scriped printing button that will print with the default settings (so it comes out at a completely different printer that I would have wanted to print it). In fact I would have expected the printing dialog to come up, and thought the button had not functioned at all. Also I could probably not have used it if I had opened the PDF file in a different PDF viewer. At my own laptop I might not even have Adobe Reader anymore - I use Evince. It's faster and has a much better UI. But I don't have a printer, so I used a computer at the university which used Acrobat by default, fortunately.And: you may only print the real shipping label once. Only 'sample' labels can be printed arbitrarily often. This is counted on the server side, btw., and also applies to the Java applet.Apparently you also must not rename the PDF file (WTF?) and you need to be online to print the file. So you can't save the PDF file and take it to an offline computer that does have a printer...Dear DHL: have you heard of that ancient technology called 'Photocopiers', sometimes also dubbed 'Xerox'? You know... lots of people still have these. Heck, I figure I could even use Adobe Reader to print the label into another PDF file with the print restrictions removed...So you gain nothing by preventing multiple printouts, but you can seriously annoy users (e.g. when their printer malfunctions)Please stop imposing such stupid restrictions on users.

Brian Aker dropped by and replied to my previous essay by making the following comment:

I believe you are hitting the nail on the organic vs nonorganic open source. I do not believe we have a model for going from one to the other. Linux and Apache both have very different models for contribution but I don t believe either are really optimized at this point. Optimization to me would lead to a system of less priests and more inclusion.

I made an initial reply as comment, and then decided it was so long that I should promote it to a top-level post. I assume that when Brian talks about “organic open source” what he means is what I was calling an “open source development community”. Some googling turned up the following definition from Mozilla Firefox’s organic software page: “Our most well-known product, Firefox, is created by an international movement of thousands, only a small percentage of whom are actual employees.” This puts it in contrast with “non-organic” software, where all or nearly all of the developers are employed by one company. (And anyone who proves talented at adding features to that source base soon gets a job offer by that one company. By that definition we can certainly see projects like Wine, Mysql, Ghostscript (at one time), and others as fitting into that model, and being quite successful. There’s nothing really wrong with the non-organic software model, although many of them have struggled to make enough money when competing with pure proprietary softare competitors, with MySQL perhaps being the exception which proves the rule. In most of these cases, though, the project started more as an organic open source, and then transitioned into the non-organic model when there was a desire to monetize the project and/or when the open source programmers decided that it would be nice if they could turn their avocation into a vocation, and let their hobby put food on the family table. Solaris, of course, is doing something else quite different, though. They are trying to make the transition from a proprietary customer/supplier relationship to trying to develop an Open Source community and what John s candidate statement pointed out is that they weren t really interested in creating an organic open source developer community at all, but they wanted the fruits of an open source community with plenty of application developers, end-users, etc., all participating in that community. We don t have a lot of precedent for projects who try to go in this direction, but I suspect they are skipping a step when they try to go to the end step without bothering to try to make themselves open to outside developers. And by continuing to act like a corporation, they end up shooting themselves in the foot. For example, the OpenSolaris license still prohibits people from publishing benchmarks or comparisons with other operating systems. Very common in closed-source operating systems and databases, but it discourages people from even trying to make things better, both within and outside of the Open Solaris core team. Instead, they respond to posts like David Miller s with Have you ever kissed a girl? . (Thanks, Simon, for that quote; I had seen it before, but not for a while, and it pretty well sums up the sheer arrogance of the Open Solaris development team.) So while Linux may not be completely optimized in terms of less priests and more inclusion, at least over 1200 developers contributed to 2.6.25 during its development cycle. Compared to that, Open Solaris is positively dominated by high priests and with a you may not touch the holy-of-holies attitude; heck, they won t even allow you to compare them to other religions without branding you a heretic and suing you for licensing violations!

Brian Aker dropped by and replied to my previous essay by making the following comment:

I believe you are hitting the nail on the organic vs nonorganic open source. I do not believe we have a model for going from one to the other. Linux and Apache both have very different models for contribution but I don t believe either are really optimized at this point. Optimization to me would lead to a system of less priests and more inclusion.

I made an initial reply as comment, and then decided it was so long that I should promote it to a top-level post. I assume that when Brian talks about “organic open source” what he means is what I was calling an “open source development community”. Some googling turned up the following definition from Mozilla Firefox’s organic software page: “Our most well-known product, Firefox, is created by an international movement of thousands, only a small percentage of whom are actual employees.” This puts it in contrast with “non-organic” software, where all or nearly all of the developers are employed by one company. (And anyone who proves talented at adding features to that source base soon gets a job offer by that one company. By that definition we can certainly see projects like Wine, Mysql, Ghostscript (at one time), and others as fitting into that model, and being quite successful. There’s nothing really wrong with the non-organic software model, although many of them have struggled to make enough money when competing with pure proprietary softare competitors, with MySQL perhaps being the exception which proves the rule. In most of these cases, though, the project started more as an organic open source, and then transitioned into the non-organic model when there was a desire to monetize the project and/or when the open source programmers decided that it would be nice if they could turn their avocation into a vocation, and let their hobby put food on the family table. Solaris, of course, is doing something else quite different, though. They are trying to make the transition from a proprietary customer/supplier relationship to trying to develop an Open Source community and what John s candidate statement pointed out is that they weren t really interested in creating an organic open source developer community at all, but they wanted the fruits of an open source community with plenty of application developers, end-users, etc., all participating in that community. We don t have a lot of precedent for projects who try to go in this direction, but I suspect they are skipping a step when they try to go to the end step without bothering to try to make themselves open to outside developers. And by continuing to act like a corporation, they end up shooting themselves in the foot. For example, the OpenSolaris license still prohibits people from publishing benchmarks or comparisons with other operating systems. Very common in closed-source operating systems and databases, but it discourages people from even trying to make things better, both within and outside of the Open Solaris core team. Instead, they respond to posts like David Miller s with Have you ever kissed a girl? . (Thanks, Simon, for that quote; I had seen it before, but not for a while, and it pretty well sums up the sheer arrogance of the Open Solaris development team.) So while Linux may not be completely optimized in terms of less priests and more inclusion, at least over 1200 developers contributed to 2.6.25 during its development cycle. Compared to that, Open Solaris is positively dominated by high priests and with a you may not touch the holy-of-holies attitude; heck, they won t even allow you to compare them to other religions without branding you a heretic and suing you for licensing violations!

A few news from the campaign of NMU's targeted at bringing po-debconf translation ratio as high as possible for as many languages as possible (and also improve packaging quality quite significantly on many loosely maintained packages).

• Galician translation are going up quickly. Each NMU features updates by Jacobo Tarrio and Galician is now ranked 10th in the translation ratio list. They could even hit the Spanish (Castillian) translation ratio and become the most translated language from Spain..:-)
• Finnish follows the same way. Starting from quite low, Esko Araj rvi bringed it to 13th rank and now above 50%
• Russian is also going up quite fast and could at least reach 70 or even 80%
• Portuguese (non Brazilian) hit the 90% mark and heads up to 100%
• German is very likely to be bringed to 100%. Crossed 97% today.
• French will be bringed to 100%, as it was for Etch. Close to 99% now.

Maintainers are incredibly cooperative (at least those who are active..:-)). l10n NMUs are always welcomed. Some prefer uploading themselves which is of course perfectly OK. I will now also focus my efforts to minimize uncoordinated changes to debconf templates. Those are generally motivated by the will to follow writing style guidelines, which is perfectly OK. I however still suggest maintainers to send call for translation updates *before* uploading packages with modified templates. Will probably soon post that suggestion to -release.

MySQL 5.1.24 was released last week. I am nearly done with preparing and testing the updated package for Debian, and will upload it to experimental within the next few days. MySQL AB decided to disable Cluster support from their official binaries with 5.1.24, but unfortunately they forgot to add a note to the release notes. It was only mentioned in a bugreport where someone asked why mysqld does no longer understand the --ndbcluster option in 5.1.24:

It was decided that we do not support NDB Cluster in the 5.1 release but do a general release of the Carrier Grade Edition 6.2 branch (which is based on MySQL 5.1) instead as it has received substantial improvements on the cluster side that can not be backported to 5.1 for now.

So while the 5.1.24 source still includes the cluster storage it is not built into the binaries released by us. A MySQL Cluster 6.2.x binary release is expected to be become available before end of April.

I am still thinking about how we should handle the new situation in Debian, hence I will keep Cluster support enabled at least for the first 5.1.24 upload. Maybe we will see a separate mysql-cluster (or mysql-cge) package in Debian soon...

[Warning, long post ahead. If you aren’t interested in Debian or it’s internals or my work - skip it. :) ] So as you might have read here, I got the delegation of the debadmin AKA FTPMaster group lately. I’ve got some pressure to not accept this delegation, but the response to my blog post about it had been extremely positive, everyone said I should not deny it. Knowing to have such a support within Debian is something I really appreciate, making it worth to spent time on my various jobs. Now, lets take this and write a little about what happened since I gained that extra group:

• Aj resigned from anything that requires additional group privileges. (As a DD run id ajt on a debian.org machine if you want to verify this).

Work I did since then:

• One of my first action was to finally finish the release transitions support, something which was waiting for it’s final merge for a long time. The release team was waiting for this and did ping us (ftpteam) multiple times, but now it is there. You can read more about this feature in my list mail and my initial blog post. (Oh, and something I have missed about everywhere I wrote about this: This whole transitions thing is based on an idea from Marga!)
• Added a mapping in dak, requested by Luk
• Merged a patch from another ftpteam member, Thomas Viehmann, which fixes daks handling of Format 3.0 (something) uploads. Currently it rejects them, without the merged patch it just crashed.
• Changed our daily cronscript to no longer VACUUM/VACUUM analyze projectb in every run. Postgres >= 8.1 has autovacuum, and its turned on, so we should be fine without extra runs of it. (And while i write daily cronscript or cron.daily - I mean dinstall, right now run twice a day).
• Fixed a breakage in code introduced by Debian maintainers which led to process-accepted (one main part of the daily cronscript) breaking. It was triggered by some octave-$FOO package upload, where the ###censored### did add someone twice into the Uploaders: field. Not helpful, but now we ignore this case and just warn us.
• Added a script to expire old database dumps. We do dump our projectb (the main archive database) twice in every run (beginning/end), which made 4 dumps a day. While old dumps seemed to have been removed on an irregular schedule - the dumps took about 71Gb in total. Which pretty much filled up merkels /org, as they get synced there. Now, my script (reworked version of a script from Florian Reitmeir) works with an “incremental deletion” scheme:

  RULES = [ ‘days’:14, ‘interval’:0 , ‘days’:31, ‘interval’:1 , ‘days’:365, ‘interval’:31 , ‘days’:3650, ‘interval’:365 , ]

  That means we now keep all files from the last 14 days. Then keep one per week. Then one per month and finally one per year for up to ten years. All the rest gets deleted. Space usage of our dumps now: 4.8Gb. Thats 66Gb gained. Source is in dak bzr if you want it, MIT licensed. If you improvie it further I love to see patches.
• Changed the way the projectb copy on merkel gets synced. Up to now it was run from a cronjob which syncs a number of files people want to see on merkel, including the database dumps, then reloads the database using the latest dump. This was run at a “random” time, scheduled to be shortly after dinstall. Problem with such a setup is that it will miss extra dinstall runs and also might import old data, as the timing of the cronjob meant it runs while dinstall on ftp-master is still doing its work. So bad bad. It is now done using the ssh trigger feature I described a while ago. So our daily cronscript now tells merkel when it should sync its database again.
• Coordinated with the qa group we now also push them, when the sync on merkel is done, so they can kick their cronjobs to reread projectb and just run whenever they get told that something changed. Yet another bad timing eliminated. (Well, will be done today). Oh, if you have something that depends on information from projectb / ftp-master and try to schedule it to run after dinstall - talk to me, we might just push you too.
• Started talking with the release team how britney, the script which manages our testing distribution, should be handled in the future. It currently is run fully under control of us ftpmasters (and reading hint files from the release team). Ideally this gets changed so that the release team has the control of the code and whatever gets run with it and then, in some way, maybe using ssh triggers, tells us to import a new dataset for testing. That needs some thinking, as obviously it should be limited to testing and we should make sure it doesn’t break the archive, but I think that is doable.
• Prepared a patch that sanity checks the Provides: field, something we currently don’t do. Mailed our internal team address about 3 days ago, to let others look at it, no response yet.

The above should nicely sum up my work in the past few days. Most of it pretty trivial, but still things that have to be done. I plan to keep doing things, slowly going from the trivial stuff to more complicated ones, but there is lots of stuff ahead. Some ideas, and this is a very incomplete list just from my head, there are tons of things that one could do, include

• Finally allowing dpkg-sig support in .deb files. I once, years ago, made a patch for it, which never got applied. (Mostly my fault here). Need to rewrite that to fit the actual code.
• Add a few more sections to our archive. There are multiple bugs asking for them, and some others also come to mind. Like java, ruby, octave, haskell, debug to name a few, but there might be more needed. (Yes, longterm all sections should die and be replaced with debtags. I just don’t think debtags is there yet?!)
• Maybe finally drop m68k and add other architectures to the archive, like some $foo*bsd one. (Thats for after lenny of course, at the time arm also gets dropped).
• Merge a patch I got from Myon which will keep uploaders in projectb.
• Enable bzip2 support for sources, not only for the binary packages. I have a patch ready, it’s not yet merged. Needs to test if anything breaks in stable if we apply it, before we can think about it. Worst case it has to wait until we are running lenny. (The archive runs on a machine with stable, so allowed features always have to be supported in stable).
• Process the currently waiting requests for pseudo-packages and either reject or approve them. After thats done - talk to the BTS maintainers to hand the management of pseudo-packages over to them, as 99% of the usage of those is in the BTS.
• Get dak to optionally mail the sponsor of a package too, if not already in the list of those getting a dak mail. Which isn’t easy. One proposal is to just look at the gpg keyid and get the Debian uid from it, send mail there. I don’t want this, as people are, unfortunately, able to disable mail receiving for their @debian.org address. Which would give us lots of bounces. No way we like that. Similar problems with “Just take one userid that looks like a mail address from their key”.
• Improve various parts run by our daily cronjob, as that one takes ages (about an hour today). Speed improvements are always nice there.
• Think about having more dinstall runs a day. Like - every 2 or 4 or 6 hours could be nice. But that needs discussion within the ftpteam and also with the mirror people before it can be done. But 6 hours seems to be something nice to have, more might be also ok.
• Coordinate with DSA to enable the continous AKA WAL archiving for our database.
• Make merkels sync be more instant, ie. no longer rely on a push of a database dump whenever dinstall run, but something thats near realtime. Having changes done on ries immediately synced to merkel.
• More (active) ftp-team member, maybe?!

Now, do you have any time to kill? Do you want to help? Some things you could do:

• Go look at our bug page and start there. Someone, I think Don, already started using user categories. It would be nice if someone is willing to keep up with the bugs and put them in categories. And continue to do that in the future.
• Generally help with our bug page. One nice task some people do, and possibly more could also do, is retitling bugs. Especially removal bugs tend to have a wrong title. The format is pretty specific…
• Feel free to submit patches to me. I promise to look at them without too much delay and at least give you a hint what might be wrong or could be done better. And if it makes sense, doesn’t break the archive, works, and is something dak should have - I will also merge it. Of course trivial patches are way easier in that regard, non-trivial ones should go via our ftpmaster address first, to leave others a chance to comment on. Bonus points if you provide bzr trees one can merge from, and the bzr tree you should work with is http://ftp-master.debian.org/bzr/ftpmaster-dak/ - patches/bzr trees against outdated versions won’t have much luck to get looked at. :)
• Keep me happy. However you do it, just don’t hug me. :)