Guess I got your attention with that title? :) Anyways, britney is no longer. And with her testing died too. Ok, not really, but todays work was hand over all of testing and put it where it should have been for years - in the hands of the release team. For those not following the “inner” workings of stuff:

• Testing is the place where our next stable release gets prepared. It is managed by a software called britney, which was written by Anthony Towns.
• Britney was, until today, run by the ftpmasters. The release team had no way to change anything, besides their hint files. They couldn’t change code, couldn’t rerun it themselve if needed, nothing. (Somewhere in the past, for some period, some of the release team was able to do extra britney runs, AFAIK, but thats long ago).

As the above is surely not nice, especially as Anthony (and as such the one in ftpmaster who really fully knew britney) left the ftpmaster group, all of the code managing testing now lives with the release team. They now just prepare a file in a special format and then tell us (via a ssh trigger) to read that in. Which simply sets testing to the new state. This new way of doing things has multiple advantages:

• The release team can run britney whenever they want to. It no longer needs to wait for a cronjob at some externally defined time or ping an ftpmaster to do an extra run. That should be pretty helpful, especially when we are near to a release again.
• They can do code changes or even replace the current britney with a totally new one. (There seems to be a new one available that does the same as the current one, just way faster. Or even more. Which never seems to have got any comment from our side. Badbad. Well, now they can play with it if they want to.). As long as they output valid data for us to import, it is now completly their decision how it gets generated.

The above action did need surprisingly few code changes in dak. It basically was

• disable the cronjob for britney as user dak
• Add a script that the release team now pushes via ssh. This script reads in the above mentioned special file and sets the database.
• Change process-accepted to write the urgencies to a different directory. And no longer try to mkdir this directory, but warn if it doesn’t exist (or is not writeable by dak) and then use a temporary one for the run.
• Move the transitions.yaml to a different location and add a redirect in apache, so everyone can still access it from outside. (It has to live in a ftpmaster controlled directory, thats all).
• Eliminate a britney call from our daily cronjob.

Comments: 8

I've been using GMail ever since they decided to allow users to download their mail using POP. I was happy reading my email as usual in mutt. The only problem was that I had to log onto the website once in a while to clear out my spam folder and fish out the false positives.

However, as the amount of spam I receive daily started growing exponentially (in part thanks to my involvement in Debian), I became more and more dissatisfied with this solution. I had to login several times a day just to keep the levels of spam down to a manageable level. I tried emailing the GMail support team a few times without success.

The solution came just as I was looking at switching to a different email provider: IMAP access.

Using fetchmail, I can now download my received and sent mail using POP and then download (and purge) the contents of my spam folder. Here are the relevant lines of my ~/.fetchmailrc:

poll pop.gmail.com protocol pop3 user "fmarier@gmail.com" is "francois" password "" fetchall ssl
poll imap.gmail.com protocol imap user "fmarier@gmail.com" is "francois" password "" folder "[Gmail]/Spam" fetchall ssl

I can then run all of my mail through my own SpamAssassin filter (to which I added these excellent custom rules) and sort my mail in one of three folders:

• definitely spam (conveniently located in /dev/null) for a score greater than 12
• probably spam (I need to check those manually once in a while)
• not spam for messages with a score less than 5

This is done using procmail with the following bit in my ~/.procmailrc:

# Use spamassassin to check for spam
# (only check messages 400kB so that it doesn't choke) :0fw:/etc/mail/spamc.lock
* 409600
/usr/bin/spamc

# Throw away messages with a score of > 12.0
:0:
* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*
/dev/null

:0:
* ^X-Spam-Status: Yes
/home/francois/mail/spam

Since I didn't find a similar tool for Linux, I've hacked together a tiny tool to do BPM counting. The usual stuff: tap any key with the music to get a BPM estimation. There is no music analysis done.To achieve better results, my tool uses an exponentially weighted average. It also computes a confidence interval. So if the song changes tempo (and quite some jazz songs do) or doesn't keep the tempo constantly (live music with human drummers often shows some tempo drift. It's just the drum computers that keep their speed very well...) - the tool can handle that quite well, while also not being overly sensitive to tapping errors (as you can see in the image).I figure there might be even better estimation tricks (e.g. doing some error detection and removing them), but this one was just very simple to implement. And by looking at the graph and the error interval it's quite easy to check that you didn't any recent mistakes and the result is okay.The Python code is about 158 lines SLOC, and a small UI description generated with Glade. It's built with PyGTK and uses Cairo for the plots, so it should be very platform independant and even run on Windows.Download will be available sometime when I get around to clean it up a little. There is for example still the code in there to compute plain averages, which can't handle songs with multiple tempos at all.[Update: it now also has a tiny 'flasher' icon that will show you the current tempo estimation. This is great for checking the quality of the result - if it keeps on flashing in sync with the music after you've stopped tapping, it's good. I was also pointed to GNU GTick which is a full featured metronome application and also has a tap button.]

In this post, Steven Hanley wonders how to keep /var/cache/apt/archives empty on a machine that has a full mirror of debian on it. That’s actually the wrong question. The right question is ‘Why is /var/cache/apt/archives even relevant on a machine with a full mirror of debian?’ /etc/apt/sources.list has always supported file:/ URLs. for example, sources.list on my local debian mirror looks like this:

deb file:/home/ftp/debian unstable main contrib non-free
deb file:/home/ftp/debian-multimedia/ unstable main

With a file URL, apt-get doesn’t need to download any packages, and thus doesn’t need to cache anything in /var/cache/apt/archives. No need to keep it empty, it just is empty. In a related post, Andrew Pollock suggests adding DPkg::Post-Invoke “apt-get clean”; ; to /etc/apt/apt.conf. That would be fine if you ONLY ever ran “apt-get update ; apt-get dist-upgrade”. It’s not fine if you run “apt-get -d” to download packages first, then install a few packages with “apt-get install package”, and then run “apt-get dist-upgrade”. If you try that, then the DPKG::Post-Invoke rule will clean out /var/cache/apt/archives when you install the individual package(s). When you then run the dist-upgrade, apt-get will have to download all the packages again. Better just to get into the habit of running apt-get clean or autoclean occasionally when you want to reclaim some disk space. Syndicated from Craig Sanders' Errata: Miscellaneous Thoughts Keeping apt archives empty…or not.

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobh n O Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-). So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun? Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important. Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important. There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project. But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way. Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for. This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario. In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics. Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]“, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand. When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however. So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

I'm about to leave for Monterrey, Nuevo Le n, some 700Km North. Why? Because I was invited to be at the Monterrey FLISOL. And what exactly is a FLISOL? A very nice and interesting idea: Festival Latinoamericano de Instalaci n de Software Libre, Latin American Free Software Install Festival.

So far, I have stayed away from install-fests. I don't like them. And I will keep what I have always said: I am going because I was invited to talk about network security (of course, giving more than a little bit of relevance to Free Software as the IMHO only way to get to a decent level of security). But I do want to be part of this. It is large. Very large. So large, you don't want to miss out.
According to Beatriz Busaniche, FLISOL will be simultaneously held in 210 cities all over Latin America, in Argentina, Bolivia, Brasil, Chile, Colombia, Costa Rica, Cuba, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Uruguay and Venezuela. In short: everywhere.
Spread the word. Spread the love. Spread the fun!

You know you ve had enough to drink when... you have been using mutt and vim with capslock on for half an hour and have even managed, quite confused though.
Heh.

Previously, my journal was managed by Movable Type. Originally, Six Apart seemed like a cool company, but I should have known better than to trust anything that wasn't open source. They started screwing with the license, it was never packaged properly, and then of course it became far too painful to upgrade. I've been wanting to fix this for some time, and have occasionally poked at other blog software. Serendipity looked like a winner for a while, but then I spent two days fighting with it and while I got it to mostly look like what I want, that was demoralizing. I hate spending large quantities of time fighting with someone else's software to get it to do what I want, particularly when I then have to write plugins and additions to get what feels to me like basic functionality. It would be one thing if I had time and inclination to get involved as a developer, but usually what Web 2.0 software developers want to pursue and what I care about are almost entirely at odds. During my recent vacation, I finally realized that I just wanted something that used all the infrastructure I already have. I have a macro language that I like for generating web pages, I have a page design that I like; the journal one isn't bad, but it doesn't match. I don't care about nearly all of the features of blog software. I could use something that's much better-designed (and free software) like ikiwiki, but I don't particularly like wikis either. (That's another story that I should write up sometime.) I really just wanted something that works like my book reviews work and lets me maintain the whole thing in spin. So I just gave up and wrote it. It probably took me more time than it would take to beat some piece of blog software into submission, but now it's done, I know how it works, it knows what I want to do, and I don't have to fight with it again with every upgrade. So this is my first post with my new journal software, with an RSS feed generated by my own code. There will probably be bugs for a while to iron out, but it will also make it far easier for me to maintain multiple RSS feeds with different topics, post reviews to my journal even more automatically, and add automatic posting for other things. The main loss is comments. With this system, my entire web site can be static, which I love. But that means no comment scripts. Since almost no one was commenting on my journal anyway (as opposed to mailing me or discussing posts elsewhere), I think I'm okay with that. Eventually I may mirror my journal somewhere else where there are comments. All of the old posts have been converted over, including old comments. Old reviews with no comments have become redirects to the permanent review pages. Let me know if you see anything weird.

Thanks to Richard Jones, from Red Hat, i have been able to be enough motivated to start thinking to do a new release. Richard provide me with a patch which help to get rid of ocaml-ast-analyze and replace it using camlp4 from OCaml 3.10.1. I spent the last week removing problems concerning the merge action of "ocaml-gettext" executable and reworking the build system to get something cleaner. I will spend a few more days to get rid of camlidl and try to see if there is a simple solution to handle UTF-8 in PO file. Probable release date: 2008/04/27....

New Gammu testing version is finally out. This time it took much longer than usual and it contains some important fixes for messages and AT driver. Full list of changes:

• Fix handling of UTF8 charset with some Motorola phones.
• Identify Motorola V635.
• Set again charset in Motorola phones after changing AT mode.
• Automatically enable OBEX for Motorola phones.
• Fix searching of correct Bluetooth channel.
• Improved SMSD error messages.
• Fix SMS listing when phone was in text mode (bug #584).
• Fix compilation when WCONTINUED/WIFCONTINUED is not defined.
• Fixed listing SMS messages from AT phones in text mode (bug #584).
• Added smscgi daemon by "Kamanashis Roy Shuva" kamanashisroy@gmail.com.
• Cache once read messages to avoid reading them again (bug #481).

You can download from usual place: http://cihar.com/gammu/, Debian users can get packages from experimental.

Today is the second International Pixel-Stained Technopeasant Day. The call is for the publication of professional-quality fiction freely on the web. I’m not a pro, and I am not qualified to judge the quality of my own work, so this quite isn’t properly a IPSTD contribution, but it is offered in the day’s spirit. Enjoy.

---

A MOMENT IN THE LIFE OF A FINNISH MAN
by
Antti-Juhani Kaijanaho
I am lucky we don’t have the death penalty in Finland. She tells me she’s glad of that, too. * * * I am a Finnish man. As a Finn, I am irritated by that they let all sorts of, hm, dark-skinned beasts that masquerade as human enter this country. They just come, you know, take our money, use it to buy apartments, cars. I know one, hm, dark-skin, who lives a billion times billion times better than I, without working! I get to toil for my meager living, and the kids and the wife hoard it as if it were due to them — (more…)

Steven Hanley was pondering how to keep /var/cache/apt/archives empty on the mirror that I help administer. This sounds vaguely similar to mounting the /usr filesystem read-write at the start of an APT run and read-only again at the end. (A practice I used to believe in, but due to various package upgrades making /usr busy for no good reason, and it artificially inflating the maximal mount-count and prematurely causing a fsck at boot, I've discontinued) So, putting

DPkg::Post-Invoke   "apt-get clean";  ;

in /etc/apt/apt.conf (or in a file in /etc/apt/apt.conf.d) ought to do the trick.

I'm happy to point to the announcement that Wikitravel Press now has a guide to Paris. Paris has long been a targeted city for Wikitravel Press, and I'm really happy we were able to get this book out. Bonus: the Wikitravel Guide to Paris includes maps from OpenStreetMap, the Open Content geo database and mapping system. We've got some new software to overlay Wikitravel listings onto the maps, and it's working really well. Thanks to OSM, editor Mark Jaroski, and managing editor Jani Patokallio for getting this great book out... just in time for Paris spring! tags: wikitravel wikitravelpress paris guidebook opencontent openstreetmap

Podcasts I asked a couple of days ago (Journal/30 Germinal CCXVI) about podcasts on the subject of Open Source and Open Content. I wanted to summarize a few that came in through comments on my blog:

• The iCommons Podcast, which (like many podcasts) seems to be running a little slow -- no updates in about a year. Still, an interesting project to be watching. (thanks Brianna)
• Wikipedia Weekly, the really great show by Andrew Lih and Liam Wyatt. I should have thought of WW before -- I was interviewed for the show last year! (thanks Padraic)
• Also worth noting is Not the Wikipedia Weekly, which is, uh, not the Wikipedia Weekly. I got this one from an episode of WW!
• LUG Radio, which I found from another podcast, LQ Radio.

This has turned into a pretty decent list, but I'd love to see some more diverse discussion of Open Content and Open Source in general. Please, feel free to send more tips. tags: podcast linux opensource opencontent

Vinismo For some reason Vinismo went over some tipping point with StumbleUpon this weekend, and we had a real flood of users from that bookmarking site. I can't say why that happens; I find SU to be a totally opaque Web site and service. In other news, Stevey likes our business cards. Me too! I think they look great. M-C Doyon, the Montreal graphic designer who laid out our Web site, also did our paper branding, and I think she did a great job. tags: vinismo stevey marieclaudedoyon cards

Yay Firefox 3.0 I don't know when it happened, but Firefox 3.0 beta does non-ASCII characters in the address bar correctly. So, if you're reading this entry on my site with FF3, you'll see the in "Flor al" rather than the URL-encoded "Flor%c3%a9al". I also like seeing Japanese Wikipedia pages in the address bar showing up correctly. Nice job, FF3 team. tags: firefox firefox3 utf8 url

Planet software should preserve categories So, apparently there's once again rising problems with content drift on Planet Debian. Personally, I think this is a problem with a technical solution. All three main flavours of RSS support post categorization. Many, many kinds of feed software provide categories (see, for example, this feed). If the Planet software would preserve these categories and pass them through to the output RSS feeds, then people who really only want to read about Debian could filter the output feeds for category "debian". People who wanted to know about other parts of their fellow Debianistas' lives would just leave the feeds unfiltered. I'm not sure if the Planet software doesn't support categories in output at all, or if it's just an option that's turned off on Planet Debian. If nobody else wants to take a look, I can look into making this work. It seems like a pretty simple fix. tags: planet planetdebian rss categories

[Warning, long post ahead. If you aren’t interested in Debian or it’s internals or my work - skip it. :) ] So as you might have read here, I got the delegation of the debadmin AKA FTPMaster group lately. I’ve got some pressure to not accept this delegation, but the response to my blog post about it had been extremely positive, everyone said I should not deny it. Knowing to have such a support within Debian is something I really appreciate, making it worth to spent time on my various jobs. Now, lets take this and write a little about what happened since I gained that extra group:

• Aj resigned from anything that requires additional group privileges. (As a DD run id ajt on a debian.org machine if you want to verify this).

Work I did since then:

• One of my first action was to finally finish the release transitions support, something which was waiting for it’s final merge for a long time. The release team was waiting for this and did ping us (ftpteam) multiple times, but now it is there. You can read more about this feature in my list mail and my initial blog post. (Oh, and something I have missed about everywhere I wrote about this: This whole transitions thing is based on an idea from Marga!)
• Added a mapping in dak, requested by Luk
• Merged a patch from another ftpteam member, Thomas Viehmann, which fixes daks handling of Format 3.0 (something) uploads. Currently it rejects them, without the merged patch it just crashed.
• Changed our daily cronscript to no longer VACUUM/VACUUM analyze projectb in every run. Postgres >= 8.1 has autovacuum, and its turned on, so we should be fine without extra runs of it. (And while i write daily cronscript or cron.daily - I mean dinstall, right now run twice a day).
• Fixed a breakage in code introduced by Debian maintainers which led to process-accepted (one main part of the daily cronscript) breaking. It was triggered by some octave-$FOO package upload, where the ###censored### did add someone twice into the Uploaders: field. Not helpful, but now we ignore this case and just warn us.
• Added a script to expire old database dumps. We do dump our projectb (the main archive database) twice in every run (beginning/end), which made 4 dumps a day. While old dumps seemed to have been removed on an irregular schedule - the dumps took about 71Gb in total. Which pretty much filled up merkels /org, as they get synced there. Now, my script (reworked version of a script from Florian Reitmeir) works with an “incremental deletion” scheme:

  RULES = [ ‘days’:14, ‘interval’:0 , ‘days’:31, ‘interval’:1 , ‘days’:365, ‘interval’:31 , ‘days’:3650, ‘interval’:365 , ]

  That means we now keep all files from the last 14 days. Then keep one per week. Then one per month and finally one per year for up to ten years. All the rest gets deleted. Space usage of our dumps now: 4.8Gb. Thats 66Gb gained. Source is in dak bzr if you want it, MIT licensed. If you improvie it further I love to see patches.
• Changed the way the projectb copy on merkel gets synced. Up to now it was run from a cronjob which syncs a number of files people want to see on merkel, including the database dumps, then reloads the database using the latest dump. This was run at a “random” time, scheduled to be shortly after dinstall. Problem with such a setup is that it will miss extra dinstall runs and also might import old data, as the timing of the cronjob meant it runs while dinstall on ftp-master is still doing its work. So bad bad. It is now done using the ssh trigger feature I described a while ago. So our daily cronscript now tells merkel when it should sync its database again.
• Coordinated with the qa group we now also push them, when the sync on merkel is done, so they can kick their cronjobs to reread projectb and just run whenever they get told that something changed. Yet another bad timing eliminated. (Well, will be done today). Oh, if you have something that depends on information from projectb / ftp-master and try to schedule it to run after dinstall - talk to me, we might just push you too.
• Started talking with the release team how britney, the script which manages our testing distribution, should be handled in the future. It currently is run fully under control of us ftpmasters (and reading hint files from the release team). Ideally this gets changed so that the release team has the control of the code and whatever gets run with it and then, in some way, maybe using ssh triggers, tells us to import a new dataset for testing. That needs some thinking, as obviously it should be limited to testing and we should make sure it doesn’t break the archive, but I think that is doable.
• Prepared a patch that sanity checks the Provides: field, something we currently don’t do. Mailed our internal team address about 3 days ago, to let others look at it, no response yet.

The above should nicely sum up my work in the past few days. Most of it pretty trivial, but still things that have to be done. I plan to keep doing things, slowly going from the trivial stuff to more complicated ones, but there is lots of stuff ahead. Some ideas, and this is a very incomplete list just from my head, there are tons of things that one could do, include

• Finally allowing dpkg-sig support in .deb files. I once, years ago, made a patch for it, which never got applied. (Mostly my fault here). Need to rewrite that to fit the actual code.
• Add a few more sections to our archive. There are multiple bugs asking for them, and some others also come to mind. Like java, ruby, octave, haskell, debug to name a few, but there might be more needed. (Yes, longterm all sections should die and be replaced with debtags. I just don’t think debtags is there yet?!)
• Maybe finally drop m68k and add other architectures to the archive, like some $foo*bsd one. (Thats for after lenny of course, at the time arm also gets dropped).
• Merge a patch I got from Myon which will keep uploaders in projectb.
• Enable bzip2 support for sources, not only for the binary packages. I have a patch ready, it’s not yet merged. Needs to test if anything breaks in stable if we apply it, before we can think about it. Worst case it has to wait until we are running lenny. (The archive runs on a machine with stable, so allowed features always have to be supported in stable).
• Process the currently waiting requests for pseudo-packages and either reject or approve them. After thats done - talk to the BTS maintainers to hand the management of pseudo-packages over to them, as 99% of the usage of those is in the BTS.
• Get dak to optionally mail the sponsor of a package too, if not already in the list of those getting a dak mail. Which isn’t easy. One proposal is to just look at the gpg keyid and get the Debian uid from it, send mail there. I don’t want this, as people are, unfortunately, able to disable mail receiving for their @debian.org address. Which would give us lots of bounces. No way we like that. Similar problems with “Just take one userid that looks like a mail address from their key”.
• Improve various parts run by our daily cronjob, as that one takes ages (about an hour today). Speed improvements are always nice there.
• Think about having more dinstall runs a day. Like - every 2 or 4 or 6 hours could be nice. But that needs discussion within the ftpteam and also with the mirror people before it can be done. But 6 hours seems to be something nice to have, more might be also ok.
• Coordinate with DSA to enable the continous AKA WAL archiving for our database.
• Make merkels sync be more instant, ie. no longer rely on a push of a database dump whenever dinstall run, but something thats near realtime. Having changes done on ries immediately synced to merkel.
• More (active) ftp-team member, maybe?!

Now, do you have any time to kill? Do you want to help? Some things you could do:

• Go look at our bug page and start there. Someone, I think Don, already started using user categories. It would be nice if someone is willing to keep up with the bugs and put them in categories. And continue to do that in the future.
• Generally help with our bug page. One nice task some people do, and possibly more could also do, is retitling bugs. Especially removal bugs tend to have a wrong title. The format is pretty specific…
• Feel free to submit patches to me. I promise to look at them without too much delay and at least give you a hint what might be wrong or could be done better. And if it makes sense, doesn’t break the archive, works, and is something dak should have - I will also merge it. Of course trivial patches are way easier in that regard, non-trivial ones should go via our ftpmaster address first, to leave others a chance to comment on. Bonus points if you provide bzr trees one can merge from, and the bzr tree you should work with is http://ftp-master.debian.org/bzr/ftpmaster-dak/ - patches/bzr trees against outdated versions won’t have much luck to get looked at. :)
• Keep me happy. However you do it, just don’t hug me. :)

...is just not enough time to see the place properly. I took Friday and Monday off, and Thursday night a week ago, Sarah and I flew down to San Diego for a bit of a "Yay, we've achieved normality" escape. We'd intended to go to SeaWorld on Friday, vege out for two days, go to San Diego Zoo on Monday, then fly home on Monday night. Well we did SeaWorld and the zoo as planned, but lazing around the pool never happened, we ended up running ourselves ragged exploring San Diego all day Saturday and Sunday instead. We stayed at The Dana on Mission Bay, which we picked because it was the closest to SeaWorld. The accommodation was more like a motel than a hotel. There were numerous two-storey buildings across a fairly sprawling area. It had frontage onto what I presume was Mission Bay, and also had a Marina attached to it, so the outlook was quite nice. It also had free WiFi, so we could upload photos throughout the stay. The food offerings were pretty good, and reasonably priced as well. SeaWorld was really good. It's been a long time since I've been to the Australian equivalent, but I don't remember it as being as interactive. San Diego's SeaWorld had a Bay Ray feeding area, where you could purchase food (small whole fish), and hand-feed the Bat Rays. That was pretty cool. It took a bit of a courage to leave your hand in the water, palm up, with something the size of sardine dangling between your fingers and let these huge rays literally swim right over your hand so they could suck the fish from between your fingers. There was also a dolphin feeding area, where you could purchase some small fish again, and give the dolphins a pat on the head and then throw them a fish. I really love dolphins, and one day I'd like to be able to swim with them. The main attraction of SeaWorld is the various shows that they do. I think we caught all of the different ones. The killer whales are just amazing. There were a few different programs where for an additional fee, you could swim with various animals. Dolphins and Beluga whales seemed to be the ones we noticed in particular. There were a few oddities, though: The park is owned by a beer company, so there was a part where you could go and get free samples. Two per person per day (I think on the honour system). At least at the food outlet where we bought lunch (they don't allow you to bring in any food or drink from the outside "for the safety of the animals", so they have a nice monopoly on catering) every kid's meal came in a commemorative Shamu blue plastic lunch pail - whether you wanted it or not. There were two empty ones left on the table that Sarah and I sat down at. The lady cleaning up the tables asked us if we wanted them, and we said no, and she promptly chucked them out with the trash. This just struck me as a terrible waste. They obviously cost money and energy to produce, and they're just going to end up in landfill. They could have been washed and reused instead. I felt this was very hypocritical for a park that was trying to send people away with a message about conservation. Other than this nitpick, I thought SeaWorld was really great. It was a good size, and it was doable in one day, in their normal opening hours. I guess it'd be a bit slower with kids. Photos from the day are here. The next two days we spent exploring San Diego. The hotel was conveniently located on an MTS loop, which ran surprising frequently for a weekend (at least compared to public transit in the Bay Area), so we bought day passes on Saturday and Sunday and used it to get to the Old Town transit centre. On Saturday we explored Old Town, which was a historic preservation of how San Diego looked "back in the day". In the afternoon, we also bought 48-hour tickets for the red double-decker "hop on, hop off" tour bus, and did one of the two loops that it offered. The next day, we caught the loop in the opposite direction, and checked out Mission Beach, which was probably the best beach I've seen in California so far (although the water was still far too cold), saw some pretty cool alternative accommodation (possibly for next time), and did part of the other loop on the tour bus, getting off at the USS Midway. This massive retired aircraft carrier is permanently moored at San Diego, and for a fee, you can crawl all over most of it. It was very interesting, and gave a good insight into the life of a sailor. We blew a good 4 hours or so here, and ran out of time to see the entire thing (we didn't make it onto the "island" part of the ship, which we were a bit bummed about). We then caught the trolley back to Old Town, and grabbed a beer at a pub that sold it by the yard glass and half yard glass. We opted for the half yard glass, since the full yard glass seemed a bit unwieldly, and we thought it looked like a hell of a lot of beer, but the half yard glass, whilst also looking like a lot of beer, was only about one and a half pints. We then grabbed some dinner at one of the very authentic looking Mexican restaurants (they had women out the front, almost on the sidewalk making fresh tortillas on the spot) and headed back home. Photos from our exploration are here. On the last day, we went to the San Diego Zoo (which every time I read the URL for, I read it as "Sandie go Zoo!"). This zoo is purportedly the best zoo in the country, but it really didn't blow my socks off. We seemed to have a really hard time navigating the place, and spent a lot of the day walking around in circles trying to find various exhibits. It's also in a bit of a valley, so the circles tended to be up and down hills, which was tiring. I really don't think we were very efficient at all in our coverage of the place. The other thing that I personally found annoying was the cages. The wire was very close together, which made it really hard to take photos, because the camera would keep focusing on the wire, instead of what was behind it. I guess this is why God invented manual focus, but that made it very hard to photograph big cats stalking their cages. The photos we did manage to get are here. Overall, we had a great time in San Diego, and it seems we didn't really scratch the surface. I'd like to go back again and see Balboa Park, the Gaslamp district, and I thought there were more naval vessels that you could look at, but I might be mistaken. Definitely a very nice city.

And fisl9.0 is over. This was a great one not only because of its size, but there were so many things at stake that I am positively surprised such a few went wrong. We ve got a lot to think until next one To begin with, there s a major difference between this inter-fisl time: we already know where next s going to be and how much it ll cost us. The event-center cost, as you can imagine, is the most expensive part of our budget, and knowing that in advance, I believe, will play an important role in our planning. Also, we got more time to plan for the next one: it ll be in June, not April, as usual. Moving back to PUCRS (from FIERGS) as fisl s event-center now I can say was scaring me to death. PUCRS is smaller and we were foreseeing a huge event. Guess what! The event was bigger than first imagined, and PUCRS held it without major drawbacks. Sure, lectures were crowded but they were already crowded at FIERGS anyway The expo had more people than FIERGS, but I am sure our sponsors did not think that was a problem ;-) Of course, not everything went OK I think we had two important problems to solve for next fisl: our network must be born already correctly configured for the streaming, and our wireless infrastructure must be stronger. The first one I already wrote about and I think it had to to with the inexperience of the network people this year: apparently they ve put everything under a crappy load-balancing and while this might work for web-surfing or sending small amounts of data, it just hurts anything that should be continuously transfered, like real-time videos. Here is a piece of advice: network planning involves the right amount of everything (the right amount of cables, the right amount of bandwidth, the right amount of uplinks), including the right amount of configuration! I think our network problems were more a case of overconfiguration than one of lack of skills. The problem with the wi-fi is deeper. Maybe I don t completely understand the technology (and people handling it were not very helpful). How do you manage an event like fisl in the wireless front? PUCRS have provided us with enough wi-fi to hold 1500 concurrent connections (so they say), and have reported we never had more than 400 concurrent connections and yet, as many people have reported, to connect our laptops was a hassle. I don t have a laptop, but I ve seem a lot of my friends trying to connect with no luck. During the second day, PUCRS wi-fi technicians told us that the problem was the Access-Points people brought to fisl (according to them there were more than 30) that were, somehow, messing with the channels they were using. So we reserved some channels to PUCRS and asked people to use other ones. I tend to believe that the majority of the people listened to our appeal and reconfigured their Access-Points but no improvement were reported. So, either PUCRS cannot hold more than 400 concurrent connections in their wi-fi system, or there s some other problem we ve not identified. What is the right thing to do? To forbid people to bring their own Access-Points? No way I am going to see the day fisl organizing committee will disallow people to carry their own communication devices!! There must be a better solution I think I am going to study wi-fi at least not to be easily fooled around by this messing the channels crap (please, comment to this article if you have any ideas). Finally, people are dreaming of a 10-thousand people fisl next year. I think it s not more than a dream but hey! When we made the first fisl, there were only 300 people it has been a 9-year long dream Maybe that 10-thousand one will come true, as the previous ones did. I will just add one more problem to our list: the first day long lines to get people s badges must be addressed in a creative way Thanks to all that came to fisl. See you next year!