Global trends For some time I have been trying to pinpoint what it is that is brewing in Italy and risks spreading elsewhere, like it happened in the past. I don't need to be decent to stay in power While following a train of thought during a political/philosophical lecture I figured that a current growing trend is to have public figures that are more and more indecent. In Italy it is very hard to find a public figure you can look up to. It is hard to name a politician that is not involved in some shady exchange of favours or some abues of their powers, and we got used to seeing people in power implicated in major corruption scandals, perverted prostitution affairs, or dealings with international criminal organisations. They do not normally end up in jail, and in fact they keep being very firmly in power, because they manage to stretch or change the laws to get away, or at least to delay trials in order to trigger some statute of limitations. Is there a pattern here that, although maybe not as clearly defined as in Italy, can be found more or less globally? Yesterday I thought that this could be such a pattern:

I don't need to be decent to stay in power

If I think of it like that, then it is most definitely not just an Italian phenomenon. If you tell "one doesn't need to be decent to stay in power" to a British, or to a French, I would not expect them to see anything strange with it. We all find it depressing, but we are all used to it. It is a pattern with repercussions, though: once that becomes normal in a society, it means that people who get to be in power are free to abuse their power much as they want, as long as they are careful enough not to end up in jail. Because, well, nowadays one doesn't need to be decent to stay in power. I don't need to follow the law to stay in power That first pattern is already quite well accepted in Italy. So much well accepted, that I think we are starting to see what comes next. At the end of March we are going to have elections for some regional governors. Funnily enough, in Lazio, the very important region around Rome, the centre-right coalition failed to submit the paperwork on time, and is out of the elections. It is not just red tape: at some point someone will have to print out the ballots and dispatch them to the voting booths, so one expects to have the coalition logos and the names of the candidates submitted in time, together with signatures supporting the candidates and whatever else the election process needs. Well, they missed the deadline, they got there after closing time and the building was, well, closed. It was a fantastic opportunity for a laugh. Memes blossomed on the Italian intarwebs and we now have 2 or 3 new expressions to mean "stupid". However, now it's hard to tell what is going to happen. On one hand, you can't exclude one of the two major coalitions because of some bureaucratic detail like an office closing time. But on the other hand, several minor coalitions have been excluded in all sorts of past elections because of similar things, and it really would not be fair to start making exceptions now. But Lombardia, the region around Milan and Emilia Romagna, the one around Bologna, both very, very important, are having similar kinds of problems. In both regions the previous governors are running again, for the 3rd time in a row, and most likely they legally can't do it, and if elected one can sue and force them to resign, because they have been in power long enough. Lots of paper is being shuffled at the moment to figure if they can get away with it or not. Oh, and the lawyers of the candidate for the Milan region also managed to get to the tribunal after closing time, but apparently there was still someone inside and they managed to shout loud enough, or somesuch. Anyway, the situation is getting hot. The Lazio coalition that has been excluded because of their incompetence is now hard at work pushing their potential voters to mount a fracas. Chances are that eventually they'll get away with it, and manage to take part to the election. If that happens, they will likely get close to winning it. So this seems to be a new pattern that is emerging:

I don't need to follow the law to stay in power

Which, again, is a pattern with quite some repercussions. It is something much more radical than just an issue with morality: it means feudalism, it means we are culturally ready to accept dictatorship. So, please do me a favour: do not think for a moment that Italy is just a funny place with lemons and tomatoes, and watch out for these patterns emerging around you.

From RFC3164, which is otherwise about syslog formats:

6. Security Considerations An odor may be considered to be a message that does not require any acknowledgement. People tend to avoid bad odors but are drawn to odors that they associate with good food. The acknowledgement of the receipt of the odor or scent is not required and indeed it may be the height of discretion to totally ignore some odors. On the other hand, it is usually considered good civility to acknowledge the prowess of the cook merely from the ambiance wafting from the kitchen. Similarly, various species have been found to utilize odors to attract mates. One species of moth uses this scent to find each other. However, it has been found that bolas spiders can mimic the odor of the female moths of this species. This scent will then attract male moths, which will follow it with the expectation of finding a mate. Instead, when they arrive at the source of the scent, they will be eaten [8]. This is a case of a false message being sent out with inimical intent. ... Along the lines of the analogy, computer event messages may be sent accidentally, erroneously and even maliciously.

This smells more like "I bet nobody ever really reads this RFC, let's put some stuff in the middle to see if they do!".

Yes, my rant was more against the things I encountered during keysigning, not keysigning itself. However, I still feel X.509 doesn't have these problems, and nothing I can think of will persuade me to think PGP/MIME better than Inline PGP. (Especially when the recipient's key contains a notation that he wants the latter, but not the former.) Jonathan does have some good points about the (PGP) Web of Trust. Again, that wlog entry of mine was a rant; I had let the topic stew over night, trying to get the anger out, but on the next day it just wanted to get out of me, I merely wanted a good old rant. I think I'll not include Planet Debian next time I do rant, though (it's not the place to do so).

The Opera-Mini Dispute I have just read an interesting article about the Opera browser [1]. The article is very critical of Opera-Mini on the iPhone for many reasons most of which don t interest me greatly. There are lots of technical trade-offs that you can make when designing an application for a constrained environment (EG a phone with low resolution and low bandwidth). What does interest me is the criticism of the Opera Mini browser for proxying all Internet access (including HTTPS) through their own servers, this has been getting some traction around the Internet. Now it is obvious that if you have one server sitting on the net that proxies connections to lots of banks then there will be potential for abuse. What apparently isn t obvious to as many people is the fact that you have to trust the application. Causes of Software Security Problems When people think about computer security they usually think about worms and viruses that exploit existing bugs in software and about Trojan horse software that the user has to be tricked into running. These are both significant problems. But another problem is that of malicious software releases. I think that this is significantly different from Trojan horses because instead of having an application which was written for the sole purpose of tricking people (as is most similar to Greek history) you have an application that was written by many people who genuinely want to make a good product but you have a single person or small group that hijacks it. Rumor has it that rates well in excess of $10,000 are sometimes paid for previously unknown security vulnerabilities in widely used software. It seems likely that a programmer who was in a desperate financial situation could bolster their salary by deliberately putting bugs in software and then selling the exploits, this would not be a trivial task (making such bugs appear to be genuine mistakes would take some skill) but there are lots of people who could do it and plausibly deny any accusation other than carelessness. There have been many examples of gambling addicts who have done more foolish things to fund their habit. I don t think it s plausible to believe that every security flaw which has been discovered in widely used software was there purely as the result of a mistake. Given the huge number of programmers who have the skill needed to deliberately introduce a security flaw into the source of a program and conceal it from their colleagues I think it s quite likely that someone has done so and attempted to profit from it. Note that even if it could be proven that it was impossible to profit from creating a security flaw in a program that would not be sufficient to prove that it never happened. There is plenty of evidence of people committing crimes in the mistaken belief that it would be profitable for them. Should We Trust a Proprietary Application or an Internet Server? I agree with the people who don t like the Opera proxy idea, I would rather run a web browser on my phone that directly accesses the Internet. But I don t think that the web browser that is built in to my current smart-phone is particularly secure. It seems usual for a PC to need a security update for the base OS or the web browser at least once a year while mobile phones have a standard service life of two years without any updates. I suspect that there is a lot of flawed code running on smart phones that never get updated. It seems to me that the risks with Opera are the single point of failure of the proxy server in addition to the issues of code quality while the risks with the browser that is on my smart-phone is just the quality of the code. I suspect that Opera may do a better job of updating their software to fix security issues so this may mitigate the risk from using their proxy. At the moment China is producing a significant portion of the world s smart-phones. Some brands like LG are designed and manufactured in China, others are manufactured in China for marketing/engineering companies based in Europe and the US. A casual browse of information regarding Falun Gong makes the character of the Chinese leadership quite clear [2], I think that everything that comes out of China should be considered to be less trustworthy than equivalent products from Europe and the US. So I think that anyone who owns a Chinese mobile phone and rails against the Opera Mini hasn t considered the issue enough. I don t think it s possible to prove that an Opera Mini with it s proxy is more or less of a risk than a Chinese smart-phone. I m quite happy with my LG Viewty [3] but I wouldn t use it for Internet banking or checking my main email account. Also we have to keep in mind that mobile phones are really owned by telephone companies. You might pay for your phone or even get it unlocked so you can run it on a different network, but you won t get the custom menus of your telco removed. Most phones are designed to meet the needs of telcos not users and I doubt that secure Internet banking is a priority for a telco. Update: You can buy unlocked mobile phones. But AFAIK the Android is the only phone which might be described as not being designed for the needs of the telcos over the needs of the users. So while you can get a phone without custom menus for a telco, you probably can t get a phone that was specifically designed for what you want to do. The Scope of the Problem Mobile phones are not the extent of the problem, I think that anyone who buys a PC from a Chinese manufacturer and doesn t immediately wipe the hard drive and do a fresh OS install is taking an unreasonable risk. The same thing goes for anyone who buys a PC from a store where it s handled by low wage employees, I can imagine someone on a minimum income accepting a cash payment to run some special software on every PC before it goes out the door that wouldn t be any more difficult or risky than the employees who copy customer credit card numbers (a reasonably common crime). It s also quite conceivable that any major commercial software company could have a rogue employee who is deliberately introducing bugs into it s software. That includes Apple. If the iPhone OS was compromised before it shipped then the issue of browser security wouldn t matter much. I agree that having the minimum possible number of potential security weak points is a good idea. They should allow Opera Mini users to select that HTTPS traffic should not be proxied. But I don t think that merely not using a proxy would create a safe platform for Internet banking. In terms of mobile phones most things are done in the wrong way to try and get more money out of the users. Choose whichever phone or browser you want and it will probably still be a huge security risk. Harald Welte is doing some really good work on developing free software for running a GSM network [4]. But until that project gets to the stage of being widely usable I think that we just have to accept a certain level of security risk when using mobile phones.

• [1] http://counternotions.com/2010/02/18/mini/
• [2] http://www.faluninfo.net/
• [3] http://etbe.coker.com.au/2009/05/31/lg-u990-viewty/
• [4] http://laforge.gnumonks.org/weblog/2010/02/13/#20100213-six_weeks_to_bcch

During recent weeks, not only sbuild and buildd were changed, but also wanna-build. Many changes were small and don't have direct impact, but will ease our life in future. This includes a bunch of code cleanups. Most changes were done by Kurt Roeckx and me, but as usual Marc Brockschmidt and Philipp Kern were also involved. This round of changes was started with redoing our priority calculation. Up to now, any package had a fixed place in the list, and our list was built from top to bottom as far as buildd power was available (putting aside manual intervention by setting build-priority by admins). That meant of course that some packages could be stalled if buildd time isn't enough anymore, like currently on mipsen. (The queue order was determined by the following sort options: build-priority, (>= standard?), already built in the past, priority, section, name, and the first difference decided list order.) Now, of course >= standard packages are still built first, but waiting days increase priority so that old extra packages could be built before young optional package (in other words, they shouldn't stall. The new formula is about: required: 50, important: 40, standard: 30, optional: 5 [priority] + libs: 4, devel: 2 [section] + contrib: -20, non-free: -40 [component] + out-of-date: 20 [notes] + max(6, waitingdays) * 2 + manual priorities, and packages are ordered by this number, then by waitingdays, then by name.) While adding code to add bonus for long-waiting packages, we stumbled across the fact that there were non-C dates in the database stored, which in turn means that export of the database stopped to work. For fixing that we replaced the last change field in the database by an postgres now() on insert, and converted that field to an date field (instead of freetext). Which in turn broke mkstats and a few more things, which are fixed as of now. While doing that, we also introduced the format option, which allows to do queries like:

wanna-build --format='%t %u: %p/%v% +b B%B' -A mipsel --list=building

which gives output like:

2010-03-03 15:24:38.642988 buildd_mipsel-mayer: cracklib2/2.8.16-1
2010-03-03 15:30:00.341313 buildd_mipsel-rem: liblouisxml/2.1.0-1+b1

Of course, there are even better possibilities what one could do with that. :) More changes are pending, like the injector for log files was changed so that we record building times in the database. This will allow us to include build time on at least a few buildds, so that large packages cannot so easily stall all buildds completely anymore. So, more to come ...

(This has ended up longer than I intended, largely because I felt I should then get into why. I'm aware I haven't got into all the nuances, so I hope readers familiar with the area will appreciate this is the compact version.)

Thorsten had a rant last week about PGP keysigning problems. He apologises for his tone, but that's not the issue I take with his rant.

It starts "Keysigning is useless". And yet his complaints seem to be:

• Dealing with the private half of your GPG key securely involves some faff (in this case booting with a live CD and having to set things up ready to keysign).
• He doesn't get on with caff.
• People reject email from machines with invalid HELOs and perform other anti spam measures on ISP access ranges (I'm not clear if it's just greylisting or outright rejects as that's not made clear).
• PGP/MIME is a protocol violation (yes, but it's much better than inline OpenPGP. Unless you have to deal with RT, which mangles it. *sigh*)

None of these seem to actually be about keysigning being useless. The process of doing it, maybe, though he misses the main valid rant about this I'd have, which is that most mass keysignings don't actually allow you to accurately verify the identity of other participants unless you already know them reasonably well. (The LCA2010 keysigning and DebConf5 in Helsinki spring to mind as 2 good examples of bad keysignings I've attended, but speaking to others suggests it's far from an isolated thing.)

Torsten does say that he'll continue to do keysigning on a per-person basis, so it doesn't sound like he's completely given up. I'm posting this largely so other Debian related people don't get the idea that it's not important to think about keysigning.

Why should we care?

Firstly, let's clarify what I mean when I sign someone else's key. If I sign your key then I think that I believe you hold the private part of a key that has your name and an email address I believe I can use to contact you on it. It means I have seen government issued ID that matches that name. It also means that I have interacted with you (and watched others interact with you) under that persona. In short I am happy that the key is a reasonable digital representation of your identity - something signed by it either comes from you or has involved the key being compromised or you coerced into using it against your will.

Why is this useful?

It gets useful thanks to the web of trust; ie the idea that there are a bunch of people I trust partially to sign other keys, and if enough of them have signed a key then I can have a reasonable expectation that the key belongs to the person I want to talk to. Which means I might be prepared to send private data to them. Or Debian might be prepared to accept an upload from them. Which, when you're dealing with a community that spans the planet and where most of the contributors haven't met each other, is pretty freakin' useful - I, as part of Debian's keyring team, don't need to personally be able to identify every Debian developer. All I need to do is be able to trust other DDs to be able to do so. (Though maybe I'm missing out on something here - perhaps Debian should be paying for Gunnar and me to travel the world verifying fingerprints. \o/)

(I still do mass keysignings btw. I'm picky about which keys I actually sign - this is in no way intended as a slight against those I don't, but a mass keysigning at least lets me know that the people involved are happy to exchange fingerprints. Though, FWIW, I normally have ID on me and frequently have fingerprint slips, so if you know me and want me to sign your key/want to sign mine then by all means ask me when you see me!)

Or should I say *we* are going to Debconf 10? Last week-end, I booked the whole set of plane tickets for the Perrier family to fly to USA next summer:

• Jean-Baptiste (aka "Habanero" on IRC) and myself will from from Paris Charles de Gaulle on Wednesday July 28th, arriving in NYC after a connection in Dublin (shame that connection flights are much cheaper than direct flights). As I already wrote earlier, I'll attend Debconf with my son, yay!
• Elisabeth, Sophie and Magali will fly the same route on Aug 5th. We'll stay partly separated during the first days: JB and I at Debconf and them doing some sightseeing in NYC.
• On Aug 8th, we'll start ou family holidays: first stay a 1 or 2 more days in NYC, sightseeing together
• Later on, we plan some trip in the NE part of USA: probably something around Boston and New England, wandering around undertermined places in NY and Vermont states, maybe up to Montreal...then back to NYC
• Finally 5-6 days in our family in the Kansas City area
• Then back to Paris, via New York again, on Aug 23rd and 24th

Nice.

The Command-Line Interface for the KDE Wallet, Version 2.02, has been released and dput into Debian unstable. (The lenny-backports version will follow.) It took me quite a while to reproduce, then track down, the bug; having unrelated problems at the same time didn t help either

But speaking of that I must confess, that by mistake I deleted one of the mails asking for a such a free ticket, instead of replying. I'm very sorry. So if you come from the Netherlands, asked me for a ticket, and haven't got an answer, yet, please use one of the following links:

• Ticket One
• Ticket Two
• Ticket three

I hope, one of the links will work long enough, till the intended recipient reads this ;)

Experimental 2.6.33 will do the switch to UUID based root args, if you haven't switched already. Please test it out and report bugs on it, before we add the libata switch to squeeze 2.6.32. 2.6.32-9 includes 2.6.32.9 and several other fixes. For the following 2.6.32-10 Ben Hutchings pulled in newer drm for lots of intel fixes: Status of kernel X drivers. It also features radeon and nouveau KMS modules. Current 2.6.32 is stabilizing well and we are seeing lots of external patches lining up. Update: Sorry due to jet lag got aboves version number wrong. You might want also to checkout: The season I have mostly been building kernels.

Sorry; I didn't made it in the promised 15 minutes (as someone turned the internet off), but here is finally the link to my (German) slides of my Debian GNU/kFreeBSD talk. (Sorry, no fancy website arround that, yet.) I would also like to make clear, that Debian does not plan to drop it's support for the Linux kernel. Debian will continue to release a linux based distribution. You may install the very same distribution based upon a FreeBSD kernel. It's your choice. A several people asked that at our booth, I guess there is somewhere a missunderstanding, but I don't know the root if that missunderstanding. If someone could tell me, why he thought so (maybe I missed a unclear formulation somewhere?) I would be very glad and try to improve it. So, thanks again for the kbsd people, who helped alot and really tried to answer every stupid question I came up with during preparing my talk :)

Today, I arrived at the CeBIT conference in Hannover, and had a first look around. I find trade fairs like that quickly boring, and I was glad to meet some some other Debian folk and listen to some of the talks at the CeBIT Open Source Forum in hall 2, including tolimar s talk about Debian GNU/kFreeBSD. Tomorrow (Wednesday), I will talk at the same place at 13:45, explaining some basic stuff about patches and bug tracker. The target audience are users of Free Software who modify it for their private or company-wide use and would like to see their changes included in the official project. There will be a live stream of the talk, which I am officially holding as an employee of the ITOMIG GmbH.

Nathanael Boehm wrote a nice essay last month called The Future of Employment?, about a disconnect between workers' and employers' views of social networks. (This post is based partly on my comment in the ensuing Hacker News thread.) Boehm wrote:

When I need help with a challenge at work or need to run some ideas past people I don t turn to my co-workers, I look to my network of colleagues beyond the walls of my workplace. Whilst my co-workers might be competent at their job they can t hope to compete with the hundreds of people I have access to through my social networks...

The late Sun Microsystems taught us that the network is the computer. It's true: we still use non-networked computers for specialized tasks, but nobody wants one on their desk it's just so useless compared to one that talks to the entire world. Boehm could have titled his essay The Network is the Employee. There are still tasks that people do in isolation, but the ability to contact a network of peers and experts makes the difference in my job, and many others. Alone together The lone computer programmer in a small business has thousands of colleagues on Stack Overflow, Reddit, and so on. It's a chaotic and messy way to find answers, but it's better than the days when your only choice was to call tech support or smack the box with your fist, whichever seemed more useful. I can't begin to list all the problems I've solved and things I've learned by Googling for others with relevant experiences, and getting help from a different expert for every problem. Decades before the web, computer geeks had virtual communities for on mailing lists, Usenet, and IRC. Now any job in the world has an online forum. Even the night clerk at the gas station has Not Always Right. Teaching has long been a solitary profession. Despite working in a crowded classroom, teachers are isolated; they rarely have colleagues observing or participating directly in their work. This has such an impact that teacher education sometimes includes training in meditation or reflection, to compensate for lack of external feedback. So I'm really curious what happens when teachers start to work together remotely the way programmers do. You will be assimilated Boehm's essay reminded me of a vague sci-fi-like idea I've been kicking around: the first group minds will evolve from the intersection of Mechanical Turk, virtual assistants, social networking, and augmented reality. Starting around the 1990s, it was possible to instantly "know" any fact that was published online. Since then, we've increased the amount of content online, our tools for searching it, and ways of connecting to the network. Today we have instant access to almost any published knowledge, anywhere. The number of people on the net has grown too, and the number of ways to find and talk to them. Most of us can contact dozens of friends at any given moment, plus friends-of-friends, co-workers, fellow members of communites like Hacker News or MetaFilter, and also complete strangers. Along with raw facts, we now have access to vast amounts of human judgement, experience, and skill. One result of this is the "virtual assistant," who provides a service that was once available only to high-powered executives. The new personal assistant can work remotely (often overseas), spread costs by serving many masters, and leverage the internet superpowers listed above. Today their services are targeted at small business owners and the Tim Ferriss crowd, but I'm sure someone soon will start marketing virtual personal assistance to all sorts of other creative workers, teachers, even stay-at-home parents. So, how long before I can simply touch a button to let a remote assistant see what I'm seeing in real-time and help me make transportation plans, translate foreign signs and speech, look up emails related to whatever I'm doing or thinking, or even advise me on what to say? Some of these queries will go to my circle of friends, others to the general public, and some to a personal assistant who is paid well to keep up with my specific needs. And that assistant of course will sub-contract out portions of each job as needed to computer programs, legions of cheap anonymous Turkers, or to his or her own network of assistants. At that point, I'm augmenting my own perception, memory, and judgement with a whole network of brains that I carry around ready to engage with any situation I meet. If nothing else, I hope someone writes a good sci-fi thriller story in which a rogue virtual assistant subtly manipulates the actions of unknowing clients, leading them to some unseen end.

I've been playing around a bit with Geo-Temporal visualization. Here's a screenshot of an experimental visualization on Google Maps:The icons are placed on approximate coordinates; multiple events in a small area are aggregated into a single marker. The red sectors correspond to temporal information: to the right is the current day, a full turn corresponds to a duration of 7 days. Typical events listed on this map cover 1 to 4 hours in the evening of a day, resulting in a rather small sectors in typical angles corresponding to the seven days of a week. There are three larger events, one being a weekend workshop in Hamburg (covering the saturday and sunday sectors), a Friday to Saturday in Leipzig and an event incorrectly set for all tuesday in Dresden. M nchen on the other hand seems to take a day off on Saturday (in fact they have a full-week workshop on Lanzarote, on a part of the map not shown ...).While this visualization is quite fancy and can scale to arbitrary time window, I will not be able to add it to the public version of this map (which can be tried out on http://swing.vitavonni.de/).The rendering of so many polygons with Google Maps is just way to slow for all the browsers I tried. Maybe I could use cached png images instead and traditional overlays to improve performance.For some visualizations, it would also make sense to turn the sectors into a spiral, for example where the angle corresponds to the day of the month and the distance from the center corresponds to the month.

For my research at the university, I've become a lead developer for ELKI, a framework for developing data mining algorithms along with index structures, to be able to one one hand quickly implement new algorithms (by being able to reuse a lot of code, in particular index structures, parsers, ...), but also on the other hand to evaluate the interaction of index structures with different algorithms, distance functions, and so on. The new version 0.3 which adds a lot of new outlier detection methods will be published beginning of April at DASFAA 2010. (Note that this is designed for research and teaching use; code extensibility, readability etc. instead of maximized performance. You might want to do a rewrite in C for maximum performance, since Java does give you quite a memory and performance overhead in these setups.)After this release, we will be doing a major redesign of the database and index layer, to allow better comparison of different index structures in parallel; right now it's hard to use more than one at a time, and building e.g. a combined index structure is a larger effort than we'd like it to be. During the process of redesigning the database layer, I'll also be improving the database update or "streaming" API.If you know of a nice API for streaming databases, please send me an Email to erich -at- debian -dot- org.Note that I'm looking for a programming API, i.e. "interfaces". Not for random data sources such as Twitter. Also I really need an API able to model data changes, not just "events" aka "instances". So please don't point me to what Twitter calls a "streaming API". What I'm looking for is a nicely-designed API to allow programmers to react to Database changes, such as insertions, deletions, updates, bulk operations etc. and update their index structures and algorithm results accordingly. An example would be the "oracle streams" API I believe; MySQL probably has a logging (used in their replication hacks) which can also be seen as a database stream. But these are designed around a RDBMS view, and not so much for data mining. Weka/MOA Streams seems to be just event/instance streams, where there is no such thing as a bulk insert or even a deletion. Of course there are many use cases where you will be happy with working on just the previous n instances. The more general case however handles arbitrary inserts and deletes, instead of having just inserts (and implicit delitions by an expiry strategy). And yes, of course you can in turn wrap database events as instances into an event stream (with "insert", "update", "delete" events) ...Yes, I'm aware that this kind of setup was abandoned for what is called a "stream processing engine" (SPE) in many use cases, that do not care about "old" data or deletions in general. We'd like to be able to support both approaches, also to be able to do fair comparisons.Of course you can also point me to badly done APIs, and explain me where they fall short for you. We're not so much interested in copying some API, but we'd just like to design a good API for people to do research on stream processing in a database context (e.g. index support for streaming data, online algorithms, ...) Or you can write me a mock-up API that you would deem useful.

Sorry, I didn't had the time to update RC Bug statistics last week, and will probably also not be able this week, as I'll be quite busy, as CeBIT is about to start tomorrow, and I haven't written a script to create the statistics, yet. While we are talking about CeBIT, you might already know, that Debian will again be present there. This as partner of Univention, a German company using Debian as base for its own distribution. I would like to thank them very much for their short hand acceptance of us at their booth. If you intend to visit us (Hall 2, stand B36) and still have not ticket, feel free to drop me an E-Mail. We got some free tickets and I'll try to send you the link (you'll need to register at the CeBIT website and print your ticket yourself).

Apologies for not posting in a while. I've been a little busy of late, but I promise to fill in some of the details in the next few days. Number 1: Jo and I are engaged. We went to a nice restaurant on Saturday for her birthday. I totally surprised her by asking the question in her birthday card: "P.S. Will you marry me" at the bottom. The look on her face as she said "yes" was awesome. *grin* I had already surprised her a couple of weekends back by taking her to Paris for Valentine's weekend. She didn't know we were going until we got to the checkin desk for the Eurotunnel. Now I've got a problem, though: what do I do for a surprise two weekends from now? :-)

• Interesting software updates in unstable:
  • reprepro 4.1.0
  • sylpheed 3.0.0
• Linux Kernel 2.6.33 is available in experimental. Special kudos to Ben Hutchings.
• Debian Installer Team has released the first alpha of the Debian Installer 6.0
• It is that time of the year again, the DPL election process begins this week.
• The Debian Qt/KDE Maintainers are already working in KDE 4.4.1. However, for uploading it to unstable Qt 4.6.2 will need to go in first.
• In case you missed the New Developer of the past month post, there were no new Developers in February.

Update March 1st, 24 UTC: Reynaldo Verdejo is ok, the list is complete! Update March 1st, 16 UTC: Alejandro Vald s, Fabi n Arias, Carlos R os Vera and Germ n P o-Caama o have been in contact with others in Chile. Probably by now some of you know or are worried about the situation in Chile. I m not chilean nor in Chile but I ve been following closely due to the considerable number of good friends I have there, most of them related to GNOME.
From The Big Picture (AP Photo/Roberto Candia) Juan Carlos Inostroza (blog down) suggested I publish the list of GNOME/Free Software people in Chile that has reported since the earthquake and are fine, here it goes: Known, found and good:

• Andrea Orellana Palma
• lvaro Olivares
• Alejandro Vald s (updated 20100301 @16 UTC, via Felipe Besoa n)
• Adrien Bustany (updated 20100301 @16 UTC, via pvanhoof)
• Cristian Barahona G.
• Carlos R os Vera (updated 20100301 @16 UTC, via his sister on fb)
• Fernando Ruiz Altamirano
• Joo Anfossi Mardones
• Jorge Bustos
• Juan Carlos Inostroza
• Fabio Dur n
• Fabi n Arias (updated 20100301 @16 UTC, via Andrea Orellana)
• Felipe Andres Besoa n Pino
• Fernando San Mart n Woerner
• Germ n P o-Caama o (updated 20100301 @16 UTC, via Pablo Estef )
• Mario Gonz les
• Miguel Angel Ruiz Manzano
• Pablo Estef
• Pedro Villavicencio Garrido
• Ricardo Fuentes
• Reynaldo Verdejo (updated 20100301 @24 UTC, via Claudio Saavedra, Juan Carlos Inostroza)
• Sven von Brand
• Sebasti n Lara

Not know, nor found, we presume good but without cellphones:

• None

If you have been in contact with any of them, leave a comment, send a DM in twitter (@diegoe) and etc. List is not complete, I probably forgot someone, please remind me. Will update if I get news from anyone.

My automatic rule-based time tracker (arbtt), which is written in Haskell, collects every minute a data sample consisting mainly of the list of currently open windows (window title and program name). Naturally, this log grows rather large. Since October of last year, I collected 70,000 samples. I already went from a text-based file format to a binary format using Data.Binary, which gave a big performance boost.
But by now, I was afraid that this is not enough. My log file is now 30MB large. Looking at the memory graph of gnome-panel, it is taking up more than half of my memory. When running arbtt-stats, the Haskell run time system reports 569 MB total memory in use and the command finishes after 28.5 seconds. Naturally, the log file is highly redundant: Compressing it with bzip2 shrinks it to 1.6MB. But as I would like to preserve the ability to just append samples at the end, without having to read the file, I chose not just to add bzip2 or gzip compression. Rather, I am now exploiting a very obvious redundancy: Two adjacent samples usually list exactly the same windows, and a focus change only changes a flag. So now, when storing a string that is part of a sample, it will check if this string was already present in the previous sample and, in this case, just store the number of that string (one byte). Only if the string was not present it will write a zero byte and then the string. When reading the sample, the process is reversed. This greatly reduces the file size: It is down to 6.2MB. It also improves the memory consumption, due to Haskell s abilities with regard to sharing: When a reference to a string in a previous sample is read, then only one instance of this string is in memory, even if it occurs several times in the log. This brings the memory consumption down to 264 MB and the runtime to 17 seconds. I released the changes as version 0.4.5.1 to Hackage, Debian and as a Windows installer. The log file is not automatically converted, but new samples will be written in the compressed format. If you want to convert your whole file, you have to stop arbtt-capture, run arbtt-recover, and then move the hopefully noticeable smaller ~/.arbtt/capture.log.recovered to ~/.arbtt/capture.log.
The required code changes were not too big. I somewhat isolated the relevant code in the Data.Binary.StringRef module. Unfortunately, I have to use OverlappingInstances to be able to provide the special instance for String is there a cleaner way (besides the trick used for the Show class)?