I've followed AJ's experiment in funding free software development with his AJ Market with interest and have thrown money in the pot once or twice for some of the projects I care about. It's an interesting approach. I'm too lazy to do anything like that, or rather, I'm already paid sufficiently to work on free software that doing a joeyh market would too much resemble work. However, I did decide to try something I've seen done occasionally, since I also wonder how that works out -- I added a paypal donations button to the website for one piece of free software. Just as a test, to see if it annoys anyone, and if anyone bothers clicking on it. I chose alien since it's well known, regularly used by a suprisingly large number of people and even appears on the first page of otherwise quite amusing results on a google search for "alien". (I suppose that adding google ads to the page would generate more money, but in a probably less interesting way.) Unlike AJ, I won't promise that clicking on the link will motivate me to work on anything. In fact, the most likely use of any money I get from this will be to pass the buck along to the AJ market or something similar.

Trying to build the unofficial Sarge DVD images for AMD64 today, jigdo-lite could not find several packages on the regular mirrors. Next it tried to fallback to amd64-cdsnap.debian.net, which I believe is a mirror listed in the jigdo template, but still failed to find them over there. Browsing the port pages, I found out that the Sarge release for amd64 is hosted on amd64.debian.net rather than that -cdsnap location. It seems those jigdo templates need an update...

I am about to give a presentation in Tacna, Per , at the VI SIIS, Universidad Nacional Jorge Basadre Grohmann. What is different this time? That I am waiting for the conference to start, sitting in my nice workplace at IIEc-UNAM. Yes, I was supposed to go and talk there (and was quite excited to go to the I ConclaDeb at Arequipa last week). Anyway, why cancel my talk, having the technical means to carry it out?
Less than five minutes from now we will start. There is only one thing I regret: As a complete multimedia neophyte, I could not suggest an application to do this from within Linux - We will be using MSN for the presentation. I have a friend's computer on my desktop, and MSN worked great (and at the first try!) for this - Who can suggest me a free alternative for doing this (specially taking into account that I cannot go to the other end and do much configuration on their side - It must be something speaking a standard format)
Anyway... Time to talk! I'll have my paper (Spanish only, sorry - Analizadores morfol gicos aplicados al lenguaje natural, aplicaciones para la b squeda de informaci n) online. Right now I only have my presentation there, but in a couple of hours, the paper should be there as well.

It's amazing what a feed aggregator such as Planet Planet needs to put up with. Apparently some posts don't have timing information with them, so planet needs to invent those times (which it does by the time a post appeared in its cache). Since, however, my planet is still rather young, that meant I had three people flooding it already. Ugh. So I just opened the foo,times file in an editor and set the times for the posts of those people to something a bit older, based on what the HTML on their blog said the time of the post was. Which begs the question: why don't people just publish the time in their RSS feed if they do publish it in the HTML anyway? Oh well. I've already added some extra feeds based on the suggestions Kris sent me; and more are, of course, always welcome!

Today, I took part in a tour to the Intelligent Sensor-Acuator-Systems (ISAS) department here at the University of Karlsruhe. They work on a project for extended-range telepresence. At first it is like a regular Virtual Realtiy device: Head-mounted display for the 3D view of the world and an acoustic tracking system to record the user’s curret position and orientation. While this is a nice toy by itself, the guys from ISAS have developed a system where you can walk through inifinitely large virtual places, including a virtual museum, a pacman game and our cafeteria, rendered by quake, and still stay in the 5x5m room at the university.This works by bending the space, at least the space you percieve: The computer calculates a path that resembles the path you want to go with regard to length and turns, but bends it a barely noticeable bit, thus guiding you in circles around in that room. This guiding is done by shifting the view on the display, making you counter-act that shift and thus turning. A bit like dizziness, when you think you tip over and you move the other way.This works pretty well, at least for movement at regular speed and not too many unpredictable turns - the less predictable the path you take in the virtual world, the less optimal is the path you will take in the real room, thus having stronger and more noticable bends.To me, this seems to be the first step to a world like the one Tad Williams describes in his “Otherland”-Series. Together with what promising but also threatening advances are made on the neurological front, e.g. retina prothesises, “real” virtual reality for mass comsumption is not so far away. And even more improvements in the human-device-interface field have been made: Remote control of human movement by stimulation of the equilibrium sense was sucessfully tried in Japan (German link).I have more pictures of today’s presentation and check out the video on the Karlsruhe telepresence installation.

The biggest difference between the hypothetical security queues and the existing byhand and new queues is that they need to be autobuilt; it’s not really feasible to try saying whether a security update is acceptable if there’s still some possibility it’s not going to build on half our architectures. Additionally, it means that there are two queues that need to be built, rather than just one. The way accepted autobuilding works isn’t too complicated, fortunately. There’s a directory where symlinks are made to uploaded packages and sources, and a Packages files (for all architectures) is generated, as is a Sources file, all of which are made available to buildds. They’re also added to the regular unstable Packages and Sources files and passed to quinn-diff and wanna-build. In order to avoid unnecessary breakage at transitional points (such as when a package moves from accepted to unstable proper), packages are kept in the accepted autobuild directory for a day or so afterwards. There’s also the issue of including the .orig.tar.gz from the archive along with the .dsc and .diff.gz from accepted for uploads that aren’t new upstream versions. Security accepted autobuilding adds two extra wrinkles. The first is that packages being autobuilt can’t be generally visible to other users on the security host, which includes the web server that’s supposed to get the packages to the security buildds. So instead of symlinking, copies have to be made, and care has to be taken to ensure that doesn’t accidently leak information in the process. The second trick is that security needs autobuilding of multiple suites (oldstable and stable, and hopefully testing as well), whereas for the regular archive, it’s sufficient to autobuild unstable packages from accepted, and leave the rest until the daily pulse run. This particularly means that uploads from accepted need to be moved into separate directories depending on the target suite, so that the buildds don’t accidently build a package for woody against the libc from sarge. At a gross level, generalising to supporting autobuilding of multiple queues is therefore straightforward: extend the “accepted_autobuild” table to include information of what queue the files are from (which implies a name change is appropriate), extend the configuration file to allow you to specify which queues should get autobuild handling for which suites, and finally add the code to actually do this when packages are moved into and out of the appropriate queues. Hurray for simplicity! But there’s one little bit of complexity still to bother us. We know we need different directories and Packages files for different suites, but do we need it for different queues as well? Worse, we need different chroots on each autobuilder for each suite, do we need different chroots for each suite for each queue? The latter could well be a complete showstopper. One reason why we might need to treat the queues we’re considering separately is that they have different secrecy requirements: embaroged updates can’t be visible to anyone, even people working on the unembargoed security updates in parallel, and yet the buildd has to forward its logs on to someone to be signed. If embargoed and unembargoed packages are built in the same chroot, by the same buildd, from the same queue, how do the logs magically go to the right person? That’s not necessarily easily answered; currently the hope is that it’ll be okay to have a single shared queue and set of buildds and chroots, and have all the logs forwarded to a single address, at which point they can be redirected to the appropriate security team members by procmail. The assumption this is based on is that there’ll already need to be some magic to do that for the builds themselves once they’re uploaded and need to be moved into either the embargoed or unembargoed queue, so the same thing should be possible with the build logs. Pending further surprises though, that’s enough to move onto the next phases, which is generalising dak to cope with arbitrary queues.

My main fear in life is pain. I'm a wuss. Nowhere is this more obvious than when I'm at the dentist. I literally start shaking when I sit in the chair. While he's prodding around inside my mouth I can feel myself tense up more and more until I force myself to relax and then the process repeats. Being 6'1" I don't get a lot of people believing me when I tell them just how nervous the whole thing makes me. Anyway. When I came to Norwich I managed to find a dentist who claimed to be suited to nervous patients. I went and didn't find them that good, but did make an appointment for my next checkup 6 months after my initial appointment. And then they cancelled it and I spent the next 3 years thinking I should really get round to sorting it out again and not doing so. Then several weeks ago I was having a conversation at an ALUG meeting with Becca and she recommended her dentist. So I decided I should give them a go and discovered that Phill also uses the same one and said they were good too. I finally got around to going today (having filled out all the forms last week). The dentist was nice and friendly and took me on board about being shit scared. And said some consoling words about my teeth being in reasonable shape. I got away with a scale and polish (which I have to go back to get completed in a couple of weeks) and no mention of a drill anywhere. Rock. Now let's see if I can manage a regular checkup schedule. :) Oh, it's the Guildhall Dental Practice at 9 Upper Goat Lane, Norwich if anyone else is looking. 01603 760032.

There’s a reason why I’m going through Baen’s pile of published books. He is the only publisher I know who gets ebooks, I mean really gets them. (Virtually?) all of his catalogue is available in ebook form – with no Digital Restrictions Management! – for a reasonable price, and quite a few are available for free in the Baen Free Library. You can read these books on the browser, or download them to your PDA, whatever works for you. Naturally, you can always buy a dead-tree version, instead. I figure, this publisher is worth the money I pour at him, and hence I pour it in quantity. This is my explanation for having expended six euros for the following ebook. Cally’s War by John Ringo and Julie Cochrane is a sequel to a four-book trilogy by the first author. The trilogy is heavy-duty military science fiction, telling an excellent version of the “aliens invade Earth” story. Cally’s War is set forty years later, and features a minor supporting character from the trilogy as the main character. By the start of the book, Cally is an assassin working for a secret society which wages a secret war against the galactic domination of the Darhel species. The plot of the book revolves around the threat of the secret society being exposed to the opposition, and Cally’s mission in this book is basic counterintelligence: find out the mole and destroy it. However, it would be wrong to think that this book tells an espionage story, or that the counterintelligence plot is integral to the book: the plot is weak, mainly serving as a thin connector between episodes involving Cally’s personal life, or the lack of it, and her sex life (from whose lack she does not suffer). This book, however, seems to suffer from the lack of an editor. There are good ideas, and the storytelling is okay, but to make a really good book, the authors should have regarded this version as the first draft to be thrown away – and rewritten the book completely. There is really no unity in the story, and several good themes worthy of exploration are just skimmed. It is obvious that Ringo did not contribute much prose to this book: the bad politics rants are conspicuosly absent. I think I would have enjoyed the book more had Ringo actually written more of the book, even if it meant suffering through new pro-Bush rhetorics. I suppose this book could work standalone, that is, it does not depend on the reader having read the previous books. On the other hand, there is very little that speaks for reading this book at all, except that it continues the story started in the four-book trilogy. I appreciated the story as a continuation of Cally’s arc, and some of the sex scenes were fun to read, but otherwise it left me wanting. I rate this book 4/10.

Jordi Mallach expressed fear, uncertainty and doubt about the European Union Consitution. I would like to make the following two points:

1. Much of what is in the proposed Constitution is already in force as part of the current foundational treaties. So, if you read the Constitution and want to bash it, please read also the current treaties. I expect you to find it to be clearer and content-wise quite similar to them. So don't think that this document is an evil attempt to seize power or something (well, it might be that but I doubt it).
2. A change to the current foundational treaties would be necessary in any case. The acceptance of new member states makes decision-making by the current rules difficult, if not impossible.

I assume most of the FUD spring from the grand name of the new document, a Constitution. It deserves the name quite rightly. It is the sole foundational document after it is accepted, replacing quite a few treaties. However, its impact is mostly illusory. A similar change was done in Finland a few years ago, when the three constitutional statutes (none of them called the Constitution) were combined and updated into a single law called the Constitution. (Long-time Debian developers probably remember my ill-conceived ITP from four years ago:) Perhaps this FUD is exactly why the consitution should not be subjected to referenda. I don't doubt that Jordi is not alone in his feelings, and such feelings could easily cause it to fail in a referendum. The only effect of such a failure would be making decision-making in the EU painful, without actually addressing the issues people have, such as Jordi's regional issues.

About two weeks ago I got a mysterious bug report regarding BNFC: for some reason, builds failed on some architectures due to ldd failing mysteriously when invoked by dpkg-shlibdeps. I finally got around to tracking it down during the Easter holidays, and it turned out that GHC6 produced static executables on those arches. Eventually Ian Lynagh, the GHC6 maintainer, told me that this had been fixed. I am so slow sometimes. Instead of just uploading a new version closing this bug with a version constraint on the build-time dependency on ghc6, I just asked the submitter to retry a build with the new version. How thoughtless of me. Fixed, now.

I started the grep-dctrl rewrite last spring. I soon had most of the code implemented, and I asked for testers on -devel. There were no debs at that time, and so it probably was not much of a surprise that nobody volunteered. I got discouraged and abandoned the project for several months. In August, just before my vacation at work ended, I decided to polish the code, and I did. I prepared debs and was ready to submit them to unstable. (At that time I was under the impression that experimental was the wrong place for any grep-dctrl stuff.) I felt it was necessary to warn users about the code, since it was sure to contain nasty surprises. I considered sending a mail to some lists: -devel-announce would have reached my audience, but the message would have been inappropriate there; I feared it would not be noticed in -devel. I considered a message at installation time,either with debconf or without. However, debconf messages are seen by system administrators and not users, so that was not an attractive option. I finally decided on that annoying banner:

==========================================================================
                                  NOTE                                    
 grep-dctrl has been rewritten from scratch.  Although this does add new  
 features, regressions are certainly possible.  Please watch for them and 
 report them to the BTS.                                                  
==========================================================================
(The above annoying banner will not be shown to you again, unless you
request it with the -B switch.  It will also be removed entirely soon.)

I took great care to make the banner as unobtrusive as possible, still taking in account that it had to be prominent to have any effect. I added logic that would suppress the banner after a user had seen it once (using a dotfile in home directories, ~/.grep-dctrl-banner-shown; if you find one in your home directory, you can now safely remove it). I did not print it to stdout (it would clobber the output) nor to stderr (it might be redirected, which would have been counterproductive); rather, I used /dev/tty. After all, many people use grep-dctrl in scripts without realizing it; especially they would need to know that there was a weak link there. Nevertheless, I expected to get flamed for this. To my considerable surprise, I got no reactions - aside from several good and substantial bug reports. The banner stayed in longer than I intended, mainly because I was exhausted by my teaching duties during the autumn. I finally removed it last week, in 1.103. And then, several days after the banner had been removed, I get a bug report (which, strangely, is still open even though I sent a closing message almost 24 hours ago). That's when I decided to post this entry.

You know that annoying release-critical bug on grep-dctrl, saying that it's not releasable? Well, I have used my new year mainly to work on that. I have fixed all but one of the known regressions; only i18n is missing. My internal checklist for that bug is getting the regressions out of the way. I hope to get that done on Tuesday. Then I hope to be able to close that bug.

I am taking part for the second time in the (Inter)National Novel-Writing Month. The goal is to write 50000 words in one month of novel-like prose, regardless of quality. Essentially, it is a group effort of producing novel first drafts in a fairly good pace. Seems to work for some people, but I'm lagging behind with no realistic hope of catching up. I am writing each day, but my output is on the order of 500 words a day, not the 1700 words a day that the official goal requires on average. So, I'd properly belong in the International Write Something Month (IWriSloMo), affectionally known as I Write in Slow Motion. The goal there is to produce 200-250 words of bad short fiction (bad, short and fiction being all optional) each day in one month. But I think I'll stick with NaNoWriMo - there's gotta be the losers, too, you know :) And I get a noticeable boost from being able to compete in my own weight class (I can order participants by word count, and I can try to up my position on that list:). My current minimum goal for each day is to restore myself to the first page of Finnish contestants. So far, I have been successful. So what am I writing? It is a Horatio Hornblower pastiche set in the 25th Century and the setting is space. I can hear you yawn, but hey, nobody said it was any good! :)

This posting is part of my Finnish criminal court series. See the introductory essay for more information and an overview of the Finnish criminal court system. The following is a fictional account of a criminal case, one of the most frequent ones, written in the form of a play script. A PDF is also available. There is also an OpenOffice-generated XHTML renderition, which may be more pleasant to read than the version below (but lacks most of its logical markup). DRIVING WHILE INTOXICATED
A glimpse of Finnish criminal justice
Part One Dramatis personae
Presiding judge
Prosecutor
Defendant Extras
Three lay judges
Court reporter Sets
Courtroom
Courthouse lobby DISCLAIMER This is a work of fiction. Although this play is based in general terms on real life, all the specific characters and events portrayed or described in this play are fictitious, and any resemblance to real people or events is purely coincidental. The verdict presented is fictitious, and while it is based on the author's understanding of the law, it may be erroneus. No legal advice should be inferred from this work of fiction. Scene One: And so it begins The courtroom is about the size of a large school classroom, and shaped as a rectangle in terms of floor layout. The judges' bench, a long wooden table, lays near one of the short walls; the space between the bench and the wall is enough to give the judges and the court clerks comfortable room to sit and to move behind the sitting judges. At the center of the bench is the seat of the presiding judge, who has easy access to a telephone, a public address system and a panel for controlling the LED display visible in the lobby above the courtroom door. To his or her right is the court reporter's seat in front of a PC and in control of recording equipment. To the reporter's right there is room for one additional individual. To the presiding judge's left, there are three chairs for the lay judges. If one were to stand directly opposite the presiding judge, on the other side of the bench, one would find an open Bible. In front of the bench, disregarding the Bible, there is some room, approximately two meters or so. Along each long wall there is a small table, and those two tables face each other, and are therefore turned sideways from the bench's point of view. By the desk on the bench's right is the seat of the prosecutor, and the other, facing desk is reserved for the witness being examined. Facing the bench, separated from it by that empty space, are two rows of tables with chairs. Beyond them there are two rows of seats for the audience without tables; the farther row is right at the other short wall. The long wall to the judges' right contains one double door leading to the court building lobby, and the other long wall contains two sets of double doors at each end of the wall, leading to a part of the building not accessible to the public. We'll call the door closer to the bench the judges' door. The PRESIDING JUDGE is a 50-something man dressed formally in black and blue. The three LAY JUDGES are more casually but still essentially formally clothed. LAY JUDGE ONE is a sixty-something male, LAY JUDGE TWO is a female in her fourties and LAY JUDGE THREE is a female just under thirty. The COURT REPORTER is a female in her fifties, also dressed formally. It is 9:00. The four JUDGES and the COURT REPORTER are sitting on the bench. Each JUDGE has papers in front of him or her, the PRESIDING JUDGE's stack being by far the thickest. The PROSECUTOR, fourty-something man, enters through the judges' door, carrying law books and a large stack of paper. He sits down at his desk. PRESIDING JUDGE (Nodding to the prosecutor.) Okay. Let's start. (Enters codes on the LED machine. Then picks up the public address microphone.) The criminal case 'prosecutor against Matti Meik l inen' begins in room three. Parties to room three. Scene Two: Main hearing The DEFENDANT enters through the double door from the lobby. He feels a little lost. PRESIDING JUDGE (To the defendant:) Matti Vihtori Meik l inen? (DEFENDANT nods.) Please, take a seat in the front row. (He waits for the defendant to comply.) This case will be tried as follows. First, the prosecutor will read the charge. You will then respond by telling us if the charge is correct or not, that is, if you in fact did commit the acts specified in the charges, and if you did not, or the charge is only partially correct, you should state briefly why the charge is in error. The prosecutor will then present the case, he will describe us his understanding of the events in question. When he is finished, you will again have opportunity to state if the prosecutor is correct or not, and if not, in what way is he wrong. If there is disagreement between you and the prosecutor, we will hear the evidence. After that, the prosecutor will state his or her concluding opinion of the case as presented here. Again, you will have the opportunity to respond. We will conclude with discussion of what punishment, if any, would be appropriate. The court will then discuss the case alone, and will finally announce the verdict. Is this understood? (DEFENDANT nods.) Okay. Prosecutor? PROSECUTOR (Reads the charge from a paper everyone has in front of his or her.) I demand punishment to Matti Vihtori Meik l inen for driving while intoxicated. Matti Vihtori Meik l inen had on the first of May this year operated his car while his exhaled air contained 0,597 milligrams of alcohol per liter. Matti Vihtori Meik l inen's driving license must be suspended. PRESIDING JUDGE (To the defendant.) Is the charge correct? DEFENDANT Yes, it is. PRESIDING JUDGE Okay. The prosecutor will now tell us the circumstances and events in detail. PROSECUTOR Thank you. It was to my understanding late evening on May day, and the defendant had come from a bar and was apparently trying to move his car to a cost-free parking space. A police patrol happened to observe this and the defendant had apparently been a little shaky in his walking, so the officers gave him a breathalyzer test, which showed significant alcohol content. The defendant was brought in the station where a precision test was performed. The first reading was 0,597 and the second reading was 0,617; the mean of these two is 0,607, and after performing the confidence subtraction, the reading 0,597 milligrams per liter was obtained. PRESIDING JUDGE (To DEFENDANT) Well then, is what the prosecutor just said true? DEFENDANT (Speaking in a colloquial manner and very somberly. He is ashamed and wishes this would be over soon.) Yeah, it is. PRESIDING JUDGE All right. It seems to me that there are no points of disagreement, so we don't need to hear witnesses. (The PROSECUTOR nods.) Okay, it's time for the concluding statements. I think we will hear consequence opinions at the same time. Prosecutor? PROSECUTOR Thank you. I don't think there is much to say. The case has been shown beyond reasonable doubt, in my opinion. We have the precision test readings and the defendant's admission. As to the penalty... the amount of alcohol we are talking about is just below the limit for serious intoxication, so this is no small deal. On the other hand, the circumstances were not very dangerous. I think suspended imprisonment could well be considered, though a stiff fine would probably be more appropriate. The driving ban must be fairly long, obviously. PRESIDING JUDGE All right. (To the defendant.) Do you have anything to say before we adjourn to consider the case? DEFENDANT Well, I suppose it's clear that something is going to be given to me. I prefer a fine, naturally. PRESIDING JUDGE Naturally. The court will consider all options. Do you have any special requests concerning a driving ban? DEFENDANT As short as possible, of course. PRESIDING JUDGE Do you have a particular need for a car, do you need it for work, for example? DEFENDANT No. PRESIDING JUDGE Okay. Now, your income. We have here the last completed tax report for you, it says you earned 500 euros a month and had no children. Has this changed? DEFENDANT No change. PRESIDING JUDGE Okay, thank you. If you will then go outside to wait while we consider this case, thank you. The DEFENDANT exits to the lobby, the PROSECUTOR exits through the judges' door. Scene Three: Deliberation We are now in the lobby outside the courtroom. There are many doors that lead to many courtrooms. Above each door, attached to the wall, is a display showing the current case number; there is also a light saying do not enter, which is lit for most of the rooms, including Three. Also there is a numeric display which is out at the moment for most of the rooms. For some rooms, they indicate when the session will resume. Each courtroom door also has a paper pasted to the door frame listing the cases for the day. The door leading outside is separated from the lobby by a routine security check area, and one of the courthouse police guards stands there waiting for people to enter. Some other guards and janitors sit in their glass booth. There are many people sitting on the chairs, waiting for their turn to enter one of the courtrooms. DEFENDANT enters from the Courtroom Three door and sits down, waiting. It takes about fifteen minutes. During this time, announcements from other courtrooms could be occasionally heard, and people would enter or exit other courtrooms. Then... PRESIDING JUDGE (Over the PA) Verdict in the case 'prosecutor against Matti Meik l inen' in Courtroom 3. DEFENDANT exits through the Courtroom Three door. Scene Four: The Verdict The DEFENDANT enters from the lobby and sits down. After a minute, the PROSECUTOR rushes through the judges' door. PRESIDING JUDGE (Announcing to all people present.) The District Court has examined this case and has decided that Matti Vihtori Meik l inen did in fact commit the crime he was accused of in the way specified in the charge. The justification is the combination of the measured alcohol content in Meik l inen's breath and his own admission of guilt. Matti Vihtori Meik l inen is therefore sentenced to pay sixty-five day-fines, that is three hundred and ninety euros. Penal code, Chapter 23, Section 3. His driving license is suspended until the first of November this year. Road Traffic Act Section 75 Paragraph 1. One who is dissatisfied with this verdict can appeal it to the Vaasa Apellate Court within thirty days. Dissatisfaction must be declared, however, either right now to me or within a week to the Office of this District Court. (To DEFENDANT) Come and take the fine ticket and written appeal instructions (DEFENDANT takes those from the COURT REPORTER.) This hearing is concluded. The DEFENDANT leaves, as the court and the prosecutor prepare for the next case. TO BE CONTINUED
(with the next case)

I spent the whole weekend looking for a DRM-encumbered Sony CD so that I could check for myself whether my code was really being redistributed without permission. I eventually found one: Suspicious Activity?, by The Bad Plus. A rather enjoyable post-modern jazz album. I am rather lucky, it could have been C line Dion. As expected, installing the CD s custom player also installed a stealth aries.sys driver along with a few other interesting files such as $sys$DRMServer.exe, all hidden in a $sys$filesystem directory. I rapidly got rid of this cruft and started studying the really important file, ECDPlayerControl.ocx. It is the file Sebastian Porst and Matti Nikki found to be containing parts of mpglib, LAME, faad2, VLC... Evidence that the DRMS code comes from VLC There are of course obvious similarities between some functions and structures of ECDPlayerControl.ocx and the ones in VLC s source. However, given how precise the implementation of a cryptographic protocol needs to be, the possibility that two separate implementations show many similarities must not be dismissed. The data structures and constant tables especially are likely to be identical. I think however that I found strong enough evidence that this is not the case here. The original drms.c was written by Jon on 01/05/04. I then hacked on that file on 01/18/04 and reorganised it, also replacing the reverse-engineered MD5 and AES functions with clean implementations, then Jon added DRMSv2 support on 05/05/04 and I reorganised the code again on 05/08/04. Some of these changes were specific to VLC (not in that they made the code unusable outside of VLC, but rather that they made it possible to use an external MD5 hash provider, for instance), so they were the bits I was looking for in ECDPlayerControl.ocx. For instance, Jon s original code had the following instruction:

p_acei[ 4 ] = 0x5476212A;

Since all bytes of this long word are comprised between 0x20 and 0x7f, it was later rewritten like this:

char p_secret1[] = "Tv!*";

This change makes the code more human-readable because many other secrets in the Apple DRMS protocol are ASCII strings. Also, I know how to look for binaries that use the string "Tv!*", but it takes me some more time to think of a way to find binaries that use 0x5476212A. And the Sony rootkit uses the following (only relevant lines shown):

 .text:100883B8   mov   cl, byte ptr ds:xxxxx+4 ; 0
 .text:100883C2   mov   eax, dword ptr ds:xxxxx ; "Tv!*"
 .text:100883C8   mov   [esp+70h+yyyyy], eax
 .text:100883DC   mov   [esp+7Ch+zzzzz], cl

The use of xxxxx+4 shows that ECDPlayerControl.ocx uses a string instead of a 32 bits integer to store the secret, disregarding the fact that the 5th character of the string (a null char) is never used. Another example: in this commit I reorganised the DoExtShuffle() function, merging FourthPass() and FifthPass() and discarding a structure allocated on the stack. What is important here is that DoExtShuffle() was built by a complex combination of choices, the main purpose of which being to reduce the size of the C functions:

 static void DoExtShuffle( uint32_t * p_bordel )
  
     uint32_t i_ret;
     i_ret = FirstPass( p_bordel );
     SecondPass( p_bordel, i_ret );
     ThirdPass( p_bordel );
     FourthPass( p_bordel );
  

Note how i_ret is not reused and could be merged into the p_bordel array, or maybe SecondPass() merged into FirstPass(). This choice is definitely a legacy of how the code evolved. And the Sony equivalent: a perfect match.

 .text:10089F3C xxxxx:
 .text:10089F3C   mov   ecx, esi
 .text:10089F3E   call  yyyyy
 .text:10089F43   push  eax
 .text:10089F44   mov   eax, esi
 .text:10089F46   call  zzzzz
 .text:10089F4B   add   esp, 4
 .text:10089F4E   call  ttttt
 .text:10089F53   call  uuuuu

After having studied the Sony code for a while, I have of course gathered dozens of such examples. But I also discovered a few new things. Where does the DRMS code come from? The code undoubtedly comes originally from VLC, but it has traveled a lot. I would be surpried if it came directly from VLC, as no other part of VLC is included with the rootkit. And although there are parts of FAAC in the Sony code and FAAC includes VLC s drms.c, the version currently in the FAAC CVS is horribly outdated. Sebastian Porst also noticed a few obvious differences between the Sony code and the version of drms.c present in VLC. The most obvious being this one:

 if( p_shuffle->i_version == 0x01000300 )
  
     DoExtShuffle( p_bordel );
  

Which became:

 .text:10089F2E   cmp   eax, 1000300h
 .text:10089F33   jz    short DoExtShuffle
 .text:10089F35   cmp   eax, 1000400h
 .text:10089F3A   jnz   short skip
 .text:10089F3C DoExtShuffle:
 .text:10089F3C   ...
 .text:10089F58 skip:
 .text:10089F58   ...

That 0x01000400 is a version check for 4th generation iPods firmwares. I also discovered at least 40 KB of new lookup tables that seem to be used for more buffer shuffling. At first I thought it was simply F4I s DRM implementation, but the shuffling calls are nested with iPod hardware information retrieval from our DRMS code, so they re really part of drms.c. Sony distributing a DRMSv3 descrambler? The only explanation I can think of is that someone took the VLC code, added support for an upcoming Apple DRMS format version and redistributed the software without letting the VLC authors know, probably violating the GPL (I only say that because I have not found the software yet, but I feel pretty safe in saying it). Since it is based on GPL software, I was already entitled to ask Sony for the source code of ECDPlayerControl.ocx. But as the copyright holder of a significant part of the software they counterfeited, maybe I have a greater chance of being heard. Not that it would be difficult to reverse-engineer it yet another time, but it would be awesome if the next opensource iTunes Music Store file player was contributed by Sony!

WARNING: nothing in this post has been implemented or tested, except maybe at a minimal proof-of-concept level ("it compiles!"). It's just some musings on how you can make the computer do your work for you.

I was reading a defense of Hungarian notation when I started to get a familiar notion tickling the back of my brain. The author was explaining that the original idea behind Hungarian was not to replicate type information in variable names, but rather to include some sort of usage information: one example would be giving a string value holding a value that is untrusted the name utName. While it's more reasonable than lpschwzMyVariable, this suggesion made me suspicious that the author was really hacking around suckiness in the C language.

Tagging values with usWhatever and then verifying that you never mix usBlah with Blah isn't just tedious and error-prone, it smells a lot like doing work by hand that the computer ought to do for you. It seemed to me that it ought to be possible to "tag" unsafe values at the type level and let the language's type-checker sort things out. The simple approach is of course to define something like this:

newtype Unsafe t = Unsafe t

NB: I'm using Haskell because it was the first language that sprang to mind when I thought "type system hack".

This newtype declaration lets you attach an Unsafe tag to any type. Presumably the type signatures of routines that return untrusted data (such as a hypothetical getHTMLFormValue or a routine to retrieve previously entered form values from a database, if they aren't sanitized first) should be modified; for instance, if the signature of getHTMLFormValue was

getHTMLFormValue :: HTMLForm -> String -> String

it should instead read

getHTMLFormValue :: HTMLForm -> String -> Unsafe String

To create a new Unsafe value you simply apply the Unsafe constructor to the value:

getHTMLFormValue = Unsafe (oldGetHTMLFormValue)

To unwrap Unsafes you use pattern-matching:

let (Unsafe unwrapped) = wrapped in ...

So, this sort of works, but it's unsatisfying on several levels:

• It allows the creation of types such as Unsafe Unsafe Integer and even Unsafe [Unsafe Integer]. If a value is already unsafe, how can it be "more unsafe"? And what in the world is an unsafe list, or an unsafe list of unsafe values?
  
  Unfortunately, this is probably a limitation that any scheme that relies on type-tagging will have. You can get around it by either explicitly defining each new unsafe type:
  
  

  newtype UnsafeString = Unsafe String

  
  
  This is arguably a good idea anyway, since virtually all unsafe values (in the sense the previously linked article mentiones) are likely to be strings, and as I note later, it's not clear how to "sanitize" a non-string. OTOH, placing unnecessary type restrictions feels un-Haskellish to me, and it seems likely that you can trust the programmer not to pile up Unsafes.
  
  
• This scheme makes it very tempting to strip off the unsafeness of a value and lose the fact that you did so.
  
  Since the only way to strip off unsafeness is via pattern-matching, it seems to me that it would be tempting to pattern-match away the unsafeness in function declarations:
  
  

  foo (Unsafe x) = ...

  
  
  This may be just fine, but my preference would be to forbid pattern-matching altogether (by not exporting the constructor directly to other modules; just export a normal function that constructs an Unsafe value) and instead providing two routines to remove unsafety:
  
  

  escapeData :: Unsafe t -> t
  unsafeExtractRawData :: Unsafe t -> t

  
  
  The first routine encodes the data into a "safe" form, while the second simply fetches the wrapped value:
  
  

  unsafeExtractRawData (Unsafe x) = x
  
  foo x = doSomeStuffWith (unsafeExtractRawData x)

  
  
  The rationale here is twofold: first, by giving the dangerous extraction routine a scary and hard-to-type name, you remind yourself to think twice before invoking it (cf unsafePerformIO); second, you can easily audit all the locations where the unsafety is "stripped away" with a textual search.
  
  There are some issues involving the safe encoding -- you probably ought to make a typeclass of things that can be safely encoded from an Unsafe value; not too hard and this entry is already too long.
  
  
• The final issue involves manipulation of Unsafe values. So far you can pack and unpack Unsafes; however, this seems like it would get a bit annoying if you had to do any significant operations on them. For instance, searching in an unsafe string for a substring seems like it would be a pain and a bit error-prone (since you'd have to remember to wrap the string back up in an Unsafe). You might think that ideally, all normal String operations should work directly on an Unsafe String or pairs of Unsafe Strings -- you just shouldn't be able to convert between the two.
  
  Unfortunately, I can't see any way of doing everything that you'd want to here. For some types you can. For instance, the following definitions let you compare values wrapped in Unsafe in the usual way:
  
  

  instance (Eq t) => Eq (Unsafe t) where
      (==) (Unsafe v1) (Unsafe v2)   = v1 == v2
      (/=) (Unsafe v1) (Unsafe v2)   = v1 /= v2

  
  
  This admits (Unsafe t) to the Eq class, and looks reasonable enough (assuming that it's safe to compare unsafe values, of course). However, for probably the most important Unsafe values -- Unsafe Strings -- I can't see how to do this. The problem is that String is not a typeclass, it's a synonym for [Char], a list of characters. The important string operations are either defined directly on the String type, or generically on [t]; i.e., "a list of anything". Because there is no typeclass, you simply can't overload these operations to apply to unsafe strings as well as regular strings.
  
  One way around this is to provide an "unwrapping combinator":
  
  

  liftUnsafe :: (t -> t) -> Unsafe t -> Unsafe t
  liftUnsafe f (Unsafe v) = Unsafe (f v)

  
  
  This would automatically convert string operations to unsafe-string operations; not ideal, but reasonable enough. Operators are still a little annoying, but you can mangle their names:
  
  

  (++u) :: Unsafe [t] -> Unsafe [t] -> Unsafe [t]
  (++u) = liftUnsafe (++)

  
  
  So you have to use ++u instead of ++ to concatenate unsafe lists; not too bad.
  

While it may not be perfect -- it only makes sure that you don't accidentally violate safety, and doesn't stop you from deliberately doing something dumb -- this approach feels a lot safer to me than ad-hoc mangling of variable names. I used Haskell because you can do type hacks fairly easily in it, but I don't see why you couldn't play a similar game (albeit much less conveniently) in C++, or even in a dynamic language like Python. Of course, Python would give you runtime errors about safety violations instead of static ones, but that still beats getting runtime security holes.

I would add as a hypothesis that it should be possible to express just about any variable-name-mangling technique whose goal is to "make wrong code look wrong" in the type system, provided your type system doesn't suck; and that doing so is preferable to doing it manually, since it means that a lot of repetitive error-checking can be handled by the computer. And since the computer is better than you are at verifying formal invariants, forcing it to handle them whenever possible is always a good idea.

As part of Penn’s Women in Computing month, I’m giving a talk about women in free and open source software development on Tuesday. There won’t be too many surprises for regular readers of join-the-dots, but if you’re in Philly, you’re welcome to come along. The talk’s at 3pm in the Wu & Chen auditorium at Penn’s Department of Computer Science, Levine Hall, 3330 Walnut St., Philadelphia.

Note: When reading this entry, please set your date to 2005-11-10. Delays in the system prevented me from posting this earlier.
I have lived together with Nadezhda for almost ten years - And this month (on November 1st, to be precise), it is ten years we first met on one of the few BBSes we had in Mexico by then - Thanks a lot, Mag, for setting it up - you changed my life in more ways I'd ever have thought!
Over the time, of course, our relation has changed a lot - the first step was telling to other people about each other's existence. The second was accepting we were not together only in between (long story... But I was about to leave the country for good. And I did. But then she followed. And we came back together). Then, little by little, we started settling down, we became a formal couple in the eyes of our families and friends.
We always thought social conventions were silly. We laughed about the prospect of marriage, of having to accept in front of others what we felt towards each other - But anyway, some months ago, we started talking it would be nice to make out of our nice, private relationship something serious and formal to commemorate our ten year anniversary.
Not much was told about it for some months... Until by mid-October we realized we would not make it in time if we didn't hurry - So we went to the Registro Civil at Coyoac n and asked for an appointment with the judge. November 1st is an official holiday, so it was not possible to make it right on the same date - We got the appointment for November 9.
Some people might be asking here why weren't they told about this - Well, first of all, as you can see, we didn't have much time to plan this. Second, we decided to take this formal step, but to have it privately. We went only with my father, my brother, Nadezhda's mother, sisters, our nephews and one long-time friend. We were summoned at 10:30AM, had to wait for the judge to marry some other couples until around 12:00PM, had some nice words from him for ~15 minutes, and went for a nice breakfast in a nearby restaurant with the whole gang.
Oh! Something interesting: We actually didn't know when we were formally getting married. We were called some 30 minutes before the official words to sign some papers, which the judge would later, and privately, certify as valid. The judge just talks, he does not sign anything with us present... But, well, it is a nice moment ;-)
And... Well, here is the proof. It took some days to receive the document, it took me some more days to finally post about it - But finally, our acquintances have been informed of our formal step towards a life together :)