Dear Lazyweb,
,
Nothing is perfect, but on Linux you can at least improve it and tune it. I just ran into memory problems on my development box - I had tons of applications open, and since this is a laptop I prefer to have swap disabled. I then started to build bind9 for a sarge backport, which meant that a long sequence of C compiler processes were launched, each allocating some memory, working with it, terminating, next. Being really at the limit of the available memory, the system became unresponsive - and that behavious is really bad... I thought that sooner or later it will really run out of memory, and decided to watch some TV in the meantime. One hour later when I returned, it was still torturing my HD and still unresponsive. Apparently the compile process has been all the time just below the memory limit, causing the kernel to remove pages from memory and load them back from the disk later, which slowed everything down just horribly. So now I'm searching for a way to prevent such situations. The traditional ulimit approach won't help - no single process was responsible for using all the memory, but each one had its reasonable share. So dear lazyweb, is there a way to tell the kernel to reserve a share of the memory (lets say, 64 MB) for buffers and caches only? I can live with out-of-memory errors a bit early (and the just enable swap), but a massive slowdown as described above (where the kernel is apparently reading some data from the HD, then forgetting it again and replacing it with the next) must not happen that easily... Right now I fear that you could bring down any Linux system easily with a "slow" forkbomb where each forked process allocates 100 MB and starts writing some random data to its memory, forking a new child every 50 MB or so. Neither the OOM killer, ulimit nor traditional forkbomb detectors will be able to counter that, since already a couple of processes will cause the system to become completely IO-bound. Oh, and I have no idea how Windows would react on that either. So don't assume any OS is better, just because I say Linux isn't doing perfectly well here. If you have any tips for me, send me an email.

Continuing on from the SCC stuff, which I should probably get in the habit of calling the mirror split, then… At the same time as I’ve been trying to work out ways to fit the mirror split stuff into the AJ market concept, I’ve been pondering on and off what can be done to improve the security situation. My thinking’s probably still biassed by my long time wish for official security updates to testing, but I’m somewhat inclined to think that if we can figure out a way of integrating the testing security team and the official security team we might end up closer to a solution – at the very least that’d provide us with a decent number of people who can actively help out with security updates that aren’t secet and get a few more people familiar with the technical aspects of how security updates are released, who’ll then have less of a learning curve if they end up being added to the security team proper. And heck, even if that doesn’t work, it at least gives people running testing an easier url to remember for security updates. Of course, integrating the teams isn’t that easy, or it would’ve already been done. So after thinking about it, I had a chat with Joey (aka Martin Schulze, aka the guy who’s issued 317 of the 340 security advisories in the past 12 months) about the problems:

<aj> so, the other thing i wonder is what you think of the testing-security-support split?
<Joey> need more input
<aj> as in separate host, funny domain name, not available on security.debian.org/, extra effort to mirror, etc
<aj> i think it sucks :)
<Joey> there are a lot of sucking things…
<aj> it’s true
<aj> so, i figure the reasons for the split are (a) you don’t want to add a bunch of people to team@ and hence vendor-sec, and (b) there’s no way to make it so they can use dak-security without being full team/vendor-sec members; and maybe (c) you don’t want them officially associated with debian security coz they’re newbies
<Joey> a) and b) are true
<Joey> However, with security.debian.org being a push mirror, it would be possible to let it be pushed from a second source into a different directory, such as /testing-security/
<aj> hrm
<Joey> However, that would require all architectures to be supported and not just some.
<Joey> all as in “all that are in testing”
<aj> ah, that’s (c) then – they don’t do enough for the official debian stamp
<Joey> ah ok

Assuming those really are the key factors (which might or might not turn out true, but they seem likely) then that’s something that can be worked on. The first point is completely out of my domain – I’ve never been on vendor-sec, and it’s not likely something that ought to be tweaked much anyway; vendor-sec’s a very closed list by design, and for most security work it’s probably a distraction (testing security as is currently operates without concerning itself with vendor-sec, AIUI); the second points purely technical; and the third point is a combination of technical issues and an issue of training, both of which will be hopefully helped a lot by integrating testing-security and security.debian.org. So the easiest point of attack is obviously the second one, since it’s “just” a matter of expanding dak’s featureset. The problem here is embargoed fixes – the security team needs to be able to prepare fixes for vulnerabilities revealed on vendor-sec, but then keep them secret from everyone not on vendor-sec until the publication date. Which is fine, except that dak takes an all or nothing approach to that sort of secrecy: secuirty uploads go into accepted, get built, and then the security team run “amber” to publish the update. But what that means is the accepted directory and its contents can only be visible to people with vendor-sec access, which means only people with vendor-sec access can do security updates. But we do actually have a different queuing mechanism that allows us to keep some things secret, while not hiding everything: thanks to the crypto-in-main transition, the NEW queue acts like that, more or less, only allowing ftpmaster to see the contents of NEW packages until they’ve passed license inspections and the appropriate export notifications have taken place. And actually, we’d already considered extending that sort of behaviour to add an “unapproved” queue that would prevent uploads from entering proposed-updates without the stable release manager’s prior approval. The idea then, would be to have a couple of queues for the security team: an embargoed queue, accessible only to those with vendor-sec access that would store embargoed fixed ‘til they’re ready to be released, and an unembargoed queue for preparation accessible by a wider group, for preparation of fixes that don’t need to be secret. Both queues would need to be autobuilt (and the build logs for embargoed uploads would need to stay restricted, while the build logs for unembargoed uploads would not), which is an added difficulty, since only accepted can be autobuilt presently. But otherwise, it seems basically plausible, both to myself and Joey. Conveniently, Andrew Pollock’s been interested in throwing some money through the AJ Market to see what happens, and since the initial idea we were tossing around fell through due to his move to the US, he’s put up the dosh to make the above happen. Well, strictly the dosh to make me make the above happen, or something. Anyway, I’m already running behind both in implementation and blogging about it, so back to the grindstone… (Okay, perhaps it’s not quite the feature Mark Twain would most like added to dak, but it’s still nifty.)

It's been a while since I blogged last, but I'll plead lack of time as a defence. Again... :-) Well, after a few weeks in the new job at Amino things are settling down. I'm thoroughly enjoying learning my way around a new codebase, and the people are really good and friendly too. I'm already in charge of the new cvs server at work, which somehow got a Debian installation on it rather than the Fedora that people had been expecting. How could that have happened? On the Debian front itself, I've been a little lax and feeling guilty for that. My advertised debian-cd developments are behind schedule as I haven't had the time to work on it. On the plus side, I've finally diagnosed and fixed the grave bug that was holding the new upstream version of cdrtools out of testing. Once things have migrated through, I'll then be clear to start merging in my JTE code. Then debian-cd upheavals. Yay!

John Carroll:

[T]echnology across software domains should be consistent. There should be a standard, however de facto, that works everywhere. Skills should be portable across those markets, so that someone with experience in desktop software development has a decent chance of developing for handhelds. Everything should just work together, and development across all devices should be relatively straightforward to someone with experience in any one of them. Microsoft has been very good at this. […] I wonder whether the open source world can match that consistency. Granted, there are movements to make things consistent, such as the LAMP set of technologies for Linux, or the Linux Standard Base project. However, the structure of development for open source products militates against the establishment of any consistent standard. Open source development relies mostly on voluntary contributions. People don’t tend to contribute to ALL open source products (that would be impossible). Rather, they concentrate mostly on the products that interest them. This results in a developer version of tunnel vision. Since all that matters is the creation of the handful of products to which the volunteer contributes, what matters is the interests of contributors to a single project, not the interests of technology which should span ALL products.

Great points, though I clearly disagree that the open source community is fundamentally unable to match Microsoft on providing a consistent environment. It’s going to take some work on our part though. One of the reasons Microsoft has historically been so successful is its unique ability to integrate disparate technologies into cohesive wholes, as well as to leave the mechanisms they use to do this just open enough that others can integrate with them to a degree but never so open that they can do so better than Microsoft can. Of course, Microsoft can do this because they develop, own, and control each of the pieces and the interfaces between them. Other platforms (Linux, Java, UNIX, even the Web to a certain extent) are comparative mishmashes for the exact reasons John describes, namely that every project is developed independently, and that there’s comparatively little thought put into how the pieces interact. The only parties that do think about these issues, the Linux distributors, tend to do so after the fact, and, of course, they have no direct control over the projects that produce the pieces to push any integration improvements they might make upstream (in the general case anyway). Furthermore, once multiple solutions in a given problem domain emerge, it’s not always possible to generalize the interfaces when the projects behind them don’t agree with each other on the “right solution”. In other words, because it’s an after-the-fact thing, “not invented here” and general inertia come into play. So, while the technical aspect of the problem is important, namely the interfaces between components (APIs at the source level and ABIs at the binary level), there’s a social aspect here too. We need to encourage the kind of competition at the implementation level that results in technical excellence while fostering a sense of cooperation at the interface level to ensure that, collectively, the various implementations fit seamlessly into a larger, consistent environment. In other words, we don’t want to get in the way of open source project and Linux distro innovation, but we do want to encourage open source projects and Linux distros to innovate in a collaborative fashion so the technology that ultimately wins in the marketplace can be standardized with as few legacy issues as possible (or, in the case where multiple best practices emerge, for legacy reasons or otherwise, we want to collaboratively define a uniform interface, i.e., cooperate on interface, compete on implementation, etc.). That’s the only way we’re going to get there.

The Science and Intellectual Property in the Public Interest project of the American Association for the Advancement of Science has conducted a survey on about 1,100 AAAS members to determine what effects patenting has had on research conducted by scientists in academia, industry, government, and nonprofit organisations. In the past, the patent lobby has often claimed that patenting is a means to foster development and save investments during research and development by not allowing competitors to use the same technique without paying licensing fees. The results of this survey were quite surprising because this exactly claim could not be validated. Industry scientists reported having more difficulty in accessing patented technologies than academic scientists. Bioscience scientists working in industry reported about problems that their research had been affected by these difficulties. Historically, academic scientists share the results of their research via papers, journal publications and conferences. They also get a lot of input by others at such occurrences. The commercial value of their research is also not always sufficiently obvious, but in the case of basic technologies or insight essential. This is not compatible with the patent system, it seems.

So it seems I cannot uphold my word - Well, yes, it was because of the trouble it would represent to the fine Moquegua guys - But anyway, I ended up accepting the invitation practically at the last minute, and went to Moquegua, Per , probably for my life's most tiresome journey.
Saturday, after lunch, we headed for the airport. I left Mexico at 17:30, and arrived five hours later in the Lima airport. Thanks to Antonio, Jos and Laura, who picked me up at 23:30 and arranged me a room in a hotel. I met the DebianPeru crowd, a very nice group, Sunday morning - We had breakfast, chatted for a while, and went for Lima's famous ceviche (assorted seafood cooked in cold in lemon. VERY good). At 3:45PM, I took the bus to Moquegua... For 17 long hours.
Got to Moquegua at ~8AM, went to the Jos Carlos Mari tegui University. I met the other guest speakers for breakfast, went with them to meet the Univerity Rector, hooked up some minutes to the net, went to a typical Moqueguan restaurant, gave my talk... Then hurried up to get the bus back to Lima. We got there five minutes before it left. I don't know why or how, but the bus back took 14 instead of 17 hours - I will not complain ;-)
Back in Lima, Carlos met me at the bus station, we went for breakfast, and hurried to the airport. I crossed the gate at 12PM, hurried by the airline personnel (and didn't manage to see Rudy/Stone_Head, who was on his way there)... Just to find the plane was delayed, although by little. Sat down for a while, hacked for a bit, and took the plane back home.
Summing it all up, I traveled roughly 40 hours, out of the 72 hours I was out of Mexico. Of course, you could add 1hr travelling to the airport and 1:30 back home... Anyway, 40/72 is quite close to 42/74 :)
I am tired. Plain tired. I am just waiting for Nadezhda to get home, as I want her to find me alive, and I'll crash down in bed.

Who can resist a good rhyme? Or a bad one? So this round of dak hacking turned out to make the AJ Market scheme another notch more confusing – hence the delay in blogging, and the teaser in my last post. The issue leading to the confusion is that the major item on the list to start hacking on was SCC, which unlike the projects I’ve undertaken so far, is more than a day’s hacking. In fact, due to the need to give mirrors a chance to adapt to the new system between it being implemented and actually used, it’s actually more of a multi-week task. And doing it as a one-day-a-week project would extend that into a multi-month task afaics. I guess that’s still better than never, but obviously it’s worth looking into alternatives. Naturally, then, the first phase of a longer project like this is threefold. We’ll call it “the three P’s”. Planning My theory at this point was to come up with a plan for what to do, try figuring out how much work it’d take, and then see what sort of financial arrangements might be plausible – not involving me cutting a few weeks out of my life for spare change, but without making the whole thing an unleapable chasm from what the AJ Market’s currently managing either. I figured writing it up as a semi-formal proposal makes most sense:

Summary Implementation of the outstanding mirror split proposal for the Debian archive to allow new architectures, particularly amd64, to be included in the archive. Benefit In spite (or perhaps because) of its simplicity, this project has been languishing for over two years, and is not currently being worked on; so at present it’s not even possible to estimate when it would otherwise be completed. It is most notably preventing amd64 from being integrated into the normal Debian development environment, causing derived distributions to maintain amd64 specific patches themselves. In the longer term, reducing the constraints imposed on the archive size may allow the introduction of additional suites, such as backports or volatile, as well as additional architectures; though significant further discussion on this would be needed. Background Since at least mid-2003 the Debian archive has been closed to new architectures due to the already large amount of space and bandwidth required to become a Debian mirror. At present, the archive uses some 158GiB of disk, and about 1GiB per day; additional architectures are expected to require approximately an additional 10GiB each, and there are likely around half a dozen architectures that will be considered for addition once the moratorium on new architectures is rescinded (incl amd64, armeb, sh variants, kfreebsd and possibly partial architectures for arch variants such as s390x and ppc64). The primary work needed to fix this involves:

• ensuring the mirror network operates correctly when a majority of mirrors are partial; this reduces the impact on bandwidth and storage capacity
• optimising portions of the archive maintenance software, particularly apt-ftparchive; this reduces the load on the archive server
• providing appropriate guidelines on the qualification criteria new architectures need to meet in order to be added to the archive; this provides a limit on future increases, allowing growth to be appropriately controlled

Actual work I expect there will be six phases to the project:

1. cleanup of the archive as it stands, and establishing a clear categorisation of its contents to define what a partial mirror by architecture or suite should officially contain
2. providing appropriate scripts to ensure mirror sites can easily comply with the previously defined expectations for partial mirroring
3. devise an appropriate structure for the new mirror network, that can easily incorporate existing mirrors, and coexist with the existing structure
4. provide information on the new structure to both mirror admins and users; assist with the transition, and resolve any problems found
5. ensure the archive management software is appropriately optimised, and that archive inclusion criteria have been debated and established
6. add new ports that have passed the qualification requirements to the archive

In theory, a couple of days for each of those sound plausible, so making that twelve days actual work (with a couple of week’s delay in between for mirrors to have some time to adapt to the new network). On the downside, twelve days at a day a week is over three months of real time, not counting the possibility of doing other things with the one day a week, or Christmas, or the aforementioned delay for mirrors. Yick. So much for planning. Preparation So the next “p” is preparation. In this case that’s finally getting around to fix dak CVS, which has been slightly broken since May. The extent of the actual breakage was just the loss of the ChangeLog history, aiui (or at least, that was the unrecovered breakage), but the result of that was months of uncommitted changes on both ftp-master and security (and reportedly from Ubuntu’s dak installation too). The changelog for the first set of commits (not counting buildd changes from ftp-master, security changes or Ubuntu changes that haven’t made it to ftp-master) looks like:

        * tiffani: new script to do patches to Packages, Sources and Contents
        files for quicker downloads.
        * ziyi: update to authenticate tiffani generated files
        * dak: new script to provide a single binary with less arbitrary names
        for access to dak functionality.
        * cindy: script implemented
        * saffron: cope with suites that don't have a Priority specified
        * heidi: use get_suite_id()
        * denise: don't hardcode stable and unstable, or limit udebs to unstable
        * denise: remove override munging for testing (now done by cindy)
        * helena: expanded help, added new, sort and age options, and fancy headers
        * jennifer: require description, add a reject for missing dsc file
        * jennifer: change lock file
        * kelly: propogation support
        * lisa: honour accepted lock, use mtime not ctime, add override type_id
        * madison: don't say "dep-retry"
        * melanie: bug fix in output (missing %)
        * natalie: cope with maintainer_override == None; add type_id for overrides
        * nina: use mtime, not ctime
        * katie.py: propogation bug fixes
        * logging.py: add debugging support, use   as the logfile separator
        * katie.conf: updated signing key (4F368D5D)
        * katie.conf: changed lockfile to dinstall.lock
        * katie.conf: added Lisa::AcceptedLockFile, Dir::Lock
        * katie.conf: added tiffani, cindy support
        * katie.conf: updated to match 3.0r6 release
        * katie.conf: updated to match sarge's release
        * apt.conf: update for sarge's release
        * apt.conf.stable: update for sarge's release
        * apt.conf: bump daily max Contents change to 25MB from 12MB
        * cron.daily: add accepted lock and invoke cindy  
        * cron.daily: add daily.lock
        * cron.daily: invoke tiffani
        * cron.daily: rebuild accepted buildd stuff
        * cron.daily: save rene-daily output on the web site
        * cron.daily: disable billie
        * cron.daily: add stats pr0n
        * cron.hourly: invoke helena
        * pseudo-packages.maintainers,.descriptions: miscellaneous updates
        * vars: add lockdir, add etch to copyoverrides
        * Makefile: add -Ipostgresql/server to CXXFLAGS
        * docs/: added README.quotes
        * docs/: added manpages for alicia, catherine, charisma, cindy, heidi,
        julia, katie, kelly, lisa, madison, melanie, natalie, rhona.
        * TODO: correct spelling of "conflicts"

Ugh. Still, that’s enough to start work. And the final “P”? Come on, be honest with yourself, you know what it’s going to be. Procrastination Okay, that’s not entirely fair; the irrelevant bit of work was actually on to TODO list before SCC (mostly because it was something that I could get done reasonably quickly) and in fact was this line of the above changelog:

        * dak: new script to provide a single binary with less arbitrary names
        for access to dak functionality.

All the various model/actress names have been getting more than a little confusing recently, with almost forty in the dak suite, and another two dozen or so in use elsewhere – and then there’s the fact that the whole hot babes thing is both a bit offensive, and getting a bit old. OTOH, you need something to rename them to. We ended up deciding on the “version control solution” and introducing a “dak” command that’d launch all the different little bits of functionality depending on arguments, in the same way cvs, svn, tla, bzr, darcs etc do. The implementation’s kinda neat: we have a list of commands (like “ls”) and their description (“Show which suites packages are in”), along with the python module and function they’re in (“madison”, “main()”). That let’s us not actually have to change any of the other scripts immediately, and lets “dak ls foo” work the same as “madison foo”. It also means that down the track we don’t need to have separate modules for each subcommand, and that we can rename modules and functions without affecting the user interface. Of course it also means that all the internal scripts haven’t changed to use the new names yet, leaving the new interface a bit underused, but hey, “dak ls” at least manages to be one character shorter than “madison”, so that’s a win! To be continued…

Ick.
So I caught some sort of cold/flu thing, which inverted my sleep schedule (especially since it happened as daylight hours began ending around 5PM). In the process of rotating it back I got sick again, probably due to lack of sleep. Last night I didn’t sleep at all, so hopefully I can pass out around 10PM tonight without a problem. Quod Libet 0.15
Released (and uploaded to Debian), featuring all the aforementioned speedups (SVN actually has even more yet, and some significant memory reductions). We are polishing everything up (and working on the audio feed browser (feedparser is seriously hot)) for 1.0. It’s all very cool. Well, except the times GStreamer segfaults; that’s not so cool. Geeky Jokes
In a lecture on quantifier semantics, my NLP professor made the comment “In America, a b F(a, b). In my country, b a F(a, b).” Much of the class laughed. Speaking of NLP, my final project for that class is likely to segue into the next thing gesten and I tackle, assuming I have any free time at all next semester. Books
I haven’t spent money on books in almost two months (aside from textbooks). This means the amount of books I own and haven’t read is actually for the first time in about a decade. Soon maybe I’ll even be able to go back and reread the books I read when I was 12 but don’t remember very well (ruining precious childhood memories in the process). Python Optimizations
Just because all my posts have had one of these lately, two recent things that surprised me: if s[:1] == c: is slightly faster than if s and s[0] == c:. And if v: is slightly faster than if v is not None: if v could be 0, “”, or (), the false immutables (this one surprised me less after some thought, but it’s still easy to forget when is is otherwise faster).

Coding Mistakes

I broke my debian administration website yesterday. Predictably enough I’d woken up around 9AM and thought that it would be a good time to make significant refactering changes. Tested them on my laptop, committed them to CVS, then updated the live site. Had a quick play to make sure nothing had obviously broken - but didn’t try to leave a comment on an article. Went to sleep and woke up around 4pm to find 15 emails telling me things were broken. Most mails on the site in a single day! Should be fixed now. But I’ll never again make changes before lunchtime, I’m just not good with the whole sleepy-coding thing.

Things I have never coded: 1

Inspired by Joey Hess’s “Ten years of free software”. Things I have never coded part 1. I have never coded a text editor. *shock* In the past I have written simple wrappers around the various text controls in different environments - for example under Windows I once coded a tabbed text editor called MyNotepad, which featured a novel dockable tree-control to the side which allowed navigation and loading of files easily. When I first started working with Java I coded a similar thing, but with the ability to load external Java .class files as plugins. This editor was called Rosetta, inspired by the band Rosetta Stone (who were in turn inspired by the actual Rosetta stone ) At times I think about text editors. I use them daily, obviously, and I love working with some more than others. My editor is usually GNU Emacs but I’m also partial to Vim. I will use both happily; although neither is pleasant without my personal customizations. Simple text editors are a traditional project for coders, and freshmeat is littered with their discared corpses. Probably the only reason I’ve not started - that and apathy. My ideal editor would be:

• Both curses and x11 based.
• Scriptable (Lua is love?)
• Extensible.
• Portable between (at least) GNU/Linux and Solaris.
• Modeless.

As predicted, the morning was not productive at all. At one point I slightly became concious and saw sladen fetching his bike, opening the door and saying me "Back in two hours!". We didn't know anything about him until 24 hours later. As we started waking up, we tried to think of a plan for the day. The morning was gone, though. Clint took us to a restaurant two blocks away for breakfast, or actually brunch, because that was the only food in many hours. I ate some "pancakes" with banana in them, and barely could finish them; they are very filling. We went to the promenade for a last look at the Manhattan skyline and took some pictures while we tried to figure out what to do.

A last view over Manhattan Mako was developing a plan in secret, though, which involved micah and his car. Micah found out soon after. We went back to Clint's, expecting to find sladen somewhere, but we didn't. I packed my things, and when we really needed to go, we resolved to put his things in the entrance so he could find then and do whatever he wanted on his return. We went on the Metro to Union Square, where, hear this helix, I saw TONS of raperos doing their break-dance thing (ok, maybe not tons, but I did see them). And two blocks away, we arrived to the most incredible book store. Mako has blogged many times about buying very rare books at the Strand for ridiculous prices, but this was totally unexpected. The Strand is huge. On the outside, there's carts with hundreds of second-hand books, all for one dollar. Mako used to go there and browse the books to find interesting stuff. The Acetarium is full of them! Most of the dollar books are total crap though. A very high percent talk about god, religion, "our troops" and stuff like that. Mentally filtering what might be good and what is not is not easy when you haven't done it before. Mako started scanning the shelves as soon as we arrived, and every now and them he'd pick some totally stupid title. I was on my own shelf, missing many good stuff, while mika, Clint and mako did their own. Clint found quite a few titles, but mako didn't find anything good. I got lucky though, and found the precious book of the day: The Spanish Anarchists. The heroic years 1868-1936, by Murray Bookchin. Although old, it was in very good condition, and mako immediately asked if I was taking it. :) I won! The inside of the Strand was incredible too. Three very big floors packed with books, plus the half-priced "reviewer's copies" at the basement. I was totally overwhelmed by the amount of books I'd be interested in buying, and the limited weigh I should be carrying back, so I did my best not to look too carefully. For a few hours, I didn't have a too clear idea of what mako's plans were. At some point, micah phoned him and we rushed to the PATH, the train that goes to New Jersey. Apparently we were meeting micah at Newark, but I still didn't know why we had to go there. During our ride, I could see how pleasant New Jersey is to the human eye. Miles and miles of heavy industries. I'm pretty sure it's the ugliest place I've ever seen. What I couldn't see, and I'm still disappointed, is any of the viciously arrogant rats of the New York metro system. Mako had told me about how they look at you, like saying "if you come down here, you'll be left with 9 fingers", and the incredible number of them on the tracks. Unfortunately I couldn't see any. I blame Michael Bloomberg. At Newark, micah appeared in his car, and we all hopped into it. We apparently were going by car to Boston... including Clint. I was puzzled, as he had to work on Monday, but he'd said that he'd simply "come back". There was a problem though, mako had dilinger's keys, and he was going back to NYC from Boston, and we didn't have time to get to Boston before he left, so we had to find a way of leaving the keys in NYC so he could get into his appartment. Finally, mako resolved that Clint would be left in a cold street, in a rough area of NYC, and he would handle the keys problem. I still didn't understand what was going on. We headed North, and I quickly fell asleep on mika's nap. I'm getting old. When the car stopped again and I suddenly woke up, I looked ahead and... HORROR! We were at the door of a McDonads. WTF WAS GOING ON? So it seems that when you're on the road, there's no much other choice than shit food. After a quick discussion with mako about how badly those burgers taste, I quickly went back to sleep, until we were in Boston. It took a while to find a parking spot, because all of them "permit only". It's impossible to park if you come from somewhere else. I was finally at the Acetarium, a very cool flat in Cambridge, 15 minutes away from the MIT. It was 3:30 in the morning. The day after, we woke up pretty late, and as I opened my laptop and found a privmsg from sladen, he was was knocking at the door. This man has an incredible ability to disappear and reappear when he's most unexpected. Apparently he managed to grab his stuff in NYC, fetch the bus and arrive at 2 in Boston, meeting dilinger at the station, and mako as he went out to MIT. micah and I went to explore Boston, but never managed. We consumed a few hours when trying to ship some equipment to micah's colo, and then trying to find food. When we were ready to go, micah got a very badly timed call, which made him go back to the Acetarium: a server at work was dying or dead. At one point, it was late enough that going to Boston wasn't worth it, so we went to visit the Media Lab instead. On our way there, micah and I kept chatting about this and that, and found out how small the internet is. We both know a bunch of common people from uncommon places like Per , from our involvement in social projects in the net like Indymedia and Sindominio.net. The MIT is incredible, specially the Stata Center by Frank Gehry. We wandered around as we only got voice mail from mako, until an hour later we found we was like 20 metres away from us. He gave us a tour through the Media lab, and it was incredible. I had never seen something like that. Every lab was colourful and fun, and very cool developments were going on everywhere. I also saw the coolest mame box ever, with an arcade case and everything.

The Stata Center, right next to the Media Lab After the tour, micah and I headed back, while sladen stayed to have a look at one of mako's projects. When we got back to the flat, there were a few people already there for mika's sushi party, and I rolled my first sushi rolls, which was fun. A bunch of Debian people joined us, including zufus and gravity, and had a great chat with them. The party ended way too late and after too many beers. sladen used his secret disappearing abilities, and a few colleagues from mako crashed, so there was a bit of overbooking. Mako's solution was simple. mika, him and I could share their bed. I can't say I have slept too much, because mako would keep pulling the blanket and leave me out in the cold, and I was trying not to squish mika. But it went ok, although we discovered too late that mika was hot while I was cold, and we swapped our places too late. Today, we'll go find the Freedom Trail and walk a bit around it, meet gravity and zufus for lunch, and I will sadly have to go back to Cambridge, quickly pack and head to the airport to catch my plane. My hours in North America are expiring.

I know, I know. A bit early, you might say. But let's face it: The shopping industry is already fully into christmas. You already get advent calendars, christmas cookies and thing in the various stores. And I got my first christmas presents the last week.... First of all I finally tried again to switch to Linux 2.6. Going through the great amount of options in my make oldconfig from 2.4 I finally tried to boot with it. The most deep changes I did was:

• Switching from APM to ACPI: This was one of the things I was worried about. But there was nothing to worry, I now can even see the temperature of the processor!
• Software Suspend: I now can shutdown my system to disk! That rocks, and seems to be more stable than the former APM sleep mode, which occasionally had problems with waking up again...
• The powerbutton now performs a shutdown. Unfortunately it is the only button that seems to get recognized, the key bindings with the function key don't do much. But I am confident that this can be settled out, they are sending keycodes. Calling acpitool manually isn't that much of an annoyance yet.
• Switching from OSS to ALSA: Another thing I was worried about. But I found out about the OSS emulation, so that reduced my worries. In the end I'm now even running without it, programs like mp3blaster I'm running with the wrapper from alsa-oss. Switched the few packages that were OSS specific on my system (like the SDL sound package) afterwards and am happy with it.
• Framebuffer module neofb works now built in! Finally I also can play around with some nice boot logo, and in connection with software suspend this isn't even wasted time. :)

This was my own christmas present. But I got some more, from a dear friend whom I'm working with and helping from time to time. More in a funny way I sent him a christmas wishlist, but he took it serious, and so there I am. Like, with a Gameboy Advance SP Kingdom Hearts Edition, including the game. I really loved the first part on my Playstation 2, am desperately waiting for the second part which shall be released in europe in spring next year... *sighs* The second major part was a Typhoon My Mini DJ music player. Most important thing for me with it is that it plays oggvorbis files natively. Finally I'm not the only one without a portable music player! Last but not least I'm going to attend LWE in Frankfurt/Main, and finishing my LPI Level 1 certification there. I'm not too worried about the test, though it for sure will be a good feeling to finally have something written and not only you lot telling me that I have some knowledge about it.

Something I'm really curious about is why the Java folks suddenly stood up and took notice of Ruby. Is it just Rails? Did Rails arrive in the right place at the right time? It's not a rhetorical question - I don't have a lot of insight into the Java world, so I'm genuinely curious as to what sparked this interest in "beyond Java". Python's been around for a long time, and isn't that different from Ruby. Tcl with Tk runs circles around Java for cross platform GUI development. PHP, even if it's not the cleanest or most glamorous thing out there, makes whipping up small web tools easy. (I'll concede that Perl has a culture that is too far removed from a lot of the things Java tries to do). And all those have been under everyone's noses for years. So why all of a sudden is Ruby the "hot new thing" that is to be embraced, rather "defended against", as we often see in such programming language "willy waving contests". Was this something that was just waiting to happen, and all it took was something to coalesce around? Were people in that world starting to get wrist injuries from typing out so much stuff, and on the lookout for "something else"? Joking aside, I really am curious, because over the years, I've developed an interest in how these things work. I talk about it some in The Economics of Programming Languages, but that's clearly not the whole story. Java and its human resources requirements Another thing I've been idly wondering about is whether perhaps Java got pushed a lot (perhaps unconsciously) by big companies, because 1) it is a good solution for big teams 2) some aspects of it are so big and unweildy that only companies that can muster the manpower to throw at it are going to do well with it, meaning that smaller shops would either have to not compete in that market, or compete poorly. Obviously, it's not strictly necessary to use Java - more often than not, you could accomplish the same thing in less time with some other system, however, the "other pincer" is the marketing out there that Java is the "serious, corporate, enterprise" solution, and that anything "less" just won't do. This gives firms an incentive to use Java even when it might not be the best solution, and helps create a market where it's a given that the solution is some Java system, and the bigger company, being able to bring more people to bear, may have an edge. Anyway, just some idle speculation - what do you think?

Yesterday, Tollef pointed me that he actually held the Debian Installer development lead for a while. Point taken, Tollef, I certainly didn't want to hide your contribution, I just wasn't aware of this part of D-I history. This lead me to come back at a small pet project: write down the history of the Debian Installer development. So, I went back to this short document I began to write a while ago. Not yet structured and just a collection of random pointers up to now. Given that my own involvment in D-I began only in October 2003 (funnily with a rant-style message to Joey which I'm now ashamed of), I'm missing the earlier parts of the story. Contributions welcomed.

(A continuation of my reminiscences as an expert witness.) In an earlier post, I made brief mention of the unpleasant realities of being a witness in a trial. More details, ahoy! Most witnesses aren't allowed in the courtroom except when they're giving evidence (to avoid bias) and have to sit out in the hallway. Bring a fat book, because you could be out there for days. I don't think much compensation is provided for your time, either. And, as a final indignity, the lawyers might not even end up calling you to the stand. For example, one of the prosecution witnesses in the case I was involved with was called to appear, and sat outside the court for eight (count them, 8!) days, before being told he wasn't needed after all, and he could go home. I don't know about you, but if that had been me, I would have ended up in the dock myself on a charge of mayhem. Another witness, after having been through her time on the stand, made a comment along the lines of "I'm going to make sure I'm overseas next time". Expert witnesses (as I was) get treated a little better. We can watch most of the bantering in court, while we evaluate the technical evidence presented by other witnesses (experts and non-experts alike), and at least the pay is sensible. As an occupation, "expert witness" would be OK. The hours are pretty sweet, the pay is good, and the conditions quite reasonable. For a regular geek like myself, though, it was fairly close to agony by the end of it. I've had three big projects on indefinite hold while I was at court, unable to give my clients a firm date when I could resume work for them. They were all understanding, bless them, but I could have lost out badly. The complete uncertainty as to what is going to happen and when means that you basically end up camping out at the courthouse for umpteen days. At least, in my case, I was being paid regular consulting rates for my camping. When I say uncertainty, I really do mean it. I was originally told I'd be needed for a day or two -- a quick look at what the other witnesses were saying, maybe jump in the box for a bit, and then I'm off. Then the first witness in the stand was there for nearly three days (along with a day or two of procedural stuff in amongst it so I had been there a week before he was finished). Then the defence was going to make a motion for dismissal, because the prosecution were being pummeled by his honour for their lack of case preparation, so I would have been out of there. That never materialised. Then more procedural faffage, another witness that ended up in the box for about 4 days -- each day of which I thought was the last -- and then it was my turn, which lasted nearly four days. At no stage did I have any indication of when it was going to be over, either, so I couldn't say with any sort of certainty when I could start helping anyone else out. While the legal system has a vital part to play, I'm wondering if something could be done to improve the running of trials to reduce this sort of pummeling of witnesses. On the sweet hours: court convenes each morning at 10am. There's usually half and hour to an hour of transcription correction (although I get the impression that isn't a standard thing) then witnesses start going through. There's a half hour morning tea break around 11 or 11:30, and then an hour lunch at 1pm. Usually there will also be an afternoon break of 10 to 15 minutes, before court adjourns for the day at 4pm (if not a bit earlier). I presume that the reason for the short days is that it's mentally very tiring, and there's lots of prep work for the lawyers mornings and afternoons, but it's a lot better life when you've got none of that to do. Unfortunately, the day isn't short enough that I could get a lot of other work done, so it's effectively just a short-paid day.

A lot of packages didn't make their way from the Debian security archive to the Debian FTP archive automatically during the past month. I just noticed this during a reverse check after preparing an update to the sarge release. I have absolutely no idea why the packages aren't uploaded automatically, but this does indeed suck a lot. Affected are 28 security updates listed below, summing up to 1.3 MB of .changes files and 1.1 GB of total files that are missing on spohr alias ftpmaster. And I've already fixed an upload yesterday, so 29 updates are affected. I have now uploaded them manually and moved them into the incoming directory on my own. This should really work automatically.
DSA 847 dia - missing input sanitising
DSA 848 masqmail - several vulnerabilities
DSA 851 openvpn - programming errors
DSA 853 ethereal - several vulnerabilities
DSA 854 tcpdump - infinite loop
DSA 855 weex - format string vulnerability
DSA 857 graphviz - insecure temporary file
DSA 858 xloadimage - buffer overflows
DSA 859 xli - buffer overflows
DSA 861 uw-imap - buffer overflow
DSA 862 ruby1.6 - programming error
DSA 863 xine-lib - format string vulnerability
DSA 864 ruby1.8 - programming error
DSA 866 mozilla - several vulnerabilities
DSA 868 mozilla-thunderbird - several vulnerabilities
DSA 870 sudo - missing input sanitising
DSA 871 libgda2 - format string
DSA 872 koffice - buffer overflow
DSA 873 net-snmp - programming error
DSA 874 lynx - buffer overflow
DSA 877 gnump3d - cross-site scripting, directory traversal
DSA 878 netpbm-free - buffer overflow
DSA 881 openssl096 - cryptographic weakness
DSA 883 thttpd - insecure temporary file
DSA 885 openvpn - several vulnerabilities
DSA 886 chmlib - several vulnerabilities
DSA 888 openssl - cryptographic weakness
DSA 890 libungif4 - several vulnerabilities In addition to that, the .changes file for openssl096 containing the source reference got lost on klecker. Without a source package, successful uploading is not possible.

<p> Just a few days ago, <a href="http://linux.duke.edu/projects/yum/">yum</a> entered <a title="Yum for Debian" href="http://packages.debian.org/yum">Debian</a>. Why is this interesting? </p> <p> Well, if you happen to be using <a title="virtual machine monitor" href="http://www.cl.cam.ac.uk/Research/SRG/netos/xen/">Xen</a> on Debian then you can easily create yourself a <a href="http://fedora.redhat.com/">Fedora</a> installation. </p> <p> You only need to perform <a title="Install a Fedora system in a Xen instance" href="http://www.fedoraproject.org/wiki/FedoraXenQuickstart#head-e41e7bbdd3c94366a44cf4ba609172f87b94d6d2">Step 5</a> on the procedures listed here after doing an <code>apt-get install yum</code>. Note, you'll want to make sure that rpm is setup correctly (<code>mkdir /var/lib/rpm; rpm --initdb</code>) before you begin. </p> <p> It is always interesting checking out the competition. Soon you be singing along to <i> <a title="Yummy yummy yummy" href="http://lyrics.rare-lyrics.com/O/Ohio-Express/Yummy.html">I've got love in my tummy</a></i> </p>

which is what was shipped with it, and I think there might be a newer firmware out there which fixes some of the issues. I just got my Nokia 770 and I must say it's a nice device. Some initial experiences are:

• No pen calibration on bootup. Nice and cool. All other handheld devices have that, but the 770 seems to work just fine without it. Kudos.
• It's not a PDA. No PIM thingy built-in, the device asks for its own name, but not the user's.
• It doesn't show that it's running Linux. I find this quite cool, as it shows that you can produce devices which run Linux and feel like they're targetted at non-geeks.
• The device feels sluggish and lacks feedback when doing stuff, whether it be closing an application, launching an application and so on. This makes it sometimes hard to know whether it has caught your action or not. Also, sometimes menus pop up for half a second, then goes away.
• The 770 has support for multiple applications at the same time. Works well enough, I guess.
• I want something to protect the screen, or I know it will develop scratches from the pencil over time. I guess I can get something from a shop in not too long.
• When putting the lid on, it's quite neat that it turns off the screen the moment the lid closes. I'll have to investigate how they do this. Hub thought it might be a magnet. Getting an X-Ray of the whole device would probably show.
• The feed reader needs work, I want to mark stuff as "read" explicitly and not just have a rolling list of news where I need to know where I was.
• Reduced-size MMC cards appear to be cheap. Around 30 for a 512MB card which should be enough for a fair amount of music. I need to find an Ogg Vorbis player as well.

All in all, mostly happy, but there are still a bunch of rough edges which should be smoothed away in a few more revisions. I'll see if I can get a development environment up and running on it, as well as getting a newer firmware onto the device (if I can find it; I wonder if the developer rootfs on maemo.org is newer or not..), but all that's for tomorrow.

Whilst I can’t claim to have done anywhere near as much for Gaim as Christian Hammond has, I did contribute to Gaim some time ago, and for my efforts I managed at least to attain the status of “Crazy Patch Writer”. However, I gave up trying to contribute to Gaim about three years ago. After starting out as the Debian package maintainer, and one of the founders of #gaim (yes, I know, I’m sorry, it wasn’t always like that), I spent about two years submitting patches which were ignored by almost all of the developers, being forced to beg people with access (including the support/bug triaging guy, who doesn’t really code C) to review and commit them. At the time, nobody was being given CVS access because of a recalcitrant and unresponsive former developer holding the reigns of the Sourceforge project. A few developers came and went, and in what I thought was a miracle, the now lead developer was made an administrator of the Sourceforge project and gradually started handing out access to other people who had been contributing. Except… I was passed over time and time again, and other contributors who had around for less time than myself were given access. I could accept this if I was given some justification, but I was ignored when I asked about getting CVS access and never given any reasons, despite having contributed hours and hours of my time and helped rewrite and clean up sizeable chunks of code. It actually had me close to tears on several occasions, and still upsets me a huge deal that no explanation was ever offered to me. After starting out with packaging work in Debian, Gaim was the first project where I became involved with actual development, and I learnt a great deal from hacking on it, but when I started at university I decided I’d spent enough time pouring my heart into such an unwelcoming recipient, and I moved on quietly.

Yesterday, Amtrak's board of directors fired Amtrak president David Gunn. Here's something unusual about this firing. David Gunn actually was the best thing to happen to Amtrak in years. He improved the physical condition of the railroad's equipment, raised ridership, cut costs, and even ended this year with a net surplus despite a crippling brake problem on Amtrak's high-speed Acela equipment that took the entire fleet out of commision for most of the summer. The Amtrak board knew this too, and even testified to that fact before Congress a few months ago. The board, though, is made up of Bush appointees. You might remember that Bush's budget proposal contained a plan to bankrupt Amtrak as a way of "reforming" it. That failed to pass Congress, and David Gunn failed to be incompetant. So you have the odd case of firing David Gunn for improving things on the railroad. Yikes. I will be so glad when this administration is gone.

I've been relatively quiet lately because I've been plodding along, working on modular packages for Xorg. For those who don't know, the next release of Xorg will actually be two releases of the same code. One release, 6.9, will be based on the old monolithic build system. This system requires me to build every single part of Xorg whenever I want to update any one part. This means that whenever I push out an Xorg fix, no matter which component it's in, you have to download all the other components even if they are 100% identical to the previous version already on your hard drive. We've all lived with this for a long time, and while it's not pleasant, we all seem to get by. The packaging apparatus for this system is mature, stable, and well understood, if complex. That's why I've been able to get the 6.9 release candidate packages out to experimental fairly quickly after they are released.

The other release happening is for 7.0, which is obviously a major version number bump. The actual code you run for 6.9 and 7.0 is 100% identical, but the build system has been totally rewritten around the venerable autotools (you know... configure /make/make install). Each piece has been split out from the others so that you can build any one component without building all the others. This means that if we want to ship a driver update, you don't have to download all of the fonts and other things that weren't touched. That also means that you won't have to install the drivers for cards you don't own if you don't want them. Furthermore, it lowers the barrier for things like hacking on X significantly.

The problem with the modular tree is that the packages for it are brand new. Ubuntu shipped modularized versions of the libraries, and I've already stolen those, fixed them up for Debian, and built them locally (they're sitting in our svn repo, but running them won't get you anything at this point.) The server and driver packages are brand new, and I've been taking the packages from Ubuntu's development branch and working on them for Debian. Today was a milestone day because in addition to the libs, with Daniel Stone's constant help I've finally managed to build packages for the modular X server itself. Now I can't run it yet (I haven't packaged the drivers and a few other bits) but the fact that I managed to produce what will be a functional server package is a huge step, and constitutes the cornerstone of the whole thing.

Once we can create packages that can actually run on at least the machines of the XSF, we can start looking at getting the packages in to the archive. Since these aren't well tested like the monolithic packages, we'll have to do a lot of polishing to get them to the same level of quality. My goal is to get that done for etch though, and I think we can do it. Exciting things are happening, and I'm hoping that the modular packages will encourage more people to hack on X, leading to even more exciting things. Build that momentum.