(Disclaimer: I work for MIT Information Services & Technology blah blah blah I am a student blah blah these are only my words and do not represent the opinion(s) of my employer.) Way back in the day in 1983, MIT started something it called Project Athena, which had the goal of investigating how computers could be used to enhance education at MIT. This project became the origin of such pieces of software as the X Window System and Kerberos, as well as a campus-wide network of workstations all running similar software. In 1991, Project Athena ended, but the Athena system continued on, taken over by MIT Information Systems. It’s 2008, and Athena still exists at MIT—walk into a computing cluster, and chances are all the machines will be running Athena software. (Unless that particular cluster has been taken over be Athena’s evil sibling, often referenced as “WinAthena”, which often seems to be maintained by no one, and whose existence is sometimes denied by those who maintain and support Real Athena. Not many machines have suffered this agonizing fate, luckily.) Unfortunately, as the years have progressed recently, fewer and fewer resources have been put into the development of Athena. In many ways it is no longer on the cutting edge of things as it has been in the past, though it is certainly still useful. Some students these days, however, even think of Athena as only “their MIT email” or maybe use it occasionally to print something in a cluster on the way to class. I’ve seen people grow frustrated and storm out of a cluster due to things like instructions for using a flash drive that involve more steps than just plugging it in. While on the one hand it’s easy to laugh at people for a lack of patience or being open to a computing system that is not either Windows or Mac and perhaps a bit more complicated on the surface, on the other hand, some of these things really should just work by now, and on other UNIX-based systems they do—but Athena has lagged behind. When you log into an Athena workstation for the first time, you are confronted with what looks like this: That’s GNOME 2.8, and it looks kind of like the GNOME 2.8 default but uglier. When you look under the hood, there are pieces of the system that obviously originated in a time where nothing better existed and so something new had to be created—the “xlogin” login system doesn’t support PAM, and the login path involves a hairy maze of scripts that perform various tasks. Under the hood, the base OS is currently RHEL 4 or Solaris 10. I still can’t remember what the right options are to feed RPM whenever I need some information that I think I should be able to get from it. These things were once necessary but today just look crufty. Some people have tossed around the idea that everyone has a laptop these days and thus clusters are uselessly expensive, but it’s been met with a lot of resistance. I’m all for progress, but I use clusters all the time! They’re one thing that makes it an easy choice for me to have a really portable laptop, because when I really need computing power or screen space on campus away from my room, there is always a cluster nearby. They’re also nice isolated places that can be associated with “work mode”, which is good for productivity. Beyond all this, Athena provides a common platform that all technical classes can depend on, from providing standard and specialized academic software to standing as a baseline of “make sure your code works on this before handing it in because that is how it will be graded”. At any rate, things are finally happening on this front. Athena can’t stay RHEL 4 forever—while AFS allows non-system software to be easily updated for all machines without requiring any OS release, hardware moves on and compatibility is lost, and the rest of the world is coming up with new awesome things that Athena machines are missing out on. A student group that I am vaguely involved with, SIPB, took the early initiative with this and created DebAthena, which takes stock Debian and Ubuntu and allows it to closely resemble an Athena environment. We even run a campus Linux dialup using the packages. (I won’t claim to have done much work on DebAthena myself because, well, I didn’t.) Amazingly, the Athena release team has decided to work with the SIPB on the next release of Athena. So, what I really just wasted over 600 words prepping for is to say that Athena 10 will be based on Ubuntu. This is only relatively new news, but I am lame about finishing blog posts. There are a variety of reasons why Ubuntu was chosen over Debian. Some of them are lame and some of them are not. Even so, I am excited about this prospect—and it should be possible to peripherally support Debian in such a way that was not possible with Athena 9.4. I really like Athena. Even coming to MIT with a background in Linux there has been a steep learning curve, though, and I think that things could be better. Lots of people here have their first encounter with a ‘nix system in the form of Athena, and we should strive to make that experience as good as possible. I’m kind of skeptical on the expected release date of this upcoming summer, but as long as it happens soon it will be a big step forward. In some ways it’s kind of sad that I’m excited about a big part of a research institution catching up with others—but inertia is a strong force, and dealing with thousands of workstations is no trivial task. I kind of long for the exciting days of creating new big pieces of free software like X. I bet I imagine those days as being better than they actually were.

It's been a while, but a new update to DebTorrent is available in unstable. There is also a new version of the helper apt-transport-debtorrent (0.2.0) to go with it, which was mostly the cause of the long delay (due to a pending apt transition to testing). It is now highly recommended to install both, read below for new reasons why. One of the previous problems with debtorrent was the incorrect status updates displayed by apt, due to it not being aware of pieces of large files that have already downloaded. Using these new versions together will help to improve the update status messages shown during a download. Now, apt-get update will display a status at the bottom that will look something like:

DebTorrent: 837MiB left at 319 KiB/s (46m03s)

You may not be able to see it all if there are multiple updates to different mirrors under way. In aptitude, the status will appear on a line by itself at the bottom of the list of downloaded files. The status line will update about once per second, and should give a good indication of how much time is really left. The other (incorrect) status information presented by apt/aptitude will still be there but you can ignore it. I've only tested apt and aptitude, so I'm not sure how (or if) synaptic, adept or gnome-apt will display these messages, though they are general apt status messages, usually used for displaying the status while logging into an FTP server. This should also fix another problem that debtorrent can have with downloading large files (usually on slower connections). Since apt can't see the download happening for a long time, the connection would sometimes time out. With these status updates the connection is constantly active, so the timeouts shouldn't be a problem when downloading large packages (although they may still occur when doing an apt-get update, as there are no status updates during that process). I did implement a better solution, as suggested by Micha Politowski on the wiki. This solution uses the creation of sparse files to let apt know how much of a file has actually been downloaded. However, apt doesn't understand how a single process can download multiple files in prarallel, so it only follows the last file that was started, and complains when a file finishes that was not the last one that started. I've disabled that for now while I see if it's possible to modify apt to work better in this situation. Here are the changes for debtorrent:

• Update to support apt debtorrent transport version 0.2
  • send piece downloaded status messages (currently disbaled due to apt not liking it)
  • send general status update messages
• Fix some minor packaging issues
• Upgrade to standards version 3.7.3 (no changes)
• Remove the unneeded binary-arch rule
• Changes the XS-Vcs-* headers to Vcs-*
• Moved Homepage from description to Source package fields

And for apt-transport-debtorrent:

• Upgrade transport version to 0.2
  • Create sparse files based on status messages (102) from DebTorrent
  • Send general status updates to apt (103) from DebTorrent
• Remove support for Range headers as they may confuse the sparse file allocation
• Fix typo in long description
• Upgrade to standards version 3.7.3 (no changes)
• Changes the XS-Vcs-* headers to Vcs-*
• Moved Homepage from description to Source package fields

Dear lazyweb, In bzr development we are now working primarily on network performance. One of the key things about being sure we have improved things is automated, repeatable benchmarks. And for that to be useful in networking environments we need to control latency and bandwidth and packet loss. I know this isn't a new problem, but it was about 5 years ago that I last did this sort of thing. What are the best tools today (for linux :)). Ideally I'd be able to bring up a bunch of local addresses like 127.0.1.1 or 127.0.0.2, with different properties - such that traffic from 127.0.0.1 to 127.0.0.2 will simulate being uploaded over adsl, and 127.0.0.2 to 127.0.0.1 will simulate being downloaded over adsl.

You are probably using DHCP on the machine currently in front of you. The Dynamic Host Configuration Protocol is a way for your computer to obtain an Internet address from a pool of available addresses, and to return it to the pool when you no longer need it. Basically every Internet service provider uses DHCP, or something similar. As a network operating system, Linux has DHCP support (and had it for ages). In true Unix fashion, Debian sports at least four DHCP clients. Debian s default is dhcp3-client, also known as dhclient. The theory is that the client requests an address lease from the server and periodically renews it. This process yields a number of events, to which the operating system can react. For instance, initially, the client issues a PREINIT event to get the interface into a state where it can talk on the network, and a BOUND event as soon as it acquired a lease, or FAIL if it, uh, failed. After a certain period of time, the client tries to renew the lease. If it succeeds, it issues a RENEW event; if it fails, it yields EXPIRE. So much for the theory. It seems that dhclient is rather stupid, which I tried to document in bug #459813 it does things differently: given a lease, after a certain period of time, it just issues an EXPIRE event, which causes the operating system to deconfigure the interface and take down connectivity. Then, the client spits out a PREINIT event, followed by BOUND or FAIL, as appropriate. I have not quite investigated what all this means, but this much is for sure: periodically, your machine goes offline, only to come back online a second later. If this were Windows, one would probably knock on wood and be glad that it works at all. But we re on Linux here, Debian even, so this cannot be. I d love to be proven wrong, so if you have a minute, please try to verify. One way of doing so is to insert

echo "$(date) got $reason" >> /tmp/dhclient-script.reasons

towards the top of /sbin/dhclient-script and monitor the output file. Once your client renews, it should read:

Wed Jan  9 11:53:27 CET 2008 got RENEW

but instead you ll see

Wed Jan  9 11:53:27 CET 2008 got EXPIRE
Wed Jan  9 11:53:28 CET 2008 got PREINIT
Wed Jan  9 11:53:33 CET 2008 got BOUND

and if you look closely enough, your interface will be unconfigured those seconds between EXPIRE and BOUND. I only spent about 2 hours with dhclient last night, as part of my efforts on netconf, but I already found bug #459813 and bug #459818, two bugs which don t suggest that dhclient is quality software. If I find more, I ll have to consider a different client, hoping that there are others which separate the DHCP client from the operating system calls in a similar way as dhclient does. NP: The Flower Kings: @Live Recording, Uppsala City Theatre, Sweden, 10 February 2003

Not a whole lot to report. Sarah's still yet to have a decent night's sleep. The cardiac rehab lady came by to go through the do's and don't's and answer any questions we had. They did an echocardiogram. I tagged along to watch. The Dacron looks a bit whiter on the ultrasound than the aortic tissue, other than that, there's not a lot to see. It's looking like she'll be discharged tomorrow.

Some people have been asking about the weather in Melbourne in late-January in terms of what to wear for Linux.Conf.Au. It is probably impossible to predict weather for a particular day this far ahead. But predicting a range for the week is not difficult. I think that you should expect at least one day that is really hot with a peak of 37C or more and reasonable humidity with a possibility of another two days the same or similar. A day with a peak of 42C or more is not unlikely over the course of a week. You should expect a range of temperatures, one or two days that are reasonably cool with a maximum of 25C would not be unexpected. Some heavy rain in short bursts is a possibility (based on the past few weeks - prior to that there was little rain and it’s possible that there may be some time without rain again), there is probably no need for a rain-coat if you have the option of waiting ~30 mins for the rain to pass before going outside. I expect that if there is any rain at a time when conference delegates are about to go out somewhere then things will be delayed. I suggest that you wear jeans while on the plane but expect to wear shorts for your entire time in Australia. A t-shirt is a reasonable option but if you plan to be outside much then wear a long-sleeved shirt. As I don’t expect to be doing much work in traditional offices in the near future I’m wearing business shirts when I go outside, long sleeves with a collar is good for protecting against sun-burn and as they are light they keep me cool (t-shirts are tighter and thicker and keep you hot). However when at LCA I will be wearing t-shirts that I designed (which should be well suited to being inside and I don’t plan to do much outside during that week). A Scott e Vest [1] is a good thing to wear. It has heaps of pockets for your electronic gear, is reasonably light, and can be worn on top of a t-shirt. The Scott company also sells a TEC shirt which is a long-sleeved shirt with plenty of pockets. I’ve had a Scott e Vest for a number of years and I might have to get myself a TEC shirt.

• [1] http://www.scottevest.com/

Share This

I'm about to release debhelper 6.0.0. But this blog is really about CDBS, and why it's entirely unlike debhelper despite being based on it.

---

With CDBS you're building a ship in a bottle. You have all the same tools available, but there's this bottle in the way, which doesn't let you bring any of them fully to bear, and requires all your techniques be changed around to fit its limited interface. This is a beautiful example of the kind of fun you're setting yourself up for by chosing CDBS. With debhelper, two commands can be easily reordered. With CDBS, you need three magic environment variables and either a hardcoded package name or some grody Makefile code: http://lists.debian.org/debian-python/2008/01/msg00039.html

Sarah slept better overnight, but still not great. She was certainly more lively this morning when I got in, than yesterday. She's resigned to the fact that you don't come to a hospital to get rest, which is kind of ironic. She did need to get put back on the oxygen overnight because her O₂ saturation dropped. She was off it again by the time I got in. Today's big achievement was getting the external pacing wires removed. They were put in during the procedure as a precaution, I think because if her heart played up, they didn't want to have to use defibrillator paddles, and because they'd cracked her chest open, manual CPR wasn't an option. The wires could be attached to an external pacemaker if necessary (but it never was). I watched the removal quite closely, as I was pretty curious. Basically the physician's assistant just came in and yanked them out. Judging from the look on Sarah's face, it was an interesting sensation. We were both amazed at how long the wires were. There was probably 30 centimetres of wire under the skin. The last bit was loosely spiralled. I was just really surprised, because the PA's hand started off at Sarah's chest, and ended up at the PA's shoulder. It was just a lot longer than we both expected. All the extra length was because the wires wrap around the heart. The wires came out at about 2:30pm. Sarah had another chest x-ray and a CT scan today. The CT scan looked good. The x-ray showed some fluid in her lungs, and her lung capacity is still sub-par. It seems when she's fully horizontal, her breathing isn't so crash hot, but it's fine when she's sitting up or walking around. All Sarah's got left is one peripheral line now. There was some talk of her possibly going home on Wednesday. She has an echocardiogram tomorrow, and I guess it'll all hinge on how her lungs are going.

Since September 19th I have been mostly offline and unreachable. This was due to having started a new job in a wonderful new project: soitu.es, an online newspaper with a strong focus on Citizen Journalism: citizens playing an active role in the process of collecting, reporting, analyzing and disseminating news and information, I must mention the master mind behind this all: Gumersindo Lafuente, one very well known journalist in Spain. On the other hand, working with Ra l Rivero, not the Cuban poet but the technical soul of the project, is something I would pay for, and I get actually paid to learn from him! Too bad he doesn t digg Debian :)



The site was finally launched during Xmas, and we are all less busy, so despite having what I call the weekend flu (that only affects me during the weekends, but I never feel so bad that I need to miss work during the week), I am back to normal, If you thought I was quite MIA, keep in mind I was not one of busiest ones at work! Launching such a project takes a toll on everyone, though. I set up a Flick set for my new workplace: Amayita s Soitu.es Flickr photo Set, in case you are curious. There is also an official Soitu.es Flickr Photo Pool.

Today is a holiday in Spain, due to D a de Reyes, when kids usually get their Xmas presents here. As I am fully recovered from this weekend s flu (today is Monday after all), I had some spare time and my cats got a nice present too: A DIY scratching post. There is a WTF column in the effing middle of my kitchen, so after some cardboard, a wall stapler, sisal rope and a bit of round and round the column... This pic shows the result:



I couldn t resist filming a video, available on YouTube: This scratching post is now mine! - No! Mine! - I ll get you!

The Frontal Cortex provides an astute explanation as to why he shies away from political blogging in this election season:

My hypothesis is that political judgments are like moral judgments. When you see a candidate, you experience a visceral, instinctive, inexplicable response. Your brain generates an emotion - Obama is uplifting, Hillary is commanding, McCain is honorable, etc. - and then the rest of your brain goes about explaining your emotion. The inner interpreter gathers together bits of evidence, post hoc justifications, and pithy rhetoric in order to make our automatic reaction seem reasonable. But this reasonableness is just a fa ade, an elaborate self-delusion.

Personally, I haven’t done much (any?) such blogging lately for lack of time, but I also generally agree with his conclusions. “A man hears what he wants to hear and disregards the rest.” Technorati Tags: Politics, Elections, Neuroscience

Share This

So, after my previous entry, here's now a small list of the oldest bugs still opened in Debian BTS:

• #1555, Package: dselect, "dselect per-screen-half focus request". There is indeed no more information left in that bug and I wonder whether it is useful to keep it opened.
• #2297, Package: xterm, "xterm sometimes gets mouse-paste and RETURN keypress in wrong order". Seems reproducible and puzzling a lot of people.
• #2298, Package: trn, "trn bug with shell escaping". trn has been replaced upstream by trn4, apparently, so we're here talking about an old dusty software (installed by 44 ppl, still).
• #3175, Package: xonix, "xonix colors bad for colorblind".
• #3180, Package: linuxdoc-tools, "linuxdoc-sgml semantics and formatting problems". There is not info in this bug report about what the bug could be. To close?
• #3251, Package: acct, "accounting file corruption". Tagget upstream, wontfix. Should this be closed more than 11 years later or is someone still hoping that this is useful?

I received a number of comments on my previous blog post, with people stating that my example code wasn't right, either. For reference, here's my code snippet again:

#include <stdio.h>
void* f();
int main()  
	long* a;
	a=(long*)f();
	printf("%Ld\n", *a);
	return 0;
 
void* f()  
	static long a=84;
	printf("%Ld\n", a);
	return &a;
 

First, let me make one thing perfectly clear: the "a" variable is not on the stack. It is a locally-scoped variable, but it is also declared as static, meaning that it's valid for the entire run of the process; if it were on the stack, then this variable would vanish when the function would end. Yes, it's wrong to return a pointer to a variable on the stack, since there's no guarantee that the stack value will still be valid; but I'm not doing that. Alternative implementations could've used a global variable, or could've malloc()'ed a pointer; but the compiled assembly code for my C code will only differ from a version with a global variable in the used label for the variable's location, and in the lack of an extra function call from a version that uses malloc(). Second, no, using A0 as a return value for pointers is not a bug in the compiler, and has not been since 1990, when this interesting book entitled "System V ABI Motorola 68000 Processor Family Supplement" was written. Allow me to quote a snippet from page 3-14 of that book:

%a0	Pointer return values appear in %a0. When calling a function that returns a structure or union, the caller allocates space for the return value and sets %a0 to its address. A function that returns a structure or union value places the same address in %a0 before it returns.

To this day, ELF-conforming implementations do it this way. Really. Go look it up, if you wish; the ISBN is 0-13-877663-6.

In rough chronological order, with capsule reviews. I don’t think I’ve missed any, but it is possible that I have. If you look through the list (behind the cut), you’ll see I read mostly Baen books nowadays. There’s a simple explanation: Baen is (almost) the only publisher that does real e-books, and I tend to avoid the inconvenience of paper books where I can. Still, there’s about a hundred titles on the list. (more…)

Chris, buildd hosts keep those time and space usage statistics in a database, which can be queried...

Script started on Sat Jan  5 19:54:15 2008
wouter@country:~$ ssh kiivi.cyber.ee
Linux kiivi 2.6.18-4-mac #1 Fri Mar 30 23:05:11 CEST 2007 m68k
No mail.
Last login: Sat Jan  5 20:41:16 2008 from d51532c45.access.telenet.be
wouter@kiivi:~$ avg-pkg-build-time -s -t   head -n 20
boost:			2015312k (2015312k lastest)
iceweasel:		1846628k (1846628k lastest)
iceape:			1674172k (1674416k lastest)
icedove:		1624848k (1624848k lastest)
xulrunner:		1611880k (1611880k lastest)
mesa:			1230432k (1230432k lastest)
koffice:		1208062k (2115156k lastest)
gcc-snapshot:		1191866k (1207164k lastest)
gcc-4.1:		1185620k (1185620k lastest)
gtk+2.0:		1148634k (1208684k lastest)
k3d:			1145580k (1145580k lastest)
ardour:			1077800k (1077800k lastest)
lyx:			924876k (924876k lastest)
gcc-4.2:		909488k (909488k lastest)
gcc-3.4:		892015k (571084k lastest)
glibc:			891408k (891408k lastest)
linux-2.6:		725961k (744700k lastest)
kdebase:		720618k (1106804k lastest)
kdelibs:		716376k (1150372k lastest)
kdepim:			711858k (784356k lastest)
wouter@kiivi:~$ avg-pkg-build-time -t   head -n 20
gcc-snapshot:		220:52:14 (2 entries, sigma 27:38:21)
axiom:			216:27:57 (1 entry, sigma 00:00:00)
gcc-4.1:		193:47:30 (1 entry, sigma 00:00:00)
k3d:			177:22:10 (1 entry, sigma 00:00:00)
gcc-3.4:		171:40:28 (4 entries, sigma 25:10:55)
boost:			141:13:38 (1 entry, sigma 00:00:00)
koffice:		140:40:23 (2 entries, sigma 53:48:58)
ace:			130:04:14 (1 entry, sigma 00:00:00)
linux-2.6:		129:36:19 (3 entries, sigma 10:12:24)
qt-x11-free:		105:00:32 (1 entry, sigma 00:00:00)
xulrunner:		97:03:38 (1 entry, sigma 00:00:00)
iceape:			96:23:13 (2 entries, sigma 03:12:58)
kdepim:			96:06:54 (2 entries, sigma 19:09:39)
icedove:		94:35:48 (1 entry, sigma 00:00:00)
iceweasel:		93:56:26 (1 entry, sigma 00:00:00)
gcc-4.2:		91:08:32 (1 entry, sigma 00:00:00)
glibc:			91:01:58 (1 entry, sigma 00:00:00)
kdevelop3:		84:27:21 (2 entries, sigma 02:47:35)
kdeedu:			82:09:55 (3 entries, sigma 10:59:18)
vtk:			81:44:20 (3 entries, sigma 00:46:32)
wouter@kiivi:~$ logout
Connection to kiivi.cyber.ee closed.
wouter@country:~$ ssh arrakis
Linux arrakis 2.6.12-1-amiga #1 Sat Aug 13 22:59:59 CEST 2005 m68k
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please read "links2 http://nefud/info.txt"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No mail.
Last login: Sat Jan  5 18:48:58 2008 from hahn.os.localnet on pts/5
wouter@arrakis:~$ avg-pkg-build-time -s -t   head -n 20
wxwidgets2.6:		1648020k (1648020k lastest)
qt4-x11:		1601086k (1753532k lastest)
gcj-4.1:		1135772k (1135772k lastest)
k3d:			1094345k (1159204k lastest)
kdebase:		1087375k (1087368k lastest)
python-kde3:		1018144k (1018144k lastest)
ghc6:			991524k (991524k lastest)
gcc-3.3:		988690k (990244k lastest)
glibc:			814536k (855720k lastest)
lyx:			804672k (804672k lastest)
bouml:			729888k (729888k lastest)
kdelibs:		685565k (1142304k lastest)
freedroidrpg:		673988k (673988k lastest)
mozilla:		663066k (636896k lastest)
ardour:			647580k (691128k lastest)
mozilla-thunderbird:	591384k (591384k lastest)
omniorb4:		576288k (576288k lastest)
subversion:		568764k (624768k lastest)
xorg-server:		527520k (527520k lastest)
mozilla-firebird:	498896k (502692k lastest)
wouter@arrakis:~$ avg-pkg-build-time -t   head -n 20
enlightenment:		198842:37:14 (3 entries, sigma 344403:17:35)
axiom:			164:43:54 (1 entry, sigma 00:00:00)
ghc6:			159:20:15 (1 entry, sigma 00:00:00)
k3d:			145:18:35 (3 entries, sigma 57:08:46)
python-kde3:		126:39:34 (1 entry, sigma 00:00:00)
qt4-x11:		121:26:11 (2 entries, sigma 03:07:35)
gcj-4.1:		116:05:34 (1 entry, sigma 00:00:00)
gcc-3.3:		108:45:19 (3 entries, sigma 02:01:50)
qt-x11-free:		99:12:17 (4 entries, sigma 14:12:32)
wxwindows2.4:		90:26:51 (1 entry, sigma 00:00:00)
openscenegraph:		74:37:21 (1 entry, sigma 00:00:00)
ardour:			63:44:55 (2 entries, sigma 07:23:05)
bouml:			62:01:16 (1 entry, sigma 00:00:00)
gcc-snapshot:		61:10:07 (1 entry, sigma 00:00:00)
mozilla-thunderbird:	60:47:29 (1 entry, sigma 00:00:00)
mico:			60:15:15 (1 entry, sigma 00:00:00)
kdebase:		58:41:04 (5 entries, sigma 08:00:17)
glibc:			58:04:26 (6 entries, sigma 40:01:12)
mozilla-firebird:	52:47:13 (2 entries, sigma 11:31:07)
zeroc-ice:		52:34:03 (2 entries, sigma 28:12:19)
wouter@arrakis:~$ logout
Connection to fremen-os.dyndns.org closed.
wouter@country:~$ exit
Script done on Sat Jan  5 22:28:46 2008

"avg-pkg-build-time" is a script that queries the statistics database. It's quite CPU-intensive, so I don't like to run it too often (certainly not from cron); but, hey, here you go. In case you didn't know, "arrakis" and "kiivi" are both official m68k/unstable buildd hosts, and especially arrakis has been for quite a long time, although it's been reinstalled once or twice since. Oh, and yes, it does appear as though some bits of the data are invalid, but most of the idea should be alright. (oh, and before you ask, no, it didn't take two and a half hours to run those commands -- I just needed to go while the last one was running, and had only come back several hours later)

Version 0.9.0 of QuantLib was released a couple of days ago on Christmas Eve; as usual, Debian packages for QuantLib were uploaded right away following a few earlier pre-releases. As QuantLib is approaching its 1.0 release, a few API changes requires updates to basically all of RQuantLib's C++ source files. Luckily most changes were minor. At the same time, we also generalised the Binary (aks Digital) option pricer to allow for a 'binType' argument (with values 'cash', 'asset' or 'gap' for CashOrNothing, AssetOrNothing or Gap digitals) as well as an 'excType' argument to switch between European and American exercise. Dominick made a small change to the DiscountCurve object to seamlessly pass a switch variable indicating whether we have 'flat' curves or not. Another change was the addition of formal unit tests using the RUnit from CRAN (which we happened to have added to Debian recently in the wave of new RMetrics packages). We use the scheme initially proposed by Gregor Gojanc and extended by Martin Maechler for RMetrics that allows the unit tests to be run from both the source and the installed package which is nice. As QuantLib itself has a massive amount of unit tests in its code; I am hoping to add more and more of those into RQuantLib itself as we add more functionality. On that front, more exciting news: RQuantLib is now hosted on R-Forge. Potential contributors are encouraged to register at R-Forge and to get in touch -- this is a great way to learn about combining C++ and R. To wrap up, the new version 0.2.8 is currently in the queue at R's master CRAN host and should hit the CRAN mirrors shortly; likewise the Debian package has been uploaded and should also propagate to Debian mirrors in due course. As usual, source are also available locally on my site.

Sorted by build-time

1. openoffice.org - 7h02
2. gcc-4.2 - 5h58
3. gcc-snapshot - 4h17
4. gcc-4.1 - 3h30
5. ghc-cvs - 3h16
6. linux-2.6 - 2h34
7. gpc-4.1 - 2h25
8. gclcvs - 2h21
9. glibc - 2h15
10. python2.5 - 2h09
11. axiom - 2h07
12. gnat-4.1 - 2h06
13. gcj-4.1 - 1h52
14. gromacs - 1h51
15. mysql-dfsg-5.0 - 1h46
16. acl2 - 1h44
17. gcj-4.2 - 1h34
18. ghc6 - 1h33
19. python-kde3 - 1h32
20. aegis - 1h25

Sorted by build-space

1. openoffice.org - 7217MB
2. nel - 2773MB
3. gcc-snapshot - 2499MB
4. koffice - 2247MB
5. wxwidgets2.6 - 2151MB
6. linux-2.6 - 2150MB
7. webkit - 2059MB
8. glibc - 2028MB
9. xulrunner - 2023MB
10. iceape - 2007MB
11. boost - 1983MB
12. mesa - 1943MB
13. gcc-4.2 - 1943MB
14. qt4-x11 - 1915MB
15. python-kde3 - 1847MB
16. gcc-4.1 - 1828MB
17. gcj-4.1 - 1783MB
18. eclipse - 1763MB
19. iceweasel - 1727MB
20. gcj-4.2 - 1722MB

No real surprises here. The data was collected from the Buildd logs using the latest “maybe-successful” i386 or amd64 build. Factoring into that the varied nature of the buildd machine specifications make this is data somewhat unreliable… It would be nice if someone could obtain (and post) results from a more, umm, controlled environment.

Anthony Towns wrote about user configuration (i.e. ~/.foo) and the XDG spec/proposal versus his own (and according to him) simpler version. The main goal of all this is to get rid of all the ~/.foo directories and files in a users home directory. Apart from technically minor differences, I don’t see why AJs proposal would be simpler than the XDG approach. All it does is hardcoding the values which are adjustable in XDG via XDG_*_DIRS. There is one notable difference though: AJ proposes fallback values which are compatible with existing application paths (i.e. HOME_ETC/XDG_CONFIG_HOME should default to ~/ - resulting in ~/.foo files/dirs). This is a good proposal in a way: If you patch an existing application to strictly follow the proposal made by AJ, it still finds its old configuration (and other files) if the new environment variables are not set. No need to move anything around. However, I think the XDG approach is better since it makes sure that compliant applications don’t clutter ~/ anymore. They do need some mechanism to move existing configurations and data around though if the XDG-compliant stuff isn’t already there, but an old style config exists. All in all, I think it is worth following the XDG spec, even if it is slightly more complex than AJs proposal. For one, it already exists for a while and I think that some applications already started following that spec. It also makes slightly more sense to me to have the system fully configurable as to where the apps store data as opposed to the hardcoded fallback/default values AJ proposes (even if we would change AJs proposal to use “$HOME-cleaning” default values).

Inspired mostly by Joey’s nonchalant way of dealing with the death of his laptop…

This seems less of a disaster than other times a laptop’s disk has died on me. When did it start to become routine? […] My mr and etckeeper setup made it easy to check everything back out from revision control. […]

…I’ve been looking at getting all my stuff version controlled too. I’ve just gotten round to checking all my dotfiles into git, and it crossed my mind that it’d be nice if I could just set an environment variable to tell apps to create their random new dot-files directly in my “.etc-garbage” repo. I figured using “$USER_ETC/foo” instead of “$HOME/.foo” would be pretty easy, and might be a fun release goal that other Debian folks might be interested in, so I did a quick google to see if something similar had already been suggested. The first thing I stumbled upon was a mail from the PLD Linux folks who apparently were using $HOME_ETC at one time which sounded pretty good, though it doesn’t seem to have gotten anywhere. That thread included a pointer to the system that has gotten somewhere which is the XDG spec. It’s actually pretty good, if you don’t mind it being ugly as all hell. They define three classes of directory – configuration stuff, non-essential/cached data, and other data. That more or less matches the /etc, /var/cache and /var/lib directories for the system-wide equivalents, though if the “other data” is stuff that can be distributed by the OS vendor it might go in /usr/lib or /usr/share (or the /usr/local/ equivalents) too. Which is all well and good. Where it gets ugly is the naming. For the “/etc” configuration stuff, we have the environment variable $XDG_CONFIG_HOME, which defaults to ~/.config, and has a backup path defined by $XDG_CONFIG_DIRS, which defaults to /etc/xdg. For the “/var/lib” other data stuff, we have the environment variable $XDG_DATA_HOME, which defaults to ~/.local/share, and has a backup path defined by $XDG_DATA_DIRS, which defaults to /usr/local/share:/usr/share. (Though if you’re using gdm, it’ll get set for you to also include /usr/share/gdm) And for the “/var/cache” stuff, we have the environment variable $XDG_CACHE_HOME, which defaults to ~/.cache. That seems to me like exactly the right idea, with way too much crap on it. If you simplify it obsessively – using existing names, dropping the desktop-centrism, you end up with: Put configuration files in $HOME_ETC/foo or $HOME/.foo. For shared/fallback configuration, search $PATH_ETC if it’s set, or just /etc if it’s not. Put data files in $HOME_LIB/foo or $HOME/.foo. For shared data, search $PATH_LIB if it’s set, or look through /var/lib, /usr/local/ lib,share and /usr/ lib,share if it’s not. Put caches in $HOME_CACHE/foo or $HOME/.foo. For shared caches, search $PATH_CACHE if it’s set, or just look in /var/cache if it’s not. That seems much simpler to me to the point of being self-explanatory, and much more in keeping with traditional Unix style. It’s also backwards compatabile if you use both old and new versions of a program with the same home directory (or you happen to like dotfiles). And having the XDG variables set based on the above seems pretty easy too. I wonder what other people think – does HOME,PATH _ ETC,LIB,CACHE seem sensible, or is XDG_ CONFIG,DATA,CACHE _ HOME,DIRS already entrenched enough that it’s best just to accept what’s fated?

Software becomes more and more important in science as in other areas of life. Scientist have a tradition to publish their work very openly but that does often not include the source code of the software that was developed to carry out simulations which has some obvious problems such as:
- other scientists cannot check the software for errors,
- other scientists cannot fix the bugs and easily reproduce the results,
- other scientists cannot base their new research on already existing software and have to write it completely from scratch again and again,
- software package from different authors cannot be combined easily.

But things are getting better. One field of scientific research where we can see some improvement is machine learning - http://en.wikipedia.org/wiki/Machine_learning which is a broad subfield of artificial intelligence and concerned with the design and development of algorithms and techniques that allow computers to "learn". S ren Sonnenburg et.al. wrote a paper about "The Need for Open Source Software in Machine Learning" which is available at http://jmlr.csail.mit.edu/papers/v8/sonnenburg07a.html. They even created a portal with the goal to support a community creating a comprehensive open source machine learning environment at http://mloss.org.

An increasing number of software package are available in Debian like
- some simple-to-use utilities to apply compression techniques to the process of discovering and learning patterns: http://packages.debian.org/sid/complearn-gui
- a python package for convex optimization: http://packages.debian.org/sid/python-cvxopt
- a library for support vector machines: http://packages.debian.org/sid/libsvm2
- a machine-learning library: http://packages.debian.org/sid/libtorch3-dev
- an object-oriented programming language designed for researchers, experimenters, and engineers interested in large-scale numerical and graphic applications: http://packages.debian.org/sid/lush
- a large scale machine learning toolbox: http://packages.debian.org/sid/shogun-python-modular
- a data mining software in java: http://packages.debian.org/sid/weka

I'd like to know if you are using some of the packages or some other scientific software in Debian. Feel free to leave comment. Or maybe you are missing something in Debian?

If you are an author or user of some free software related to the topic of machine learning please consider registering it at http://mloss.org.

Happy new year etc. Dear Lazyweb, how do I set up java so that it either uses arts or properly uses software mixing so that I can have sound in parallel between Java and other applications? (At least, I assume that Java is the culprit, because I can have parallel sound between flash within firefox, which I assume not to use arts, and arts based stuff. I could be wrong on this, though, so the question might be how to set up arts to not lock sound exclusively...) (Yes, the blog doesn't take comments. Drop me an email and I'll update this entry.)