Yesterday I got my laptop back. It was in service since the 6th of December.

They replaced the battery, and although it looked like the built-in charge indicator was defect, actually the button on this battery is buried deeper than it was on the previous one and I have to press really hard on it to trigger it.

According to the tests made yesterday night, this battery holds for about 2.5-3 hours when playing music. Also it has 96% of its designed capacity. I guess is OK enough since I don't want to be spared from my laptop anymore.

Next on the agenda:

• restore data sanity (I did clean up most of the data from the HDD before handing down the laptop for the case when I was forced to hand it over; lucky for me, I managed to keep the HDD)
• install armel Debian on my new[1] NSLU2, Kinder (more news on that later ;-) )
• fix my local network chaos triggered by the problems that hit Ritter (my older NSLU2)
  
• finish the wiki theme
• work a little more on svn-buildpackage and kill more of its bugs
• walk through my (game) packages' bugs and try to fix them, answer, etc.
• try to make bluetooth transfers to work from and to my laptop

BTW, I still hold to my opinion that Dell and Emag suck, although I might buy desktops from Dell, laptops are a no-no until they fix their broken batteries and their broken "1-year warranty for batteries" policy.

[1] I bought a new NSLU2 after I left my laptop in service and it has been waiting since then to run the Debian Arm EABI (armel) port

At casa Geier, there is a slightly broken Asus A3G laptop, formerly known as salmiakki. The symptom is that upon pressing the power button, the machine ponders for a few seconds, then decides to turn off again. Since these laptops come with CD playback function keys on the outside, I think this is the safety feature against accidentally powering on; it just turns off again when the lid is closed. Which makes me suspect that only the lid switch is broken. Trouble is, there is no switch. This thing uses black magic to find out if it's open. There are no wires going to the hinges, there is no magnet either, or anything else that looks like it could be used in this way. So dear lazyweb, I'm interested in any hints how this might work. Pure speculation is also fine.

[ Apologies for Debian planet readers expecting something pithy and Debian related. This isn't. But then pkg-xfce packaging just continues. We get more bugs, we fix some of them (if you have a dual-headed setup and want to help us fix or reproduce more we'd love to hear from you). Corsac became a DD at last and has made me more or less redundant in a good way. I should probably investigate libburnia again and prod George Danchev about #450873 since basically it seems to just need the ubuntu packaging brought across into Debian to replace libburn etc. But anyway, on with the irrelevant stuff... ] Books part 1 was back in April and I've since found myself with some time on my hands before I get a new job so here we go again.

Making Money - Terry Pratchett

This was a Christmas present and I quite enjoyed it and enjoyed the character but didn't really think it lived up to the laugh a minute Pratchett books that I remembered from the good old days.

The Lovely Bones - Alice Seebold

This is quite a weird concept for a book given that it's from the point of view of a dead girl in heaven but it seems to work. It's very well done and I enjoyed it.

Alex Rider series - Anthony Horowitz

I had seen Stormbreaker and wanted to read some more of these as light holiday reading. They work well for that. There's enough plot to keep me interested but not enough to make them at all hard to read. I read a couple of them in French when I was in France (in between traipsing between different bits of Paris since the m tro workers were on strike). I wish they'd been around when I was younger.

My Sister's Keeper - Jodi Picoult

Ooh this is a really, really good book. I loved it except perhaps for the very end but I can forgive it that. The idea is that she's suing her parents for the rights to her own body because she was conceived as a donor for her sister to fight off her sister's leukaemia. It's a very thought provoking read with several interesting characters with their own stories woven together.

The Language Instinct - Steven Pinker

This book is fascinating to me. It takes ideas mainly from linguistics, evolution and psychology and explains a theory that seems to hold together and is well illustrated and explained. The central point is that we all are born with the ability to develop a universal grammar from an early age which can be adapted to any human language and which sticks around in the young child and then disappears. If you have any interest in language at all read this book.

A Spot of Bother - Mark Haddon

I found this a bit hard to get into as essentially it's about the normal lives of a family (albeit quite a special family). It doesn't really grab you. Towards the end though I was interested to find out how it would all unravel and was pleased with it.

The Mephisto Club - Tess Gerritsen

This is a nice, honest thriller that does what you expect. It keeps you flicking the pages wanting to know what happens next.

The God Delusion - Richard Dawkins

I quite enjoyed this and did find new arguments against religion but I don't think he's going to convert anyone with this book. Of course I'd recommend anyone read it because it raises lots of interesting points but it's polemic essentially.

The Blind Watchmaker - Richard Dawkins

I enjoyed this though it builds on previous work I'd read. I guess if you're just interested in evolution then read this and not the God Delusion.

The Raw Shark Texts - Steven Hall

I was recommended this by a friend. It's very surreal possibly a bit too surreal for me but I enjoyed it nonetheless.

Do Androids Dream of Electric Sheep? - Philip K Dick

I hadn't read the book and saw it at a friend's and borrowed it. You probably all know what it's like. I'm glad I read it because of the references to it but it's not my normal reading material.

Harry Potter and the Deathly Hallows - J K Rowling

I had to read this of course to finish off the series but I thought it was a lot better than some of the others. I think I enjoyed the first, the one with the tri-wizarding championship and this one the most.

Love in Idleness - Charlotte Mendelson

This is well written and you really get into the character that's painted for you. I really liked some of the descriptions of justifying things to yourself and coping with boredom.

Blood, Sweat & Tea - Tom Reynolds

This was an interesting look at the life of a paramedic and if you don't already read Random acts of reality then read the book first and start reading the blog.

Telling Lies - Paul Ekman

This came from my Blink/Tipping Point reading and I found it hard going. It was interesting but quite detailed and not really a book for late night reading. The theories in it are very interesting though and explain why you probably aren't as good at detecting things as you think you might be and how to look for factors that will help you.

The Man Who Mistook His Wife for a Hat - Oliver Sacks

I read this ages ago and it was fascinating. It's about a number of different cases of problems with the brain. Often physical defects in various areas of the brain that cause odd problems and how it sheds light on how things relate. I really enjoyed it.

As always, do please punt your own recommendations at me.

Since September 19th I have been mostly offline and unreachable. This was due to having started a new job in a wonderful new project: soitu.es, an online newspaper with a strong focus on Citizen Journalism: citizens playing an active role in the process of collecting, reporting, analyzing and disseminating news and information, I must mention the master mind behind this all: Gumersindo Lafuente, one very well known journalist in Spain. On the other hand, working with Ra l Rivero, not the Cuban poet but the technical soul of the project, is something I would pay for, and I get actually paid to learn from him! Too bad he doesn t digg Debian :)



The site was finally launched during Xmas, and we are all less busy, so despite having what I call the weekend flu (that only affects me during the weekends, but I never feel so bad that I need to miss work during the week), I am back to normal, If you thought I was quite MIA, keep in mind I was not one of busiest ones at work! Launching such a project takes a toll on everyone, though. I set up a Flick set for my new workplace: Amayita s Soitu.es Flickr photo Set, in case you are curious. There is also an official Soitu.es Flickr Photo Pool.

Today is a holiday in Spain, due to D a de Reyes, when kids usually get their Xmas presents here. As I am fully recovered from this weekend s flu (today is Monday after all), I had some spare time and my cats got a nice present too: A DIY scratching post. There is a WTF column in the effing middle of my kitchen, so after some cardboard, a wall stapler, sisal rope and a bit of round and round the column... This pic shows the result:



I couldn t resist filming a video, available on YouTube: This scratching post is now mine! - No! Mine! - I ll get you!

MythTV's commercial skipping really spoils us, and we practically never watch anything live (we don't watch a lot of TV at all, really, more DVDs, thanks to Netflix), so we're totally insulated from all the commercials except when we're staying in a hotel, or, as is the case right now, having a stay in hospital. It seems all of the commercials during the news programs are pharma-related. The majority are prescription drugs to "ask your doctor about", with the rest being health-related. Today's exciting product I learned about was Might Putty (you can watch the ad on their site). I just wish I had something to use it for. The other thing that piqued my curiosity was the Takara Patch. It sounds like a bit of a gimmick, but I've always liked gimmicks that look like they're doing something. The Biore Pore Strips for example. I'll have to do some more research.

I had been eyeing an inexpensive network-storage device -- for the non-geeks: think of a hard disk with an ethernet port and some controlling software -- for some time. I was aware of some of the hacking efforts around a few of these but somehow nothing really appealed. I was sort-of looking for nslu2-with-a-disk, and preferably not too expensive. Lo and behold, that's what I saw today in my techbargains feed: a Buffalo LinkStation Live which contains a 500gb SATA for $199 after rebates. Some quick googling lead to these wiki pages which looked promising: anything from enhancing the stock Linux setup by enabling a few more services to a custom Linux distro (similar to my wrt54 router running linux) to reportedly some work-in-progress for a native Debian installation. Nice! So off I went, ordered the thingie for local pickup for an additional 5% off and picked it up a little later at the local Circuit City (where visits are seemingly a recurring event these days). The documentation is very brief and insist that you install something on Windows -- just to find that the little box autoconfigures itself just fine. Presumably some network discovery is going to find the assigned dhcp address which is needed for the web interface. A few minutes later, a new (fixed) IP address was assigned, ntp was enabled and that was about that. After dinner, I quickly followed this tutorial to get the box a bit more Unixy without going too far (yet) from the default: start up telnet via a simple Java command-line tool, login, then enable ssh, set it up in /etc/init.d, add some extra binaries. All very quick and simple [ with the caveat that the addons.tar didn't want to get there via the Java tool, so manual scp once 'inside' did the trick ]. NFS, which I like for shuffling files around, appears to be little trickier for this ARM-based LinkStation Live. So at least for now I am content with simple rsync'ing of my backup directories on the few machines here. Much better than the current setup with mutual backups between workstations and semi-permanently being out of space. All in all a rather pleasant gadget and recommended at the price. The extra $100 in rebates are valid from today to the 12th.

There have been several long-term wishlist bugs on Mutt to include the (in?)famous "sidebar" patch. It adds a panel on the left side of the screen to list mailboxes with message counts. We do ship several patches with Debian's Mutt package that are not considered for inclusion by the upstream authors, but this one is different. It touches the core of the UI renderer and adds extensive counting of messages. There's been quite a hype about the patch, it was even included in the (now discontinued [1]) mutt-ng fork. Unfortunately, the patch was technically questionable, it was written for the Mutt tarballs and didn't easily apply to CVS, and most versions floating around contained lots of cruft like backup files and temporary editor files. Furthermore, it is far from bug-free and even segfaults occasionally. Some months ago we decided we would build a second binary package to put the patch in so we don't destabilize the standard package. Thanks Dann Frazier, we got a clean patch, Adeodato had the right idea on how to build two binaries from the same, differently-patched source (make install would run havoc, the trick was to build the patched version before the regular one), and I resolved some more involved conflicts with the maildir-mtime patch. So, finally, there it is: mutt-patched. Disclaimer: As said, there are bugs. YMMV. On related news, there's now also a -dbg package now for the victims of the IMAP and header cache segfaults that we still see sometimes. Update: The sidebar patch was not the reason for the creation of mutt-ng, but many mutt-ng users were using it because of this patch. Update 2008-01-08: [1] mutt-ng used to be a Mutt fork. It is now maintained as a patch collection for Mutt.

Chris, buildd hosts keep those time and space usage statistics in a database, which can be queried...

Script started on Sat Jan  5 19:54:15 2008
wouter@country:~$ ssh kiivi.cyber.ee
Linux kiivi 2.6.18-4-mac #1 Fri Mar 30 23:05:11 CEST 2007 m68k
No mail.
Last login: Sat Jan  5 20:41:16 2008 from d51532c45.access.telenet.be
wouter@kiivi:~$ avg-pkg-build-time -s -t   head -n 20
boost:			2015312k (2015312k lastest)
iceweasel:		1846628k (1846628k lastest)
iceape:			1674172k (1674416k lastest)
icedove:		1624848k (1624848k lastest)
xulrunner:		1611880k (1611880k lastest)
mesa:			1230432k (1230432k lastest)
koffice:		1208062k (2115156k lastest)
gcc-snapshot:		1191866k (1207164k lastest)
gcc-4.1:		1185620k (1185620k lastest)
gtk+2.0:		1148634k (1208684k lastest)
k3d:			1145580k (1145580k lastest)
ardour:			1077800k (1077800k lastest)
lyx:			924876k (924876k lastest)
gcc-4.2:		909488k (909488k lastest)
gcc-3.4:		892015k (571084k lastest)
glibc:			891408k (891408k lastest)
linux-2.6:		725961k (744700k lastest)
kdebase:		720618k (1106804k lastest)
kdelibs:		716376k (1150372k lastest)
kdepim:			711858k (784356k lastest)
wouter@kiivi:~$ avg-pkg-build-time -t   head -n 20
gcc-snapshot:		220:52:14 (2 entries, sigma 27:38:21)
axiom:			216:27:57 (1 entry, sigma 00:00:00)
gcc-4.1:		193:47:30 (1 entry, sigma 00:00:00)
k3d:			177:22:10 (1 entry, sigma 00:00:00)
gcc-3.4:		171:40:28 (4 entries, sigma 25:10:55)
boost:			141:13:38 (1 entry, sigma 00:00:00)
koffice:		140:40:23 (2 entries, sigma 53:48:58)
ace:			130:04:14 (1 entry, sigma 00:00:00)
linux-2.6:		129:36:19 (3 entries, sigma 10:12:24)
qt-x11-free:		105:00:32 (1 entry, sigma 00:00:00)
xulrunner:		97:03:38 (1 entry, sigma 00:00:00)
iceape:			96:23:13 (2 entries, sigma 03:12:58)
kdepim:			96:06:54 (2 entries, sigma 19:09:39)
icedove:		94:35:48 (1 entry, sigma 00:00:00)
iceweasel:		93:56:26 (1 entry, sigma 00:00:00)
gcc-4.2:		91:08:32 (1 entry, sigma 00:00:00)
glibc:			91:01:58 (1 entry, sigma 00:00:00)
kdevelop3:		84:27:21 (2 entries, sigma 02:47:35)
kdeedu:			82:09:55 (3 entries, sigma 10:59:18)
vtk:			81:44:20 (3 entries, sigma 00:46:32)
wouter@kiivi:~$ logout
Connection to kiivi.cyber.ee closed.
wouter@country:~$ ssh arrakis
Linux arrakis 2.6.12-1-amiga #1 Sat Aug 13 22:59:59 CEST 2005 m68k
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please read "links2 http://nefud/info.txt"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No mail.
Last login: Sat Jan  5 18:48:58 2008 from hahn.os.localnet on pts/5
wouter@arrakis:~$ avg-pkg-build-time -s -t   head -n 20
wxwidgets2.6:		1648020k (1648020k lastest)
qt4-x11:		1601086k (1753532k lastest)
gcj-4.1:		1135772k (1135772k lastest)
k3d:			1094345k (1159204k lastest)
kdebase:		1087375k (1087368k lastest)
python-kde3:		1018144k (1018144k lastest)
ghc6:			991524k (991524k lastest)
gcc-3.3:		988690k (990244k lastest)
glibc:			814536k (855720k lastest)
lyx:			804672k (804672k lastest)
bouml:			729888k (729888k lastest)
kdelibs:		685565k (1142304k lastest)
freedroidrpg:		673988k (673988k lastest)
mozilla:		663066k (636896k lastest)
ardour:			647580k (691128k lastest)
mozilla-thunderbird:	591384k (591384k lastest)
omniorb4:		576288k (576288k lastest)
subversion:		568764k (624768k lastest)
xorg-server:		527520k (527520k lastest)
mozilla-firebird:	498896k (502692k lastest)
wouter@arrakis:~$ avg-pkg-build-time -t   head -n 20
enlightenment:		198842:37:14 (3 entries, sigma 344403:17:35)
axiom:			164:43:54 (1 entry, sigma 00:00:00)
ghc6:			159:20:15 (1 entry, sigma 00:00:00)
k3d:			145:18:35 (3 entries, sigma 57:08:46)
python-kde3:		126:39:34 (1 entry, sigma 00:00:00)
qt4-x11:		121:26:11 (2 entries, sigma 03:07:35)
gcj-4.1:		116:05:34 (1 entry, sigma 00:00:00)
gcc-3.3:		108:45:19 (3 entries, sigma 02:01:50)
qt-x11-free:		99:12:17 (4 entries, sigma 14:12:32)
wxwindows2.4:		90:26:51 (1 entry, sigma 00:00:00)
openscenegraph:		74:37:21 (1 entry, sigma 00:00:00)
ardour:			63:44:55 (2 entries, sigma 07:23:05)
bouml:			62:01:16 (1 entry, sigma 00:00:00)
gcc-snapshot:		61:10:07 (1 entry, sigma 00:00:00)
mozilla-thunderbird:	60:47:29 (1 entry, sigma 00:00:00)
mico:			60:15:15 (1 entry, sigma 00:00:00)
kdebase:		58:41:04 (5 entries, sigma 08:00:17)
glibc:			58:04:26 (6 entries, sigma 40:01:12)
mozilla-firebird:	52:47:13 (2 entries, sigma 11:31:07)
zeroc-ice:		52:34:03 (2 entries, sigma 28:12:19)
wouter@arrakis:~$ logout
Connection to fremen-os.dyndns.org closed.
wouter@country:~$ exit
Script done on Sat Jan  5 22:28:46 2008

"avg-pkg-build-time" is a script that queries the statistics database. It's quite CPU-intensive, so I don't like to run it too often (certainly not from cron); but, hey, here you go. In case you didn't know, "arrakis" and "kiivi" are both official m68k/unstable buildd hosts, and especially arrakis has been for quite a long time, although it's been reinstalled once or twice since. Oh, and yes, it does appear as though some bits of the data are invalid, but most of the idea should be alright. (oh, and before you ask, no, it didn't take two and a half hours to run those commands -- I just needed to go while the last one was running, and had only come back several hours later)

Version 0.9.0 of QuantLib was released a couple of days ago on Christmas Eve; as usual, Debian packages for QuantLib were uploaded right away following a few earlier pre-releases. As QuantLib is approaching its 1.0 release, a few API changes requires updates to basically all of RQuantLib's C++ source files. Luckily most changes were minor. At the same time, we also generalised the Binary (aks Digital) option pricer to allow for a 'binType' argument (with values 'cash', 'asset' or 'gap' for CashOrNothing, AssetOrNothing or Gap digitals) as well as an 'excType' argument to switch between European and American exercise. Dominick made a small change to the DiscountCurve object to seamlessly pass a switch variable indicating whether we have 'flat' curves or not. Another change was the addition of formal unit tests using the RUnit from CRAN (which we happened to have added to Debian recently in the wave of new RMetrics packages). We use the scheme initially proposed by Gregor Gojanc and extended by Martin Maechler for RMetrics that allows the unit tests to be run from both the source and the installed package which is nice. As QuantLib itself has a massive amount of unit tests in its code; I am hoping to add more and more of those into RQuantLib itself as we add more functionality. On that front, more exciting news: RQuantLib is now hosted on R-Forge. Potential contributors are encouraged to register at R-Forge and to get in touch -- this is a great way to learn about combining C++ and R. To wrap up, the new version 0.2.8 is currently in the queue at R's master CRAN host and should hit the CRAN mirrors shortly; likewise the Debian package has been uploaded and should also propagate to Debian mirrors in due course. As usual, source are also available locally on my site.

I don't often blog about movies we watch, but if you have the opportunity to see that one by Florian Henckel von Donnersmarck, just go ("The lives of Others" in en and "La vie des autres" in fr). It's been ranked as "2007 movie" in many occasions and I'd vote for it as well even though, technically, I watched it only yesterday. Again, just go and watch it (if not done already, of course: you've maybe been less lazy than we were....or you're German..:-)).

I rebuilt all Debian packages twice. The first time with bash as /bin/sh, and the second time with dash as /bin/sh. About 120 packages built fine with bash, but failed to build with dash. I filed all the bugs that weren’t filed already. Then I debdiff’d the resulting binary packages, and found about 40 packages that built fine with bash and dash, but produced different binary packages (mostly files missing/being added)! All the bugs (including those that were already filed) have been usertagged. Most of those bugs are quite easy to reproduce and fix, so, if you are bored, send patches! I was a bit surprised by the high number of problems I found, since Ubuntu has been using dash as /bin/sh for more than a year, even on their buildds. Some packages have been fixed in Ubuntu, but the Debian maintainers didn’t include the fix. But many packages were simply broken in Ubuntu. Now I’ll try to find time to work on the Build Daemon From Hell idea, and on rebuilds inside qemu.

Anthony Towns wrote about user configuration (i.e. ~/.foo) and the XDG spec/proposal versus his own (and according to him) simpler version. The main goal of all this is to get rid of all the ~/.foo directories and files in a users home directory. Apart from technically minor differences, I don’t see why AJs proposal would be simpler than the XDG approach. All it does is hardcoding the values which are adjustable in XDG via XDG_*_DIRS. There is one notable difference though: AJ proposes fallback values which are compatible with existing application paths (i.e. HOME_ETC/XDG_CONFIG_HOME should default to ~/ - resulting in ~/.foo files/dirs). This is a good proposal in a way: If you patch an existing application to strictly follow the proposal made by AJ, it still finds its old configuration (and other files) if the new environment variables are not set. No need to move anything around. However, I think the XDG approach is better since it makes sure that compliant applications don’t clutter ~/ anymore. They do need some mechanism to move existing configurations and data around though if the XDG-compliant stuff isn’t already there, but an old style config exists. All in all, I think it is worth following the XDG spec, even if it is slightly more complex than AJs proposal. For one, it already exists for a while and I think that some applications already started following that spec. It also makes slightly more sense to me to have the system fully configurable as to where the apps store data as opposed to the hardcoded fallback/default values AJ proposes (even if we would change AJs proposal to use “$HOME-cleaning” default values).

Are there any active LUGs in Singapore? A friend in Singapore is about to set up his laptop to dual-boot and would like the possibility of getting some direct help if it ends up not booting Windows… I looked at the Linux.org page listing LUGs in Singapore [1]. There are two entries, one has a site that has not been updated for over a year, the other has no mention of meetings. Is visiting a nearby country the best option for someone in Singapore who wants to meet other Linux users? For anyone in a country without an active LUG, it would be good if you could create one. Even if it’s only a bunch of people having a scheduled meeting every month for food/drink. Once you have the meetings arranged other things can happen, random people who need help can bring in laptops to get advice. Also there was one occasion where I gave a lecture about SE Linux (which went for approximately an hour) in a back room at a bar, if you have an audience and a venue then sometimes guest speakers will just show up.

• [1] http://www.linux.org/groups/singapore.html

Share This

A few weeks back, someone asked on debian-68k about an unexpected result of some piece of software when it was compiled on ARAnyM, an m68k emulator that can run Linux. His initial idea was that this could perhaps be a bug in ARAnyM, since it only occurred to him inside ARAnyM, and not on any of the other architectures he tried. In fact, it was a bug in his code. Let's have a look at what's going on here. The example program which Sergei provided and which exhibits the problematic behaviour is spread over two files. The first file contains this:

#include <stdio.h>
void *f();
main()  
	void *a;
	a = f();
	printf("%d\n", (long)a);
 

while the second contains this:

#include <stdio.h>
long f();
long f()  
	long a = 84;
	printf("%d\n", a);
	return a;
 

Many people familiar with the C programming language will intuitively expect that this program, when run, will output the following:

84
84

When in fact, when compiled and ran on an m68k machine, the output is as follows:

84
-1072577264

What's going on here? In order to understand that, we have to take a look at the assembly code. This is done by use of "objdump -d", and gives (amongst others) the following output:

800004c4 <main>:
800004c4:	4e56 0000	linkw %fp,#0
800004c8:	61ff 0000 001a	bsrl 800004e4 <f>
800004ce:	2f08		movel %a0,%sp@-
800004d0:	4879 8000 05aa	pea 800005aa <_IO_stdin_used+0x4>
800004d6:	61ff ffff feec	bsrl 800003c4 <printf@plt>
800004dc:	508f		addql #8,%sp
800004de:	4e5e		unlk %fp
800004e0:	4e75		rts
800004e2:	4e75		rts
800004e4 <f>:
800004e4:	4e56 0000	linkw %fp,#0
800004e8:	4878 0054	pea 54 <_init-0x800002f0>
800004ec:	4879 8000 05aa	pea 800005aa <_IO_stdin_used+0x4>
800004f2:	61ff ffff fed0	bsrl 800003c4 <printf@plt>
800004f8:	7054		moveq #84,%d0
800004fa:	4e5e		unlk %fp
800004fc:	4e75		rts
800004fe:	4e75		rts

So what's happening here?

• main acquires some stack space, and then (0x800004c8) immediately jumps to a subroutine 26 bytes ahead; our function f.
• f also acquires stack space (0x800004e4), then pushes two addresses on the stack (...4e8 and 4ec), and jumps to the printf function. The first of these two addresses is the constant 84 that is hardcoded in the binary; the second address is our format string.
• f then moves the constant value 84 to the D0 register (...4f8), frees its stack space (4fa), and returns (4fc; the second rts can be ignored, that's a harmless quirk in the compiler).
• main now copies whatever is in the A0 register to the stack (4ce), pushes the exact same format string to the stack (4d0), and again jumps to the printf function (4d6; the difference in hexadecimal values is due to the fact that the bsr opcode uses processor-indirect addressing).
• Finally, main clears up what it's been doing, and exits

The difference should be clear: the f function stores its result value in the D0 register, but main goes looking for it in A0. The reason for this discrepancy very simple. The m68k processors has three sets of registers: one set is for integer values (D0 through D7), one set is for address values (A0 through A7), and one is for floating-point values (FP0 through FP7). The m68k ABI specifies that integer return values should be stored in integer registers, that address return values should be stored in address registers, and that floating-point return values should be stored in floating-point registers. This makes sense, since register-indirect addressing modes require the address to be stored in an address register; and calculating values requires the value to be stored in either a floating-point or integer register. When main, then, tried to look for a return value in A0, it found something, but obviously not what it should have found... This issue would have been a non-issue had this function been written the way it should have, like so:

#include <stdio.h>
void* f();
int main()  
	long* a;
	a=(long*)f();
	printf("%Ld\n", *a);
	return 0;
 
void* f()  
	static long a=84;
	printf("%Ld\n", a);
	return &a;
 

To close up: there's a reason why compilers emit warnings if you do something strange or "clever". In this particular case, the warning was suppressed, since both files contained a declaration of an f() function, even if the declaration was different. This is a horrible hack that tries to work around those warnings, rendering them utterly useless. Please, pretty please, with sugar on top: don't do something like that. If you write C code, please compile it with -Wall -Werror, and make sure it compiles that way. If you want to access a function in a different file, create a common header file that both will #include, so that the compiler can notice differences between declaration and definition of a function; and don't expect that something will work because you fixed it for a common and well-known case, because there will often still be other places where your bug will still trigger. In this case, by declaring the function as one returning a long value, they made sure it could not break on 64-bit architectures. However, as shown above, that doesn't make the bug go away...) Writing clean and portable code is way more fun. Trust me.

According to this test, sympycore is from 2.5x to 5x faster than Maxima. This is an absolutely fantastic result and also a perfect certificate for Python in scientific computing. Considering that we compare pure Python to LISP.

Ok, this made us excited, so we dugg deeper and ran more benchmarks. But first, let me say a few general remarks. I want a fast CAS (Computer Algebra System) in Python. General CAS, that people use, that is useful, that is easily extensible (!), that is not missing anything, that is comparable to Mathematica and Maple -- and most importantly -- I want it now and I don't care about 30 years horizons (I won't be able to do any serious programming in 30 years anyway). All right. How to do that? Well, many people tried... And failed. The only opensource CAS system, that has any chance of becoming the opensource CAS, in my own opinion, is Sage. You can read more about my impressions form Sage here. I am actually only interested in mathematical physics, so basically Sage.calculus. Currently Sage uses Maxima, because Maxima is old, proven, working system and it's reasonably fast and quite reliable, but written in LISP. Some people like LISP. I don't and I find it extremely difficult to extend Maxima. Also even though Maxima is in LISP, it uses it's own language for interacting with the user (well, that's not the way). I like python, so I want to use Python. Sage has written Python wrappers to Maxima, so Sage can do almost everything that Maxima can, plus many other things. Now. But the Sage.calculus has issues.

First, I don't know how to extend the wrappers with some new things, see my post in the sage-devel for details, it's almost 2 months old with no reaction, which shows that it's a difficult issue (or nonsense:)).

And second, it's slow. For some examples that Sage users have found out, even SymPy, as it is now, is 7x faster than Sage and sympycore 23x faster and with the recent speed improvements 40x faster than Sage.

So let's improve Sage.calculus. How? Well, no one knows for sure, but
I believe in my original idea of pure Python CAS (SymPy), possibly with some parts rewritten in C. Fortunately, quite a lot of us believe that this is the way.

What is this sympycore thing? In sympy, we wanted to have something now, instead of tomorrow, so we were adding a lot of features, not looking too much on speed. But then Pearu Peterson came and said, guys, we need speed too. So he rewrote the core (resulting in 10x to 100x speedup) and we moved to the new core. But first, the speed isn't sufficient, and second it destabilized SymPy a lot (there are still some problems with caching and assumptions half a year later). So with the next package of speed improvements, we decided to either port them to the current sympy, or wait until the new core stabilizes enough. So the new new core is called sympycore now, currently it only has the very basic arithmetics (and derivatives and simple integrals), but it's very fast. It's mainly done by Pearu. But for example the latest speed improvement using sexpressions was invented by Fredrik Johansson, another SymPy developer and the author of mpmath.

OK, let's go back to the benchmarks. First thing we realized is that Pearu was using CLISP 2.41 (2006-10-13) and compiled Maxima by hand in the above timings, but when I tried Maxima in Debian (which is compiled with GNU Common Lisp (GCL) GCL 2.6.8), I got different results, Maxima did beat sympycore.

SymPyCore:

In [5]: %time e=((x+y+z)**100).expand()
CPU times: user 0.57 s, sys: 0.00 s, total: 0.57 s
Wall time: 0.57

In [6]: %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.25 s, sys: 0.00 s, total: 0.25 s
Wall time: 0.25

Maxima:

(%i7) t0:elapsed_real_time ()$ expand ((x+y+z)^100)$ elapsed_real_time ()-t0;
(%o9)                                0.41
(%i16) t0:elapsed_real_time ()$ expand ((x + y+z)^20*(x+z)^19)$ elapsed_real_time ()-t0;
(%o18)                         0.080000000000005

So when expanding, Maxima is comparable to sympycore (0.41 vs 0.57), but for general arithmetics, Maxima is 3.5x faster. We also compared GiNaC (resp. swiginac):

>>> %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.03 s, sys: 0.00 s, total: 0.03 s
Wall time: 0.03

Then we compared just the (x+y+z)**200:

sympycore:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 1.80 s, sys: 0.06 s, total: 1.86 s
Wall time: 1.92
swiginac:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 0.52 s, sys: 0.02 s, total: 0.53 s
maxima:
(%i41) t0:elapsed_real_time ()$ expand ((x + y+z)^200)$ elapsed_real_time ()-t0;
(%o43)                         2.220000000000027

Where GiNaC still wins, but sympycore beats Maxima, but the timings really depend on the algorithm used, sympycore uses Millers algorithm which is the most efficient.

So then we tried a fair comparison: compare expanding x * y where x and y are expanded powers (to make more terms):

 sympycore:
 >>> from sympy import *
 >>> x,y,z=map(Symbol,'xyz')
 >>> xx=((x+y+z)**20).expand()
 >>> yy=((x+y+z)**21).expand()
 >>> %time e=(xx*yy).expand()
 CPU times: user 2.21 s, sys: 0.10 s, total: 2.32 s
 Wall time: 2.31
 swiginac:
 >>> xx=((x+y+z)**20).expand()
 >>> yy=((x+y+z)**21).expand()
 >>> %time e=(xx*yy).expand()
 CPU times: user 0.30 s, sys: 0.00 s, total: 0.30 s
 Wall time: 0.30
 maxima:
 (%i44) xx:expand((x+y+z)^20)$
 (%i45) yy:expand((x+y+z)^21)$
 (%i46) t0:elapsed_real_time ()$ expand (xx*yy)$ elapsed_real_time ()-t0;
 (%o48)                         0.57999999999993

So, sympycore is 7x slower than swiginac and 3x slower than maxima. We are still using pure Python, so that's very promising.

When using sexpr functions directly then 3*(a*x+..) is 4-5x faster than Maxima in Debian/Ubuntu. So, the headline of this post is justified. :)

Conclusion

Let's build the car. Sage has the most features and it is the most complete car. It has issues, some wheels need to be improved (Sage.calculus). Let's change them then. Maybe SymPy could be the new wheel, maybe not, we'll see. SymPy is quite a reasonable car for calculus (it has plotting, it has exports to latex, nice, simple but powerfull command line with ipython and all those bells and whistles and it can also be used as a regular python library). But it also has issues, one wheel should be improved. That's the sympycore project.

All those smaller and smaller wheels show, that this is indeed the way to go, but very important thing is to put them back in the car. I.e. sympycore back to sympy and sympy back to Sage and integrate them well. While also leaving them as separate modules, so that users, that only need one particular wheel, can use them.

happy newS year So this was Christmas, new year, et al. Too bad Murphy struck again on the first day of my (theoretical) vacation, delivering me a nice flu (not yet completely absorbed). Nevertheless I managed to do something funny in the last few days:

• I remembered that I do have, as all other DDs, a personal lintian page and I have thus spent some time fixing warning ignored since ever (almost all uploads are still in delayed queues however); while I was at it I also decided to "destroy" my lintian page moving my name to Uploaders for packages that still weren't co-maintained. (/me eager to see this implemented, go Russ, go!)
• I have fixed the vcs field stats page to consider as equal Vcs-Svn fields pointing to URLs like svn://svn.debian.org/foo and svn://svn.debian.org/svn/foo which are indeed the same final path on alioth. That's why in the graph you now see an inversion in the relative positions of field with wrong value and correct field, in case you were wondering. Thanks to Sandro Tosi for the hint.
• I have finally managed to leave the 18th century my mail set-up was living in: goodbye pop + fetchmail + mutt, welcome imap4 + offlineimap + mutt. In the former setup I was unable to read my old mails while away from my laptop, now I have decent replicas of my full mailboxes in various places. I'm still a bit unhappy about how can I force a send/receive email cycle: with fetchmail I used to have a demonized fetchmail running in the background and just executing fetchmail I was able to force that. Now, with offineimap, I've an autorefresh-ed offlineimap running inside screen and I need to bring up the screen and then hit 1 to trigger a sync cycle.

... now back to that paper introduction due for December 26th

Actually, they were out a few weeks ago, but I've been on vacation at the in-laws' near Monselice, Italy. The release announcement is here: http://www.tcl.tk/software/tcltk/8.5.tml It's worth writing about, because it's been a long time in the making, and contains a number of improvements. Particularly noteworthy in my opinion are 'real' hash tables (dicts), an even better sandbox model (Tcl and Java have the best sandboxes that I know of), and a theme engine for Tk that, in the right hands, significantly improves how it looks. In the fickle world of language fads, Tcl doesn't get the respect that it should these days. Like anything, it has problems, but what doesn't, when examined up close? It's ironic that for years it has gotten things right like Unicode support, and OS-level threading, that are currently being worked on in Ruby. Hecl obviously owes a great deal to Tcl, but since it's a new language (more or less necessitated by cell phone limitations), we're free to take a stab at improving things that Tcl doesn't get right. In any case, Tcl is worth a look if you want to play with something that's fundamentally different from a lot of popular languages, and the core team has put a lot of work into the latest release, so give it a whirl (perhaps using tkcon instead of the readline-less tclsh executable) with an open mind, and see if you don't learn something new.

Glen Turner writes about how Internet censorship could hurt science [1]. The ABC has an article about what is planned [2] which includes “Senator Conroy says it will be mandatory for all internet service providers to provide clean feeds, or ISP filtering, to houses and schools“. If Senator Conroy sticks to that plan then section two of Glen’s post (concerning high-speed access to research data) will not be a problem. This is no criticism of Glen for mentioning the issue as some idiot might try to change the plan to filter corporate and university access. Senator Conroy also earns a Godwin point for “If people equate freedom of speech with watching child pornography, then the Rudd-Labor Government is going to disagree“. There is a good discussion paper by Electronic Frontiers Australia about Labour’s plans in this regard before they won the election [3]. One issue they raise is the small number of sites identified as having child-porn. Imposing filters on an entire country to block the 3,236 web pages identified as prohibited by the Australian Communications and Media Authority is not a sensible solution. Another issue that EFA raises is the variety of reasons for content getting an R18+ rating (if that is to be a filtering criteria). One notable issue is “Adult themes” which includes “issues such as suicide, crime, corruption, marital problems, emotional trauma, drug and alcohol dependency, death and serious illness, racism, religious issues“. I doubt that there is much agreement between parents as to the relative significance of those “Adult themes” and which ones their children should be protected from. I also expect that a significant number of parents would like to have information about safe sex (including safe gay sex) and safe drug use available in case their children are interested in such things. I hope that the number of parents who disapprove of homosexuality and drug use so strongly that they are willing to risk their children’s lives is quite small (although the response to the cervical cancer vaccine indicates that it’s sadly larger than expected). Glen has some good points about this in the first section of his post.

• [1] http://www.gdt.id.au/~gdt/blog/strategy/2008-01-02.1024px


• [2] http://www.abc.net.au/news/stories/2007/12/31/2129471.htm


• [3] http://www.efa.org.au/Issues/Censor/mandatoryblocking.html

Share This

I’ve always been annoyed by the discussions about “is Ubuntu really giving back to Debian?”. Debian Developers usually don’t see a lot of “giving back”, and Ubuntu Developers complain about Debian Developers ignoring their bug reports and patches. So, a few months ago, I proposed that Ubuntu developers use a usertag when they report bugs to the Debian BTS, so they can be tracked. Results are available:

• Bugs with the origin-ubuntu usertag (bugs reported by Ubuntu Developers, with their Ubuntu hat on)
• Bugs with the ubuntu-patch usertag (as above, but a solution is provided by the Ubuntu developer)

Comments:

• It’s really good to see Ubuntu developers reporting bugs and contributing patches to Debian!
• … But more bugs (and patches) would be better, of course. Let’s continue the good work!
• Many patches are applied very fast in Debian (as usual), but in some cases, the patches are ignored (as usual, too). It would be great if Debian Developers could treat those bugs as higher priority, since it makes life easier on the Ubuntu side (less difference means less work)

If you are an Ubuntu Developer, read wiki.u.c/Bugs/Debian/Usertagging for the details on how to tag the bugs you file. Note that the submittodebian script in the ubuntu-dev-tools package already sets the usertags.