Search Results: "ag"

4 January 2008

Wouter Verhelst: Broken C code

A few weeks back, someone asked on debian-68k about an unexpected result of some piece of software when it was compiled on ARAnyM, an m68k emulator that can run Linux. His initial idea was that this could perhaps be a bug in ARAnyM, since it only occurred to him inside ARAnyM, and not on any of the other architectures he tried. In fact, it was a bug in his code. Let's have a look at what's going on here. The example program which Sergei provided and which exhibits the problematic behaviour is spread over two files. The first file contains this:
#include <stdio.h>
void *f();
main()  
	void *a;
	a = f();
	printf("%d\n", (long)a);
 
while the second contains this:
#include <stdio.h>
long f();
long f()  
	long a = 84;
	printf("%d\n", a);
	return a;
 
Many people familiar with the C programming language will intuitively expect that this program, when run, will output the following:
84
84
When in fact, when compiled and ran on an m68k machine, the output is as follows:
84
-1072577264
What's going on here? In order to understand that, we have to take a look at the assembly code. This is done by use of "objdump -d", and gives (amongst others) the following output:
800004c4 <main>:
800004c4:	4e56 0000	linkw %fp,#0
800004c8:	61ff 0000 001a	bsrl 800004e4 <f>
800004ce:	2f08		movel %a0,%sp@-
800004d0:	4879 8000 05aa	pea 800005aa <_IO_stdin_used+0x4>
800004d6:	61ff ffff feec	bsrl 800003c4 <printf@plt>
800004dc:	508f		addql #8,%sp
800004de:	4e5e		unlk %fp
800004e0:	4e75		rts
800004e2:	4e75		rts
800004e4 <f>:
800004e4:	4e56 0000	linkw %fp,#0
800004e8:	4878 0054	pea 54 <_init-0x800002f0>
800004ec:	4879 8000 05aa	pea 800005aa <_IO_stdin_used+0x4>
800004f2:	61ff ffff fed0	bsrl 800003c4 <printf@plt>
800004f8:	7054		moveq #84,%d0
800004fa:	4e5e		unlk %fp
800004fc:	4e75		rts
800004fe:	4e75		rts
So what's happening here? The difference should be clear: the f function stores its result value in the D0 register, but main goes looking for it in A0. The reason for this discrepancy very simple. The m68k processors has three sets of registers: one set is for integer values (D0 through D7), one set is for address values (A0 through A7), and one is for floating-point values (FP0 through FP7). The m68k ABI specifies that integer return values should be stored in integer registers, that address return values should be stored in address registers, and that floating-point return values should be stored in floating-point registers. This makes sense, since register-indirect addressing modes require the address to be stored in an address register; and calculating values requires the value to be stored in either a floating-point or integer register. When main, then, tried to look for a return value in A0, it found something, but obviously not what it should have found... This issue would have been a non-issue had this function been written the way it should have, like so:
#include <stdio.h>
void* f();
int main()  
	long* a;
	a=(long*)f();
	printf("%Ld\n", *a);
	return 0;
 
void* f()  
	static long a=84;
	printf("%Ld\n", a);
	return &a;
 
To close up: there's a reason why compilers emit warnings if you do something strange or "clever". In this particular case, the warning was suppressed, since both files contained a declaration of an f() function, even if the declaration was different. This is a horrible hack that tries to work around those warnings, rendering them utterly useless. Please, pretty please, with sugar on top: don't do something like that. If you write C code, please compile it with -Wall -Werror, and make sure it compiles that way. If you want to access a function in a different file, create a common header file that both will #include, so that the compiler can notice differences between declaration and definition of a function; and don't expect that something will work because you fixed it for a common and well-known case, because there will often still be other places where your bug will still trigger. In this case, by declaring the function as one returning a long value, they made sure it could not break on 64-bit architectures. However, as shown above, that doesn't make the bug go away...) Writing clean and portable code is way more fun. Trust me.

Ross Burton: Social Whoring

Linked In always had the atmosphere of a more serious and professional social networking site, unlike MySpace and Facebook where people seem to collect friends like stickers when they were younger. Then I discovered TopLinked, a site dedicated to letting people grow their network massively to people they've never met. The top member has 37 thousand connections. I just don't understand this, what is the point of having so many connections when there is no value in the connections themselves? NP: Bricolage, Amon Tobin

Anthony Towns: User configuration

Inspired mostly by Joey’s nonchalant way of dealing with the death of his laptop…
This seems less of a disaster than other times a laptop’s disk has died on me. When did it start to become routine? […] My mr and etckeeper setup made it easy to check everything back out from revision control. […]
…I’ve been looking at getting all my stuff version controlled too. I’ve just gotten round to checking all my dotfiles into git, and it crossed my mind that it’d be nice if I could just set an environment variable to tell apps to create their random new dot-files directly in my “.etc-garbage” repo. I figured using “$USER_ETC/foo” instead of “$HOME/.foo” would be pretty easy, and might be a fun release goal that other Debian folks might be interested in, so I did a quick google to see if something similar had already been suggested. The first thing I stumbled upon was a mail from the PLD Linux folks who apparently were using $HOME_ETC at one time which sounded pretty good, though it doesn’t seem to have gotten anywhere. That thread included a pointer to the system that has gotten somewhere which is the XDG spec. It’s actually pretty good, if you don’t mind it being ugly as all hell. They define three classes of directory – configuration stuff, non-essential/cached data, and other data. That more or less matches the /etc, /var/cache and /var/lib directories for the system-wide equivalents, though if the “other data” is stuff that can be distributed by the OS vendor it might go in /usr/lib or /usr/share (or the /usr/local/ equivalents) too. Which is all well and good. Where it gets ugly is the naming. For the “/etc” configuration stuff, we have the environment variable $XDG_CONFIG_HOME, which defaults to ~/.config, and has a backup path defined by $XDG_CONFIG_DIRS, which defaults to /etc/xdg. For the “/var/lib” other data stuff, we have the environment variable $XDG_DATA_HOME, which defaults to ~/.local/share, and has a backup path defined by $XDG_DATA_DIRS, which defaults to /usr/local/share:/usr/share. (Though if you’re using gdm, it’ll get set for you to also include /usr/share/gdm) And for the “/var/cache” stuff, we have the environment variable $XDG_CACHE_HOME, which defaults to ~/.cache. That seems to me like exactly the right idea, with way too much crap on it. If you simplify it obsessively – using existing names, dropping the desktop-centrism, you end up with: Put configuration files in $HOME_ETC/foo or $HOME/.foo. For shared/fallback configuration, search $PATH_ETC if it’s set, or just /etc if it’s not. Put data files in $HOME_LIB/foo or $HOME/.foo. For shared data, search $PATH_LIB if it’s set, or look through /var/lib, /usr/local/ lib,share and /usr/ lib,share if it’s not. Put caches in $HOME_CACHE/foo or $HOME/.foo. For shared caches, search $PATH_CACHE if it’s set, or just look in /var/cache if it’s not. That seems much simpler to me to the point of being self-explanatory, and much more in keeping with traditional Unix style. It’s also backwards compatabile if you use both old and new versions of a program with the same home directory (or you happen to like dotfiles). And having the XDG variables set based on the above seems pretty easy too. I wonder what other people think – does HOME,PATH _ ETC,LIB,CACHE seem sensible, or is XDG_ CONFIG,DATA,CACHE _ HOME,DIRS already entrenched enough that it’s best just to accept what’s fated?

Torsten Werner: Open Source licensed scientific software



Software becomes more and more important in science as in other areas of life. Scientist have a tradition to publish their work very openly but that does often not include the source code of the software that was developed to carry out simulations which has some obvious problems such as:
- other scientists cannot check the software for errors,
- other scientists cannot fix the bugs and easily reproduce the results,
- other scientists cannot base their new research on already existing software and have to write it completely from scratch again and again,
- software package from different authors cannot be combined easily.

But things are getting better. One field of scientific research where we can see some improvement is machine learning - http://en.wikipedia.org/wiki/Machine_learning which is a broad subfield of artificial intelligence and concerned with the design and development of algorithms and techniques that allow computers to "learn". S ren Sonnenburg et.al. wrote a paper about "The Need for Open Source Software in Machine Learning" which is available at http://jmlr.csail.mit.edu/papers/v8/sonnenburg07a.html. They even created a portal with the goal to support a community creating a comprehensive open source machine learning environment at http://mloss.org.

An increasing number of software package are available in Debian like
- some simple-to-use utilities to apply compression techniques to the process of discovering and learning patterns: http://packages.debian.org/sid/complearn-gui
- a python package for convex optimization: http://packages.debian.org/sid/python-cvxopt
- a library for support vector machines: http://packages.debian.org/sid/libsvm2
- a machine-learning library: http://packages.debian.org/sid/libtorch3-dev
- an object-oriented programming language designed for researchers, experimenters, and engineers interested in large-scale numerical and graphic applications: http://packages.debian.org/sid/lush
- a large scale machine learning toolbox: http://packages.debian.org/sid/shogun-python-modular
- a data mining software in java: http://packages.debian.org/sid/weka

I'd like to know if you are using some of the packages or some other scientific software in Debian. Feel free to leave comment. Or maybe you are missing something in Debian?

If you are an author or user of some free software related to the topic of machine learning please consider registering it at http://mloss.org.

Masayuki Hatta: a2ps 4.14-1 uploaded

I've released new upstream release, but I forgot to upload the Debian package. It's kinda weird that I'm now the Debian maintainer AND the upstream author... Seems the gnuwin32 project already made A2Ps for Windows 4.14. Thanks! Also, please send me any patches if you have.

Tag this post with del.icio.us!

Ross Burton: Postr 0.10

A new year, a new release of Postr. This release has some useful bug fixes. Now to finish off that grand refactoring... The tarball is here, and packages for Debian are building now.

Masayuki Hatta: Japan's Big Push To Regulate The Internet

Once again Techdirt carries an excellent summary for the situation we Japanese are currently facing. Also I recommend the original article Techdirt refers.

Tag this post with del.icio.us!

Adrian von Bidder: Aggressive Java

Happy new year etc. Dear Lazyweb, how do I set up java so that it either uses arts or properly uses software mixing so that I can have sound in parallel between Java and other applications? (At least, I assume that Java is the culprit, because I can have parallel sound between flash within firefox, which I assume not to use arts, and arts based stuff. I could be wrong on this, though, so the question might be how to set up arts to not lock sound exclusively...) (Yes, the blog doesn't take comments. Drop me an email and I'll update this entry.)

Russell Coker: No more War On Terror in the UK

Military.com reports that the UK government will no longer use the term War On Terror [1]. Sir Ken Macdonald (the UK’s chief prosecutor) said that “terrorists” are criminals and need to be responded to in that way. This of course is the only logical and sensible thing to do. Soldiers who are taken prisoner are released when the war ends, if members of al Quaeda are considered to be soldiers then they would have to be treated in the same manner. The next logical step is to persue criminals who are members of al Quaeda in the same way that other criminals are persued. As far as I am aware there is no country where the majority of murderers are members of al Quaeda. Other suspected murderers have the right to a fair trial and people accused of al Quaeda membership deserve the same. Another interesting statement is that “The term “Islamic terrorist” will also no longer be used. Officials believe it is unhelpful because it appears to directly link the religion to terrorist atrocities“. Finally they realise that there is a huge number of Muslims who want nothing to do with terrorism and that such people are the best potential source of leads when it comes to tracking down criminals associated with al Quaeda. Thanks to Bruce Schneier for blogging about this [2]. Bruce’s blog post has some interesting comments, one is “you can’t make “War” on “Terrorism”. “Terrorism” is a tactic, not an enemy. To declare war on Terrorism is about as confused as declaring war on Blitzkrieg” by Carlo Graziani. Carlo also writes “It’s stupid to declare a “war” if you have no idea of when and how the war will end, and no clue about how to bring it to an end. If there is no real prospect for declaring “victory”, the “war” will go on for ever. This is tantamount to saying that insofar as we take the rhetoric of war seriously, we are agreeing to live under what is essentially martial law, in perpetuity. We are stipulating that the sort of emergency measures that a nation might consider taking in time of war — suspension of civil rights for certain suspect groups, suspension of laws limiting government surveillance powers, etc. — may be only a decree or a vote away, forever. There can be no more corrosive climate to liberty than war. If we really allow this idiotic rhetoric to be taken seriously, our polity is doomed.“. A particularly insightful comment from umacf24 is “How did the early 20th century Anarchists stop? Well, one of the attacks precipitated an unprecedentedly bloody and catastrophic war in which both sides used WMD. Military setbacks caused revolutions in the Russian and German empires which led in turn to most of misery of the rest of that century. Not a happy parallel“.

3 January 2008

Andrew Pollock: [life] Post-op, Day 0

I'm exhausted, but very relieved. The important news: the operation was a success, the surgeon, Dr D. Craig Miller, bless his cowboy boots, managed to spare the aortic valve (he'd given us a 99% probability of being able to do so beforehand), so we don't have to be concerned about pig/cow valves for now, and mechanical valves later. He was reluctant to say that her own valve will last her the rest of her life, but it should last a "long time, long enough to have a family". When I left the hospital, Sarah was in intensive care, extubated, conscious, and talking to me, but very very tired. She's got a daunting array of things coming out of her at the moment (a lot of which I couldn't see because she had blankets over her). It's been a long day, so I decided to head home, rather than potentially keeping her awake for the next two ICU visiting times tonight. Here's the time line that I recorded: 05:15 - Arrive at the hospital to go through the admissions process 06:35 - I leave Sarah shortly before they're going to wheel her off to the operating room (I'll get into my fun at the end). 09:30 - I ask for an update, and they tell me she's going on bypass 10:30 - It looks like they can save the aortic valve 12:25 - Still doing the valve stuff, making a new sinus of Valsalva 14:15 - I ask for another update, everything is going okay, someone will come out to talk to me 14:55 - Still going, at least another hour until they start to close up 15:15 - Closing up, surgeon will be out to talk to me 16:10 - Surgeon comes out to talk to me about how the operation went 16:50 - I got to see Sarah briefly in the ICU, she was still intubated, but conscious and seemed to respond to my voice ~18:15 - Extubated. Spoke with her briefly. She was very tired, and I decided to let her rest for the remaining visiting times and go home and get some rest myself So what the surgeon had to say for himself was very interesting indeed. He didn't end up doing exactly what he'd said previously, based on how things looked once he actually got in there. He said that the sinuses of Valsalva were paper thin. So thin, you could see the blood flowing through them. They were about 0.5mm thick, rather than the normal 1.5mm thick. Sounds a bit like a disaster waiting to happen. Apparently it's not possible to detect this with imaging, you have to get in there and see it for yourself. He didn't end up removing as much of the aorta as previously planned (specifically the part around the transcending arch) as whilst the diameter was a bit bigger, the tissue apparently looked healthy. He said the part of the aorta he did remove (preceding where the braciocephalic, left common carotid and left subclavian arteries branch off) was "cheesy" in consistency, so it's not like she just happened to have an enlarged aorta, no aneurysm, and way too thin sinuses of Valsalva. There was definitely some unhealthiness to the aortic tissue, just not as bad as thought from the imaging. So it's all good. I think she'll spend a day or two more in the ICU before getting moved to a normal ward, but it's too soon to say what's going on with all of that. There's some elevated risk of stroke due to blood clots for the next 24-48 hours, but she's on blood thinners, so it shouldn't be a problem. The day was pretty crazy for me. I'm not sure if it was stress, or the stomach bug that's been doing the rounds of Central Park Apartments, but I went to bed last night at 11pm, was up again at 1am, with shall we say, a gastrointestinal upset, and up again at 3am, throwing up. We got up at 4:15am to get to the hospital. I felt really nauseous all morning, and was an absolute wreck while I was in with Sarah when they were putting a peripheral line in her before they wheeled her off to the OR, so I bailed about 5 minutes early and just made it to the bathroom in time to throw up again. I haven't eaten all day, just tried to keep the fluids up. I'm debating having some plain boiled rice for dinner, or waiting until tomorrow to have something easy on the stomach. I definitely felt a lot better as the day progressed, but my stomach is still pretty tender. Sarah and I ate pretty much the same stuff yesterday, so I don't think it can be food poisoning. She tends to have a very sensitive stomach at the best of times, so I'd expect her to be showing problems before me if it was food poisoning. But enough about me, I'll live. I hope it's not a stomach bug, as I don't want Sarah to come down with it as well, now that she's on the road to recovery from this surgery. Many thanks to Shona, Briana, Laura, and Christina for spending time with me throughout the day to keep me occupied. ICU visiting hours are every even hour between 10am and 10pm, for thirty minutes at a time, so I plan on spending all day at the hospital tomorrow. Hopefully I can find a good spot to get some EVDO coverage, so I can update things as the day goes along, rather than at the end of the day when I get home, like I had to today. Hopefully I'll sleep better tonight than I did last night.

Steve Kemp: Its a lot like life

Assume for a moment that you have 148 hosts logging, via syslog-ng, to a central host. That host is recording all log entries into an MySQL database. Assume that each of these machines is producing a total of 4698816 lines per day. (Crazy random numbers pulled from thin air; globviously). Now the question: How do you process, read, or pay attention to those logs? Here is what we've done so far:
syslog-ng
All the syslog-ng client machines are logging to a central machine, which inserts the records into a database. This database may be queried using the php-syslog-ng script. Unfortunately this search is relatively slow, and also the user-interface is appallingly bad. Allowing only searches, not a view of most recent logs, auto-refreshing via AJAX etc.
rss feeds
To remedy the slowness, and poor usability of the PHP front-end to the database I wrote a quick hack which produces RSS feeds via queries, against that same database, accessed via URIs such as:
  • http://example.com/feeds/DriveReady
  • http://example.com/feeds/host/host1
The first query returns and RSS feed of log entries containing the given term. The second shows all recent entries from the machine host1. That works nicely for a fixed set of patterns, but the problem with this approach, and that of php-syslog-ng in general, is that it will only show you things that you look for - it won't volunteer trends, patterns, or news.
The fundamental problem is a lack of notion in either system of "recent messages worth reading" (on a global or per-machine basis). To put that into perspective given a logfile from one host containing, say, 3740 lines there are only approximately 814 unique lines if you ignore the date + timestamp. Reducing logentries by that amount (78% decrease) is a significant saving, but even so you wouldn't want to read 22% of our original 4698816 lines of logs as that is still over a million log-entries. I guess we could trim the results down further via a pipe through logcheck or similar, but I can't help thinking that still isn't going to give us enough interesting things to view. To reiterate I would like to see: To that end I've been working on something, but I'm not too sure yet if it will go anywhere... In brief you take the logfiles and tokenize, then you record the token frequencies as groups within a given host's prior records. Unique pairings == logs you want to see. (i.e. token frequency analysis on things like "<auth.info> yuling.example.com sshd[28427]: Did not receive identification string from 1.3.3.4" What do other people do? There must be a huge market for this? Even amongst people who don't have more than 20 machines!

Gustavo Franco: There will be no year of Linux on desktop

Ok, cool! Time to post again: Like these mega interesting random TOP 10 lists people prepare every December and January, some love to speculate if the current year will be or not the year of Linux on desktop.

Please permit me state the obvious and argue that there will be no year of Linux on desktop, like there was never a year for Microsoft Windows or one being prepared for Apple Mac OS X! It is a evolving process, as dumb as it sounds. In 20, 30 years for historic reasons, some will call the 90s the decade of Linux rise and I agree. I believe that some history books (books?) will not omit the Linux impact (including of course some other key open source software) on Internet availability for the masses.

Do people really believe that "the year of the linux" will be the one when it reach a bigger domestic user base than Windows? Great, what about Linux usage through Internet? It's Linux on desktop and on steroids including - get ready - portability and tons of buzzwords I can't even keep up with. Don't get me wrong, I care about Linux usage on the real hardware that end users are using at home and not only through the Internet but come on, let's stop running in circles.

MJ Ray: MSN Santa Sex Scandal

I missed this one when it happened earlier last month, but I'm still putting it on my 2007 page of Bad Technology. Thanks to a colleague for pointing it out. Apparently MSN's Santa-bot made inappropriate comments to children and was exposed in Microsoft's sex-obsessed RoboSanta spouts filth at children The Register After that, it was almost inevitable that it would be taken off-line, even if just for a short while, so I think the next headline could have been predicted:- Microsoft kills Santa Claus The Register What surprised me was the ham-fisted PR attempt described in Microsoft accuses kids of bullying Santa into sex chat The Register If it was a mistake, then admit it. How did it happen? Unruly programmers wanting easter eggs at Christmas, did someone make the Santa-bot by modifying a porno-bot, or was it a learning bot finally corrupted by all the abuse it was sent - like Arnold on UEA/tsw's #pubfood IRC? Truly bizarre.

Anthony Towns: tempus fugit

I blogged a fair bit about darcs some time ago, but since then I’ve not been able to get comfortable with the patch algebra’s approach to dealing with conflicting merges – I think mostly because it doesn’t provide a way for the user to instruct darcs on how to recover from a conflict and continue on. I’ve had a look at bzr since then, but it just feels slow, to the point where I tend to rsync things around instead of using it properly, and it just generally hasn’t felt comfortable. On the other hand, a whole bunch of other folks I respect have been a bit more decisive than I have on this, and from where I sit, there’s been a notable trend:
Keith Packard, Oct 2006
Repository formats matter, Tyrannical SCM selection
Ted Tso, Mar 2007
Git and hg
Joey Hess, Oct 2007
Git transitions, etckeeper, git archive as distro package format
Of course, Rusty swings the other way, as do the OpenSolaris guys. The OpenSolaris conclusions seem mostly out of date if you’re able to use git 1.5, and I haven’t learnt quilt to miss its mode of operation the way Rusty does. And as far as the basics go, Carl Worth did an interesting exercise in translating an introduction to Mercurial into the equivalent for git, so that looks okay for git too.

Ond&#345;ej &#268;ert&iacute;k: SymPy/sympycore (pure Python) up to 5x faster than Maxima (future of Sage.calculus?)

According to this test, sympycore is from 2.5x to 5x faster than Maxima. This is an absolutely fantastic result and also a perfect certificate for Python in scientific computing. Considering that we compare pure Python to LISP.

Ok, this made us excited, so we dugg deeper and ran more benchmarks. But first, let me say a few general remarks. I want a fast CAS (Computer Algebra System) in Python. General CAS, that people use, that is useful, that is easily extensible (!), that is not missing anything, that is comparable to Mathematica and Maple -- and most importantly -- I want it now and I don't care about 30 years horizons (I won't be able to do any serious programming in 30 years anyway). All right. How to do that? Well, many people tried... And failed. The only opensource CAS system, that has any chance of becoming the opensource CAS, in my own opinion, is Sage. You can read more about my impressions form Sage here. I am actually only interested in mathematical physics, so basically Sage.calculus. Currently Sage uses Maxima, because Maxima is old, proven, working system and it's reasonably fast and quite reliable, but written in LISP. Some people like LISP. I don't and I find it extremely difficult to extend Maxima. Also even though Maxima is in LISP, it uses it's own language for interacting with the user (well, that's not the way). I like python, so I want to use Python. Sage has written Python wrappers to Maxima, so Sage can do almost everything that Maxima can, plus many other things. Now. But the Sage.calculus has issues.

First, I don't know how to extend the wrappers with some new things, see my post in the sage-devel for details, it's almost 2 months old with no reaction, which shows that it's a difficult issue (or nonsense:)).

And second, it's slow. For some examples that Sage users have found out, even SymPy, as it is now, is 7x faster than Sage and sympycore 23x faster and with the recent speed improvements 40x faster than Sage.

So let's improve Sage.calculus. How? Well, no one knows for sure, but
I believe in my original idea of pure Python CAS (SymPy), possibly with some parts rewritten in C. Fortunately, quite a lot of us believe that this is the way.

What is this sympycore thing? In sympy, we wanted to have something now, instead of tomorrow, so we were adding a lot of features, not looking too much on speed. But then Pearu Peterson came and said, guys, we need speed too. So he rewrote the core (resulting in 10x to 100x speedup) and we moved to the new core. But first, the speed isn't sufficient, and second it destabilized SymPy a lot (there are still some problems with caching and assumptions half a year later). So with the next package of speed improvements, we decided to either port them to the current sympy, or wait until the new core stabilizes enough. So the new new core is called sympycore now, currently it only has the very basic arithmetics (and derivatives and simple integrals), but it's very fast. It's mainly done by Pearu. But for example the latest speed improvement using sexpressions was invented by Fredrik Johansson, another SymPy developer and the author of mpmath.

OK, let's go back to the benchmarks. First thing we realized is that Pearu was using CLISP 2.41 (2006-10-13) and compiled Maxima by hand in the above timings, but when I tried Maxima in Debian (which is compiled with GNU Common Lisp (GCL) GCL 2.6.8), I got different results, Maxima did beat sympycore.

SymPyCore:

In [5]: %time e=((x+y+z)**100).expand()
CPU times: user 0.57 s, sys: 0.00 s, total: 0.57 s
Wall time: 0.57

In [6]: %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.25 s, sys: 0.00 s, total: 0.25 s
Wall time: 0.25

Maxima:

(%i7) t0:elapsed_real_time ()$ expand ((x+y+z)^100)$ elapsed_real_time ()-t0;
(%o9) 0.41
(%i16) t0:elapsed_real_time ()$ expand ((x + y+z)^20*(x+z)^19)$ elapsed_real_time ()-t0;
(%o18) 0.080000000000005


So when expanding, Maxima is comparable to sympycore (0.41 vs 0.57), but for general arithmetics, Maxima is 3.5x faster. We also compared GiNaC (resp. swiginac):

>>> %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.03 s, sys: 0.00 s, total: 0.03 s
Wall time: 0.03


Then we compared just the (x+y+z)**200:

sympycore:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 1.80 s, sys: 0.06 s, total: 1.86 s
Wall time: 1.92
swiginac:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 0.52 s, sys: 0.02 s, total: 0.53 s
maxima:
(%i41) t0:elapsed_real_time ()$ expand ((x + y+z)^200)$ elapsed_real_time ()-t0;
(%o43) 2.220000000000027


Where GiNaC still wins, but sympycore beats Maxima, but the timings really depend on the algorithm used, sympycore uses Millers algorithm which is the most efficient.

So then we tried a fair comparison: compare expanding x * y where x and y are expanded powers (to make more terms):

sympycore:
>>> from sympy import *
>>> x,y,z=map(Symbol,'xyz')
>>> xx=((x+y+z)**20).expand()
>>> yy=((x+y+z)**21).expand()
>>> %time e=(xx*yy).expand()
CPU times: user 2.21 s, sys: 0.10 s, total: 2.32 s
Wall time: 2.31
swiginac:
>>> xx=((x+y+z)**20).expand()
>>> yy=((x+y+z)**21).expand()
>>> %time e=(xx*yy).expand()
CPU times: user 0.30 s, sys: 0.00 s, total: 0.30 s
Wall time: 0.30
maxima:
(%i44) xx:expand((x+y+z)^20)$
(%i45) yy:expand((x+y+z)^21)$
(%i46) t0:elapsed_real_time ()$ expand (xx*yy)$ elapsed_real_time ()-t0;
(%o48) 0.57999999999993

So, sympycore is 7x slower than swiginac and 3x slower than maxima. We are still using pure Python, so that's very promising.

When using sexpr functions directly then 3*(a*x+..) is 4-5x faster than Maxima in Debian/Ubuntu. So, the headline of this post is justified. :)

Conclusion

Let's build the car. Sage has the most features and it is the most complete car. It has issues, some wheels need to be improved (Sage.calculus). Let's change them then. Maybe SymPy could be the new wheel, maybe not, we'll see. SymPy is quite a reasonable car for calculus (it has plotting, it has exports to latex, nice, simple but powerfull command line with ipython and all those bells and whistles and it can also be used as a regular python library). But it also has issues, one wheel should be improved. That's the sympycore project.

All those smaller and smaller wheels show, that this is indeed the way to go, but very important thing is to put them back in the car. I.e. sympycore back to sympy and sympy back to Sage and integrate them well. While also leaving them as separate modules, so that users, that only need one particular wheel, can use them.

David Welton: No next big language?

Ola Bini, one of the JRuby hackers, and a very bright guy, posits that there "won't be a next big language": http://ola-bini.blogspot.com/2008/01/language-explorations.html
There might be some that are more popular than others, but the way development will happen will be much more divided into using different languages in the same project, where the different languages are suited for different things. This is the whole Polyglot idea.
I'm dubious, and wonder what he would consider to be the underlying sociological and economic factors driving this change. Programming languages are, in the end, about people in all their weirdness, so to understand where languages are going to go, you have to consider those human factors, as I've attempted to do here. One trend that points to a slow proliferation of languages is of course the lock-in cited in my article. Today's big languages (Java, and on the web, PHP) won't just go away from one day to the next, just as C, Cobol and Fortran have not disappeared with the advent of Java. That process will continue, making it likely that new languages will carve out new territory for themselves rather than exclusively cannibalizing existing installations from older languages. This naturally leads (slowly) to more languages, even if the next generation has a Next Big Language. And why shouldn't there be one? Ola talks about a situation where various languages run and interact on top of a runtime (JVM). Isn't that similar to what we've had with C, though? Perl, Tcl, Python, etc... all run on top of C. Sure, the JVM is a step up from that in some ways, most notably GC, being a bit more consistently cross-platform, and having a wider array of libraries, but in the end, it still comes down to the network effects of being able to read and write a common language, whatever it runs on. Obviously, the network externalities of programming languages are not so strong that they hit a tipping point after which one language crushes all the others, but they are strong enough to consolidate leadership in one or a few languages. Programmatically, Jython, JRuby, (and Hecl?:-) may even find it easier to interact on the same platform, but the humans writing the code will still push for consistency and the minimum set of common tools in order to aid the sharing and review of code. Another way to look at it might be from an organizational point of view. Today's biggest fish in the pond, Google, only allows four languages for their production systems. What would a "no big language future" organization look like? I can't believe they'd welcome a big hodge podge of things. In conclusion, as computers are ever more widely used, it's certain that more languages will be utilized. However, it's also likely that from time to time a few languages, with one in the lead, will emerge as the leaders.

Stefano Zacchiroli: happy newS year

happy newS year So this was Christmas, new year, et al. Too bad Murphy struck again on the first day of my (theoretical) vacation, delivering me a nice flu (not yet completely absorbed). Nevertheless I managed to do something funny in the last few days: ... now back to that paper introduction due for December 26th :-/

Marc 'Zugschlus' Haber: Late Happy Birthday, #405040

Sometimes, it is nearly as frustrating to use Debian than it is to use commercial software. For example, when one sees a simple bug completely unreacted on for more than one year. #405040 has passed its first anniversary since it was reported and touched for the last time. Visible reaction of the package maintainer: Nil. It’s a small thing, but an annoying one. And I still consider it unacceptable to let bugs rot for a year without the slightest trace of action.

Ian Wienand: 24 Technobabble

I was watching 24 the other day and (without giving anything away) Edgar Styles had to "patch a system call" to save the world. After chiding his assistant for giving him the wrong opcode for a "load A, conditional" the following briefly appears on the screen. The
system call that saved the world It's actually a pretty good piece of technobabble. The register names and use of 0x80 suggest whoever wrote this has at least some familiarlity with x86 assembley. It would have been slightly better if they called the "microshell" a "microkernel" and put the system call in eax before calling the interrupt, but if you belive Jack's amazing ability to get cell phone reception then it's only a small issue!

2 January 2008

Christian Perrier: "Cannot open input directory" at boot?

I get the following for quite a while when I boot:
Cannot open input directory: No such file or directory
This appear very early in the boot process (before loading the initrd image?) even when booting in "quiet" mode (which I usually do as I don't like the usual bunch of verbosity). I wonder what this might be.... No real harm done: everything works as expected, just annoying.

Next.

Previous.