The Cherokee web server project is hosting the Cherokee Summit 2010 in Madrid, Spain on May 7th and 8th. The summit will be focused on high-performance and scalable Web technologies. I truly believe this is a major opportunity for hackers, developers and administrators looking for the new alternatives on the Web to bond and share experiences to benefit each other's projects and enterprises, and keep up to date on this field. If you are interested on attending, proceed to the registration site (it's free of charge but seat-limited) and see you there in May!

The DSPAM team have just announced their first official release, one year after the project was taken over by the Community. You can download the sources from the SourceForge project. Unofficial Debian packages are being built right now, but will only be uploaded to my repository after a bit more testing, during the week-end I guess.
They are meant to be uploaded to the official Debian archive once they are considered as ready by the pkg-dspam team. Hopefully, they will be part of Squeeze.

Mostly an update to my previous post on the subject, I ve just completed the FreeBSD port. It also builds and runs on GNU/kFreeBSD, by the way. The filescanner for forked-daapd on FreeBSD is now up to par with its Linux counterpart, as much as possible. Contrary to the latter, it will lose some metadata (play count, for instance) when files get moved around, because it s not possible to track moves and renames accurately. It also performs more rescans than the Linux/inotify implementation. Still, it works just as good. On a related note, the machine I tried to install FreeBSD 8.0 on and finally ended up installing GNU/kFreeBSD on just seemingly committed suicide. I m not sure what message it s trying to send me. It s previously been my main workstation for the best part of 10 years, so this is a bit sad. At least I found some bugs in the process and fixed some too:

• 2 bugs in forked-daapd, one of them a crasher,
• a bug in Avahi on kFreeBSD,
• a bug in GRUB2 as used on the kFreeBSD daily d-i images.

Not bad, is it?

I have been pondering the idea behind this article for a while, and finally had a bit of time to implement it. The basic idea is this: certain platforms have higher costs in terms of memory per concurrent connection. Those translate into increased costs in dollar terms.
Nota Bene: Having run LangPop.com for some time, I'm used to people getting hot and bothered about this or that aspect of statistics that are rough in nature, so I'm going to try and address those issues from the start, with more detail below.

• Constructive criticism is welcome. I expect to utilize it to revisit these results and improve them. Frothing at the mouth is not welcome.
• There is something of a "comparing apples and oranges" problem inherent in doing these sorts of comparisons. As an example, Rails gives you a huge amount of functionality "out of the box", whereas Mochiweb does much less. More on that below.
• I am not familiar with all of these systems: meaning that I may not have configured them as I should have. Helpful suggestions are, of course, welcome. Links to source code are provided below.
• You can likely handle many more 'users' than concurrent connections, which means multiple browsers connecting to the site at the same time.
• Programmer costs are probably higher than anything else, so more productive platforms can save a great deal of money, which more than makes up for the cost of extra memory. There's a reason that most people, outside of Google and Yahoo and sites like that, don't use much C for their web applications. Indeed, I use Rails myself, even though it uses a lot of memory and isn't terribly fast: I'd rather get sites out there, see how they do, and then worry about optimizing them (which is of course quite possible in Rails).

Methodology All tests were run like so: my new laptop with two cores and four gigs of memory was used as a server, and my older laptop was used to run the ab (Apache Benchmark) program - they're connected via ethernet. I built up to successive levels of concurrency, running first 1 concurrent connection, 2, 10, and so on and so forth. The "server" computer is running Ubuntu 9.10, "karmic". Platforms The platforms I tested:

• Apache 2.2, running the worker MPM, serving static files.
• Nginx 0.7.62, serving static files.
• Mochiweb from svn (revision 125), serving static files.
• Jetty 6.1.20-2, serving static files.
• Rails 2.3.5, serving up a simple template with the current date and time.
• PHP 5.2.10.dfsg.1-2ubuntu6.3, serving up a single php file that prints the current date and time.
• Django 1.1.1-1ubuntu1, serving up a template with the date and time.
• Mochiweb, serving a simple template (erltl) with the date and time.
• Jetty, serving a simple .war file containing a JSP file, with, as clever observers will have surmised, the date and time.

As stated above, it's pretty obvious that using Rails or Django for something so simple is overkill: Better Tests for the Future I would like to run similar tests with a more realistic application, but I simply don't have the time or expertise to sit down and write a blog, say, for all of the above platforms. If I can find a few volunteers, I'd be happy to discuss some rough ideas about what those tests ought to look like. Some ideas:

• They should test the application framework with a realistic, real world type of example.
• The data store should figure as little as possible - I want to concentrate on testing the application platform for the time being, rather than Postgres vs Sqlite vs Redis. Sqlite would probably be a good choice to utilize for the data store.
• Since this first test is so minimalistic, I think a second one ought to be fairly inclusive, making use of a fair amount of what the larger systems like Rails, Django and PHP offer.
• I'd also be interested in seeing other languages/platforms.
• The Holy Grail would be to script all these tests so that they're very easy to run repeatably.

Results With that out of the way, I do think the results are meaningful, and reflect something of what I've seen on various platforms in the real world. First of all, here we look at the total "VSZ" (as ps puts it) or Virtual Size of the process(es) in memory. Much of this might be shared, between libraries, and "copy on write" where applicable. The results are impressive: Rails, followed by Django and PHP eats up a lot of memory for each new concurrent connection. Rails, which I know fairly well, most likely suffers from several problems: 1) it includes a lot of code. That's actually a good thing if you're building a reasonably sized app that makes use of all it has to offer. 2) Its garbage collector doesn't play well with "copy on write". Which is what "Enterprise Ruby" aims to fix. Django and PHP are also fairly large, capable platforms when compared to something small and light like mochiweb. That said, excuses aside, Erlang and Mochiweb are very impressive in how little additional memory they utilize when additional concurrent connections are thrown at them. I was also impressed with Jetty. I don't have a lot of experience with Java on the web (I work more with J2ME for mobile phones), so I expected something a bit more "bloated", which is the reputation Java has. As we'll see below, Jetty does take up a lot of initial memory, but subsequent concurrent connections appear to not take up much. Of course, this is also likely another 'apples and oranges' comparison and it would be good to utilize a complete Java framework, rather than just a tiny web app with one JSP file. So what's this mean in real world terms of dollars and cents? As your Rails application gets more popular, you're going to have to invest relatively more money to make it scale, in terms of memory. For this comparison, I utilized the bytes/dollar that I'm getting for my Linode, which works out to 18,889,040.85 ($79.95 for 1440 MB a month). As we can see, to have a similar amount of concurrent users is essentially free for Mochiweb, whereas with Rails, it has a significant cost. This information is particularly relevant when deciding how to monetize a site: with Erlang and Jetty it would appear that scaling up to lots of users is relatively cheap, so even a small amount of revenue per user per month is going to be a profit, whereas with Rails, scaling up to huge numbers of users is going to be more expensive, so revenue streams such as advertising may not be as viable. It's worth noting that 37 signals, the company that created Rails, is a vocal supporter of charging money for products. There's another interesting statistic that I wanted to include as well. The previous graph shows the average cost per additional concurrent user, but this one shows how much the platform costs (using when there is just one user, so it acts as a sort of baseline: As we can see, Jetty is particularly expensive from this point of view. The default settings (on Ubuntu) seem to indicate that, for instance, the basic $20 a month Linode package would not be sufficient to run Jetty, plus a database, plus other software. I think that the Apache Worker number is off a bit, and may reflect settings made to handle a large number of connections, or perhaps a different MPM would make sense. Source Code / Spreadsheet The spreadsheet I put together is here: http://spreadsheets.google.com/ccc?key=0An76R90VwaRodElEYjVYQXpFRmtreGV3MEtsaWYzbXc&hl=en And the source code (admittedly not terribly well organized) is here: http://github.com/davidw/marginalmemory/

<comment />

Your Expectations
I won't fulfil them for you
they are only yours Feeling been let down
it is understandable
but it's also wrong There are more than me
who can do the piece of work
why just bug me then? Effect is stop work
so that others won't expect
your problem is solved

German Version If you are looking for some packages to take care of within Debian, take a look at this list and feel free to contact me if you are interested.

I've had alias net-pf-10 off and alias ipv6 off in my configuration for ages, but with recent kernels, IPv6 is apparently compiled in, so my eth0 now has a (link local) IPv6 address. ... which means that I now had Java apps suddenly fail to connect to the database. Since stuff like sqlplus (Bah!) works, I didn't immediately think about a network problem and started testing various JVM versions and hunting missing libraries etc. Only, it turned out that running java with -Djava.net.preferIPv4Stack=true is the magic bullet. Is there a proper fix? I know IPv6 is (supposed to be) the nice and shiny future, but currently I just don't have any IPv6 connection so it just creates problems for me right now. (Ok, let's add here that I dislike Java anyway because its philosophy, or the philosophy of most software I have to deal with, is to define its own world instead of integrating nicely with the host platform. If you look at it from another direction you may think I should have titled this I suck )

I am passing on a message from my daughter Shailaja. She would like you to go here and vote for computer engineer. (Or news anchor if you want to annoy her father.)

(a review of Homer s Iliad)

Here, therefore, huge and mighty warrior though you be, here shall you die. - Homer (The Iliad)

And with that formidable quote, I begin my review of The Iliad. I shall not exhaust you with a rehashing of the plot; that you can find on Wikipedia. Nor shall I be spending page upon page of analyzing the beautiful imagery, the implications of our understanding of fate and destiny, or all the other things that compel English majors to write page after page on the topic. Nor even shall I try to decipher whether it is a piece of history or a piece of legend. Rather, I intend to talk about why I read it: it s a really good story.

Sing to me, O goddess Muse, the wrath of Achilles, son of Peleus, which brought countless ills upon the Acheans.

As I d be reading it, I d think to myself: Ah ha! Now here we finally have a section of the story that doesn t speak to modern life. Perhaps it was a section the fear of being enslaved or butchered by conquerors, or about pride leading both sides to fight a war they need not have, or a graphic description of a spear going clear through someone s head or neck. And then, I d have to sit and think. Are we really so far removed from that? Here we are, in a supposedly civilized world. We use remotely-operated drones to drop bombs on people, and think it naught but regrettable collateral damage when the brains and intestines of dozens of innocent victims are scattered about, killing them, all for the chance of killing one enemy. But we aren t the only ones to blame; we live in a world in which killing the innocent is often the goal. People fly airplanes into skyscrapers, drop atomic weapons to flatten entire cities, and kill noncombatants in terrible numbers. And for what? A little power, some prestige, some riches, some vengeance, some wounded pride? Slavery is not dead, either. We that can happily afford Internet access most likely abhor the thought. What, though, can we make of the fact that people are starving in this world in record numbers? That our actions may literally wipe some nations off the map? Are those people as free as we are? Do our actions repress them? I suspect you can t read the Iliad without being at least a bit introspective.

Fear, O Achilles, the wrath of heaven; think on your own father and have compassion upon me, who am the more pitiable

One of the great things about reading the Iliad is that I can learn about the culture of the ancient civilizations. Now, of course, one can read about this in history books and Wikipedia. But reading some dry sentence is different from reading a powerful story written by and for them. I know that the Iliad isn t exactly a work of history, but it sure shows what made people tick: their religion, their morals, their behavior, and what they valued. I feel that I finally have some sort of insight into their society, and I am glad of it.

The day that robs a child of his parents severs him from his own kind; his head is bowed, his cheeks are wet with tears, and he will go about destitute among the friends of his father, plucking one by the cloak and another by the shirt. Some one or other of these may so far pity him as to hold the cup for a moment towards him and let him moisten his lips, but he must not drink enough to wet the roof of his mouth; then one whose parents are alive will drive him from the table with blows and angry words.

There are gory battle scenes in the Iliad, but then there are also heart-wrenching tender moments: when Hector leaves his wife to go fight, for instance, and worries about the future of his child if he should die. That s not to say I found the entire story riveting. I would have liked it to be 1/3 shorter. The battle just dragged on and on. And yet, I will grant that there was some purpose served by that: it fatigued me as a reader, which perhaps gave me a small sense of the fatigue felt by the participants in the story.

Why, pray, must the Argives needs fight the Trojans? What made the son of Atreus gather the host and bring them?

A question that was never answered, for either side: why are we so foolish that we must go to war? One tragedy among many is that neither side got smart about it until way too late, if ever they did at all. I read The Illiad in the Butler translation, which overall I liked. Some of these quotes, however, use the Lattimore one. This completes the first item on my 2010 reading list. I leave you with this powerful hope for the future, written almost three thousand years ago. Despite my criticisms above, we have made a lot of progress, haven t we? What a beautiful ideal it is, and what a long ways we have yet to walk.

I wish that strife would vanish away from among gods and mortals, and gall, which makes a man grow angry for all his great mind, that gall of anger that swarms like smoke inside of a man s heart and becomes a thing sweeter to him by far than the dripping of honey.

Sing to me, O goddess, the anger of Achilles, that one day his story may speak less to our hearts, for then will we have outgrown it.

Thanks to Joey Hess for the request to set up tor bridges. I wholeheartedly agree, and recommend the video he points to. I've installed a Tor bridge at home, and recommend others to do so. For the most part, I'm not politically active at the moment - I'm doing a PhD Part-time, and thats consuming all my "spare" time, other than work and family. Just given a blog site, however, one subject strikes me as very important - free speech. Hence the side links to Amnesty International, etc. In Ireland this campaign has a particular focus at the moment - the Repeal of the Blasphemy Law. This law came into effect in Ireland this January. While the government claims that this law will 'never be used', its bad in several ways. Firstly, it promotes the ideas of censorship as a method of hiding social issues, and secondly it may actually be used, as pointed out in : <object height="300" width="400"><param name="allowfullscreen" value="true"><param name="allowscriptaccess" value="always"><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=5196379&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1"><embed allowfullscreen="true" allowscriptaccess="always" height="300" src="http://vimeo.com/moogaloop.swf?clip_id=5196379&amp;server=vimeo.com&amp;show_title=1&amp;show_byline=1&amp;show_portrait=0&amp;color=&amp;fullscreen=1" type="application/x-shockwave-flash" width="400"></embed></object>Blasphemy is a Victimless Crime from Limerick Blogger on Vimeo. Basically, with Europe-wide arrest warrants, if two European countries have criminal blasphemy laws, then someone may be extradited to face prosecution in another country. While the Irish government says they will never prosecute, they open the possibility of say, Irish citizens being extradited to Greece to safe prosecution for blasphemy. On a wider level, I recommend Index on Censorship. It keeps documenting the cost of censorship around the world, and has been instrumental in campaigning for Libel reform in the UK, which is ncreasingly important for open scientific discussion. Tags censorhip, free-speech, politics

A clear announcement went out in plenty of time and the meeting agenda is already posted for Wednesday s SPI board IRC meeting which will be at 2100 UTC (an hour later than recently). As I write this, it s a pretty lean meeting, though I guess that s understandable when the run-up included holidays for most people. What do you think SPI s new year resolutions should be?

There is a drupal6 package in Debian, but for me as a Drupal User its support in Debian is somewhat unsatisfactory. Apparently it's not just me, but several other users are in the same situation. Drupal releases new version when a security issue was fixed in Drupal Core. Currently there are several security issues pending that are fixed in newer upstream version 6.15, which was releases on 2009-12-16. Bug #561726 deals with this as well as #562165.
It seems as if the maintainer is busy with other things, which is fairly ok, and if he isn't following Drupal security mailing list as we, as the users, are always need to forward and remember him about the security updates. Not just now, but also in the past. Remembering the maintainer about security updates is ok for me, alas it's somewhat tedious doing so. And beside the current situation, which made be ok giving Christmas vacation, Happy New Year and such, I'm quite satisfied with the work of the maintainer. But: could someone else please upload a new version of Debian in the meanwhile, please?! ;-)

UPDATE:
Luigi uploaded drupal 6.15 to unstable and I'm subscribed to pkg-drupal-devel on Alioth... ;)

I was recently driving through the California desert and came across the Salton Sea. Long story short - it rained a lot and the Colorado River overflowed a bunch of dams and dikes meant to contain it and created a huge inland sea. Oops. Some enterprising souls must have decided that despite the lack of any natural flushing dooming the sea to a salty, polluted existence, there was ripe opportunity to create a sea-side metropolis. From the ground, it is a bit of a fun ghost town to explore. The typical "everything just abandoned" type thing. But when I came to geotag some photos I took there, I was quite astonished to see this. That looks exactly like what I used to do in SimCity. I'd use the F-U-N-D-S cheat at the start to max out my money, then build my little empire with neat roads and school and harbours and whatnot they've even got an airport! Then I'd press "go" and people would slowly move in to the residential areas, one house on one block at a time. I guess poor old Salton City never made it past "turtle speed"!

Looking over some articles in the March 2009 issue of the Communications of the ACM magazine, I found a very good piece column (Is software engineering engineering?, Denning, Peter J., and Riehle Richard D. , Communications of the ACM, 03/2009, Volume 52, Number 3, p.24-26, (2009) ). Quoting from it:

The search for an alternative to the programmer image is already a generation old. In 1989 we asked: Are we mathematicians? Scientists? Engineers? We concluded that we are all three. We adopted the term "computing", an analogue to the European "informatics", to avoid bias towards any one label or description. (...) Predictable outcomes (principle of least surprise) Engineers believe that unexpected behaviors can be not only costly, but dangerous; consequently, they work hard to build systems whose behaviorthey can predict. In software engineering, we try to eliminate surprises by deriving rigurous specifications from well-researched requirements, then using tools from program verification and process management to assure that the specifications are met. The ACM Risks Forum documents a seemingly unending series of surprises from systems on which such attengion has been lavished. Writing in ACM SIGSOFT in 2005, Riehle suggested a cultural side of this: where researchers and artists have a high tolerance, if not love, for surprises, engineers do everything in their power to eliminate surprises. Many of our software developers have been raised in a research tradition, not an engineering tradition.

It would be an interesting excercise to find how people rate themselves in this regard in a large group of developers, and find the differences in their coding styles, or how varied (in developer profiles) are each of the sub-groups.

I have to admit, I was a bit lazy in my RC bug squashing efforts during the last two weeks. anyway, a short summary:

• #489351 hlbr: "hlbr: upgrade fails if not started"
  close, already fixed
• #526674 cwidget: "FTBFS with GCC 4.4: missing #include"
  add info to the bug report
• #548480 nxcomp: "build failure with gcc-4.4"
  apply patch from BTS, upload to DELAYED/2
• #560617 src:libio-socket-multicast6-perl: "libio-socket-multicast6-perl: FTBFS: tests failed"
  add build dependency on netbase and upload (pkg-perl)
• #562582 squidtaild: "fails to install, trying to overwrite other packages files"
  close, already fixed
• #564358 src:libhttp-body-perl: "libhttp-body-perl: FTBFS: tests failed"
  grab patch from upstream bugtracker, upload (pkg-perl)
• #564369 src:libnet-twitter-perl: "libnet-twitter-perl: FTBFS: tests failed"
  add patch and upload (pkg-perl)
• #564402 src:libhtml-formfu-perl: "libhtml-formfu-perl: FTBFS: tests failed"
  upload package fixed by ansgar (pkg-perl)
• #564411 src:libdatetime-format-iso8601-perl: "libdatetime-format-iso8601-perl: FTBFS: tests failed"
  add info to the bug report (pkg-perl)

I find myself opening a shell and typing "man ascii" more than I would like. More recently, I've found myself looking (with no success) for a equivalent web page that isn't ugly -- sometimes a browser tab is just more convenient. Yesterday, I decided to stop searching and just write one myself: man ascii. (I was a bit surprised that this domain, and all the variants of it Joker came up with, were available!) I think it gives me everything I got out of ascii(7), but I would love any bug reports.

Lenovo have announced their innovative new U1 Hybrid laptop [1]. It consists of a tablet-style device with a resistive touch-screen that runs Linux on a 1GHz ARM processor which attaches to a base computer that has a keyboard and a Core2 processor running Windows 7. They apparently have some special software to synchronise web browsing on both computers so you can maintain your web sessions when you detach the computers. How they manage to use the power of the Core2 CPU for javascript and flash intensive web sites while allowing the active browser sessions to migrate to a different OS must be a neat technical trick. Revision3 have a youtube review of the U1 which shows what it can do in terms of the hardware interface [2] Running two computers requires having two batteries, motherboards, etc which means more weight or means less battery life for the weight, whichever way you think of it the user is losing because of this choice. The idea of having two computers in one is one of these cool technical ideas which just won t provide a benefit for the users. The best thing for the users would be to have a single light-weight computer that provides an adequate amount of performance. A 1GHz ARM processor should give good performance when running Linux for most tasks (web browsing, office applications, and most games). So it seems that a good tablet with some USB ports and a matching USB keyboard would be a better option. Maybe you could have a spare battery integrated with the keyboard as most times when you want a full sized keyboard weight isn t such a big problem. The problem with MS-Windows in regard to such machines is that performance is poor and processors other than i386 and AMD64 have never really been supported (sure you can buy MS-Windows-based PDAs that have ARM and PPC CPUs, but the application support for them is minimal). These problems will never be solved. MS wants to compel users to continually upgrade their software, this requires always adding new features (bloat) while an older or less featureful OS is often a better option for lesser hardware. Vendors of proprietary applications will never support the range of CPU architectures that a free software distribution such as Debian or NetBSD supports (not that NetBSD is an ideal tablet OS but it does demonstrate what can be done if you want a portable OS). These inherent flaws in the proprietary software environment lead to some unusual hardware being designed to work around them such as the Lenovo U1. I don t have any reason to believe that the Lenovo is a bad machine, it sounds like a reasonable work-around for some of the problems that MS-Windows has forced on the industry. But I believe that it would be a much better machine if it was lighter, had a better battery life, and had a choice of keyboards all of which would have been achieved if it was designed as an ARM-only tablet machine that you can connect to an external keyboard. They could even have used a multi-core ARM CPU. But the market for such systems apparently isn t large enough for Lenovo (or anyone else) to ship ARM laptops for serious use. There have been a couple of netbooks released recently with CPUs that don t support the i386 or AMD64 instruction set, but they were aimed at web browsing use not general purpose computing. Almost everything that I do on computers could be done at least as well with a CPU that doesn t run an Intel-based instruction set, the only exception is virtualisation which doesn t seem to be well supported on architectures other than AMD64 nowadays.

• [1] http://lenovoblogs.com/insidethebox/?p=366
• [2] http://www.youtube.com/watch?v=-5DYuVN6nuY