Culture Machine has just come out with a new issue and it is all on pirates. Looks like a great collection.

In June of 2005, I finished reading "The Professor and the Madman" by Simon Winchester, which is an intriguing story of two significant contributors to the Oxford English Dictionary, or OED. I first heard this book mentioned in an interview on the Charlie Rose show, and it sounded interesting, so I bought a paperback copy, and it was indeed an interesting read. I wrote these thoughts down then, but neglected to post them until now. That was likely in part because others beat me to the punch. In particular, I think I remember listening to a presentation by Nat Friedman on the topic at around that time. In any case, I stumbled today over the text I wrote back then, and thought the words were still worth sharing. The most interesting thing I learned in reading this book wasn't actually about the principal characters, but instead involves a speech given by Richard Chenevix Trench to the Philological Society, entitled "On Some Deficiencies in Our English Dictionaries." This is the talk that eventually led to the creation of the OED, and the intriguing bit to me is an idea Trench proposed that was new at the time, but absolutely fundamental to the Open Source & Linux world today. Trench well understood that an undertaking of such magnitude as compilation of an exhaustive dictionary of the English language, detailing the history and lifetime of each word illustrated with many example of usage from existing literature, could never be accomplished by one individual. Instead, he proposed explicitly involving a large group of unpaid volunteer contributors. Winchester claims that while this may sound obvious today, that this idea had never been put forward before for such an undertaking. Absolutely brilliant. As an up-close observer of the evolution of the community approach to software development and maintenance, I couldn't help but draw connections. We are today engaged in an equally grand effort that may well have an enduring impact, that fundamentally would not be possible to accomplish without the contributions of an astoundingly diverse array of unpaid volunteer contributors...

Given the recent brouhaha in Debian, and General Resolution regarding Lenny s Release policy as it relates to Firmware and Debian s Social Contract, which has led to the resignation of Manoj Srivastava from the position of Secretary for the Debian Project, I m reminded of the following passage from Gordon Dickson s Tactics of Mistakes (part of Dickson s Childe Cycle, in which he tells the story of the rise of the Dorsai):

No, said Cletus. I m trying to explain to you why I d never make an Exotic. In your calmness in the face of possible torture and the need to kill yourself, you were showing a particular form of ruthlessness. It was ruthlessness toward yourself but that s only the back side of the coin. You Exotics are essentially ruthless toward all men, because you re philosophers, and by and large, philosophers are ruthless people. Cletus! Mondar shook his head. Do you realize what you re saying? Of course, said Cletus, quietly. And you realize it as well as I do. The immediate teaching of philosophers may be gentle, but the theory behind their teaching is without compunction and that s why so much bloodshed and misery has always attended the paths of their followers, who claim to live by those teachings. More blood s been spilled by the militant adherents of prophets of change than by any other group of people down through the history of man.

The conflict between idealism and pragmatism is a very old one in the Free and Open Source Software Movement. At one end of the spectrum stands Richard Stallman, who has never compromised on issues regarding his vision of Software Freedom. Standing at various distances from this idealistic pole are various members of the Open Source Community. For example, in the mid-1990 s, I used to give presentations about Linux using Microsoft Powerpoint. There were those in the audience that would give me grief about using a non-free program such as MS Powerpoint, but my response was that I saw no difference between driving a car which had non-free firmware and using a non-free slide presentation program. I would prefer to use free office suite, but at the time, nothing approached the usability of Powerpoint, and while dual-booting into Windows was a pain, I could do a better job using Powerpoint than other tools, and I refused to handcap myself just to salve the sensibilities of those who felt very strongly about Free Software and who viewed the use of all non-Free Software as an ultimate evil that must be stamped out at all costs. It is the notion of Free Software as a philosophy, with no compromises, which has been the source of many of the disputes inside Debian. Consider, if you will, the first clause of the Debian Social Contract:

Debian will remain 100% free We provide the guidelines that we use to determine if a work is free in the document entitled The Debian Free Software Guidelines. We promise that the Debian system and all its components will be free according to these guidelines. We will support people who create or use both free and non-free works on Debian. We will never make the system require the use of a non-free component.

This clause has in it no room for compromise. Note the use of words such as 100% free and never make the system require the use of a non-free component (emphasis mine). In addition, the Debian Social Contract tends to be interpreted by Computer Programmers, who view such imperatives as constraints that must never be violated, under any circumstances. Unfortunately, the real world is rarely so cut-and-dried. Even the most basic injunctions, such as Thou shalt not kill have exceptions. Few people might agree with claims made by the U.S. Republican Party that the war in Iraq qualified as a Just War as defined by Thomas Aquinas, but rather more people might agree that the July 20, 1944 plot to assassinate Hitler would be considered justifiable. And most people would probably agree most of the actions undertaken by the Allied Soldiers on World War II battlefields that involved killing other soldiers would be considered a valid exception to the moral (and for those in the Judeo-Christian tradition, biblical) injunction, Thou shalt not kill . As another example, consider the novel and musical Les Mis rables, by Victor Hugo. One of the key themes of this story is whether or not Thou shalt not steal is an absolute or not. Ultimately, the police inspector Javert, who lived his whole life asserting that law (untempered by mercy, or any other human considerations) was more important than all else, drowns himself in the Seine when he realizes that his life s fundamental organizing principle was at odds with what was ultimately the Right Thing To Do. So if even the sixth and eighth commandments admit to exceptions, why is it that some Debian developers approach the first clause of the Debian Social Contract with a take-no-prisoners, no-exceptions policy? Especially given the fourth clause of the Debian Social contract:

Our priorities are our users and free software We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments. We will not object to non-free works that are intended to be used on Debian systems, or attempt to charge a fee to people who create or use such works. We will allow others to create distributions containing both the Debian system and other works, without any fee from us. In furtherance of these goals, we will provide an integrated system of high-quality materials with no legal restrictions that would prevent such uses of the system.

This clause does not have the same sort of absolutist words as the first clause, so many Debian Developers have held that the needs of the users is defined by 100% free software . Others have not agreed with this interpretation but regardless of how needs of the users should be interpreted, the fact of the matter is, injuctions such as Thou shalt not kill are just as absolute and yet in the real world, we recognize that there are exceptions to such absolutes, apparently unyielding claims on our behavior. I personally believe that 100% free software is a wonderful aspirational goal, but in particular with regards to standards documents and firmware, there are other considerations that should be taken into account. People of good will may disagree about what those exceptions should be, but I think one thing that we should consider as even higher priority and with a greater claim on how we behave is the needs of our users and fellow developers as people. For those who claim Christianity as their religious tradition, Jesus once stated,

Thou shalt love the Lord thy God with all thy heart, and with all thy soul, and with all thy mind. This is the first and great commandment. And the second is like unto it: Thou shalt love thy neighbour as thyself. On these two commandments hang all the law and the prophets.

Even for those who do not claim Christianity as their religious tradition, most moral and ethical frameworks have some variant on the Golden Rule: Do unto others as you would have them do unto you . I would consider, for example, that the Golden Rule is at least a high priority claim on my behavior as the notion of free speech, and in many cases, it would be a higher priority claim. The recent controversy surrounding Josselin Mouette was started precisely because Joss has taken a something which is a good thing, namely Free Speech, and relegated it to a principle more important than all else, and claiming that any restraint on such a notion was equivalent to censorship. I think the same thing is true for free software, although it is a subtler trap. Philosophical claims than 100% free software as most important consideration is dangerously close to treating Free Software as the Object of Ultimate Concern or in religious terms, idolotry. For those who are religious, it s clear why this is a bad thing; for those who aren t if you are unwilling to worship a supernatural being, you may want to very carefully consider whether you are willing to take a philosophical construct and raise it to a position of commanding your highest allegiance to all else, including how you treat other people. Ultimately, I consider people to be more important than computers, hardware or software. So over time, while I may have had some disagreements with how Mark Shuttleworth has run Canonical Software and Ubuntu (but hey, he s the multimillionaire, and I m not), I have to give him props for Ubuntu s Code of Conduct. If Debian Developer took the some kind of Code of Conduct at least as seriously as the Social Contract, I think interactions between Debian Developers would be far more efficient, and in the end the project would be far more successful. This may, however, require lessening the importance of philosophical constructs such as Free Speech and Free Software, and perhaps becoming more pragmatic and more considerate towards one another.

Hey planet, it s been a very long time since I took my e-pen to write something down here, my work is taking pretty much all my time nowdays. Anyway, it seems that there is sort of a polemic rising here and there about RubyGems and its packaging philosophy. I d like to take that opportunity to contribute to the debate with the experience I had at Yoolink. Yes, we provide Ruby on Rails applications and are defacto dealing (or trying to) with Ruby and its gems. Wouter enlightened the main reason why it s such a mess to put into production a Rails app and to maintain it : the Ruby on Rails ecosystem is unstable. If you start writing your app, there are very big chances that when it gets mature for production mode, one or more of the Ruby libs you depend on will have a new version. And sadly, chances are there too that one of those libs will provide an API change. This can happen in a lot of places in a Rails app, because of its modular structure: plugins, gems or even Rails core upgrades (try to upgrade a real-working Rails app from 1.x to 2.x if you like tricky games). I agree with Joss when he said that Debian packages and Ruby gems are not designed for the same purpose, the latter are for a developer ecosystem and the first suitable for long-working production ecosystems. That s why setting up a complex Rails app in production is a nightmare : no Debian packages for all your Ruby needs on the first hand and API-unstable and hard-to-upgrade gems on the other hand. At Yoolink, the only working way we ve found to maintain in production our Rails apps is to freeze pretty much every Ruby-thing we can. Our production servers are then Rails/Ruby free (nothing installed system-wide) and the apps provides all their Ruby dependencies in their vendor/ directory. You can t do that for native gems but that works for 90% of a random Rails app s dependencies. To conclude, I d say that most of these issues come from the philosophy of the Ruby/Rails people : they don t fear to change anything they want from one release to another, just take a look at the gettext support in Rails 2.2 and you ll understand what I mean . It s like they don t even take in mind that people are now using their stuff, in real world, with customers relying on their apps. It s not a pet project anymore guys, if Rails want to remain the killer app it is, it will have to be someting stable and maintainable.

There is again some ongoing debate about Ruy gems and how to integrate them cleanly into the operating system. Let s put aside the fact such tools are useful on Windows and MacOS which don t have decent packaging systems. This is not a reason to dictate stupid requirements for other operating systems. Still, even without that, gems, eggs and the like are great tools for development; tools that developers can t work without today. They allow to easily setup a development environment with everything needed, at the latest versions. However they were not meant for integration and production environments, and will never be suitable for them. We re living in another world indeed: a world where servers must be up 24/7, where security issues flood your inbox every day, where you need dozens of servers running the very same software. This world is the Earth. Python eggs and Ruby gems were designed for the same target audience and with similar requirements; as such, they share the same deficiencies when it comes to other audiences:

1. They replace the operating system s packaging tool. This is very nice to not wait for the packagers when you want the latest development version, but you lose track of what is actually installed. In production, you need to know exactly what s installed and at what version. You also need to know what you will need to update in case of a security issue.
2. They install packages automatically on the system. This is really the biggest WTF; there are some reasons to do that to begin with, but they are all wrong.
3. They use a specific file format. This requirement comes mostly from the Windows world, and it really makes no sense on Unix. It makes it much harder to integrate complex applications using components in different languages.
4. They ship all files at the same place, happily violating the FHS and making it even harder to integrate with other things.

Python tools (setuptools and easy_install) have shifted their philosophy and brought a few small improvements which allow us to easily work around issues 1-3. These tools now include all you need to build a development environment in any directory that will not have an impact on the system packaging, making a clear distinction between the two targets. Furthermore, the metadata is shipped separately and can safely be included in a distribution package. Similarly, modules will not be installed automatically on your system. I have yet to see such a move from Ruby developers. However, what we are still missing is something along the lines of dh_make_perl: starting from a CPAN module, it will generate a clean and working Debian package in a few minutes. There have been some similar attempts for Python, but nothing as efficient. And I am certainly guilty of not having done something in this area, as one of the most knowledgeable of issues we encounter while packaging Python modules. As a note, I should add the underlying issue behind all of this is still the same: the NIH syndrome. Developers are reinventing the wheel, engine and transmission. Which is not that bad per se, but by not looking at existing solutions for the problem of making a car move, they are inventing a square wheel, a steam-powered engine and a superconductor-powered magnetic transmission. For example, while trying to fix the 4th of items listed earlier and they are really willing to fix it and several more general issues, setuptools developers forgot to look at the existing solutions:

• API versioning in /usr/include/package-X.Y
• ABI versioning of libraries
• Build/install-time requirements gathering using an extensible format with pkg-config
• Automated generation of config.h containing all system-specific macros (including paths), with autoconf
• The Mono Global assembly cache which solves some of the specific issues of interpreted languages

Certainly, the existing tools are everything but perfect, and not necessarily suitable for Ruby and Python, but some of the design ideas behind them are clearly the good solutions to some of the issues of software development and integration. Adapting these ideas to radically different languages is going to take more time than just throwing other layers of symlink farms.

Thanks to some unexplained comments on some oldish entries on my blog, I found -with a couple of days of delay- Rubigem is from Mars, Apt-get is from Venus, in Pelle's weblog. And no, I have not yet read the huge amount of comments generated from it... Still, I replied with the following text - And I am leaving this blog post in place to remind me to further extend my opinions later on.
Wow... Quite a bit of comments. And yes, given that the author wrote a (very well phrased and balanced) post, I feel obliged to reply. But given that he refered to me first, I'll just skip the chatter for later - I'm tired this time of day ;-)
Pelle, I agree with you - This problem is because we are from two very different mindsets. I have already said so - http://www.gwolf.org/soft/debian+rails is a witness to that point.
But I do not think the divide is between sysadmins and developers. I am a developer that grew from the sysadmin stance, but that's not AFAICT that much the fact in Debian.
Thing is, in a distribution, we try to cater for common users. I have a couple of Rails apps under development that I expect to be able to package for Debian, and I think can be very useful for the general public.
Now, how is the user experience when you install a desktop application, in whatever language/framework it is written? You don't care what the platform is - you care that it integrates nicely with your environment. Yes, the webapp arena is a bit more difficult - but we have achieved quite a bit of advance in that way. Feel like using a PHP webapp? Just install it, and it's there. A Python webapp? Same thing. A Perl webapp? As long as you don't do some black magic (and that's one of the main factors that motivated me away from mod_perl), the same: Just ask apt-get to install it and you are set.
But... What about installing a Rails application? From a package manager? For a user who does not really care about what design philosophy you followed, who might not even know what a MVC pattern is?
Thing is, distributions aim at _users_. And yes, I have gradually adopted a user's point of view. I very seldom install anything not available as a .deb - and if I do, I try to keep it clean enough so I can package it for my personal use later on.
Anyway... I will post a copy of this message in my blog (http://gwolf.org/), partly as a reminder to come back here and read the rest of the buzz. And to go to the other post referenced here. And, of course, I invite other people involved in Ruby and Debian to continue sharing this - I am sure I am not the only person (or, in more fairness, that Debian's pkg-ruby-extras team is not the only team) interested in bridging this huge divide and get to a point we can interact better - And I am sure that among the Rubyists many people will also value having their code usable by non-developers as well.

Until a couple of weeks ago, I was under the impression that one could only have a single HTTPS site per IP address, but it seems I was wrong. The procedure is described here on the CACert wiki. In short, you need multiple SubjectAltName fields on your server's certificate, such that all the VirtualHost names that you want to work are either directly mentioned, or will match via wildcards. The page above includes a link to a nice script that generates a key and CSR (Certificate Signing Request) ready to be pasted into CAcert's Server Certificate page. Note: the CN is pretty much ignored by some browsers, so you'll want to put the machine's main name as one of the SubjectAltNames as well. BTW if you get a warning like:

  [warn] _default_ VirtualHost overlap on port 443, the first has precedence

you probably need to add

  NameVirtualHost *:443

to your /etc/apache2/sites-enabled/ssl before the <VirtualHost *:443> line. Of course, this glosses over the details of doing things like setting your name and address in the certificate, but since CAcert will strip all that out anyway, it only matters if you wanted to get it signed by someone else. Even so, this should get you started -- you can always edit those details into the csr script.

Next time I have to explain why I (really) prefer dogs to cats, I will not provide an argument. Instead, I will point people to this video. While the video is not an instance of joking, it is nonetheless a potent example of humor, at least in the way that the philosopher, Simon Critchley has defined it in his compact and very powerful (and very beautifully written) book, On Humour:

Jokes tear holes in our usual predictions about the empirical world. We might say that humour is produced by a disjunction between the way things are and they way they are represented in the joke, between expectation and actuality. Humour defeats our expectations by producing a novel actuality, by changing the situation in which we find ourselves…

I love it when a lack-of-humor and lack-of-appropriateness-originated flamewar causes somebody to point me towards a very nice display of intelligent humor. Specially when it is so close to me, to my roots, to my family and my personal history. FWIW, for several years, while I was a BBS user, I used WereWolf as my nickname. Great thanks to Frank K ster - and, of course, to Christian Morgenstern.

The Werewolf - English translation by Alexander Gross A Werewolf, troubled by his name,
Left wife and brood one night and came
To a hidden graveyard to enlist
The aid of a long-dead philologist. "Oh sage, wake up, please don't berate me,"
He howled sadly, "Just conjugate me."
The seer arose a bit unsteady
Yawned twice, wheezed once, and then was ready. "Well, 'Werewolf' is your plural past,
While 'Waswolf' is singularly cast:
There's 'Amwolf' too, the present tense,
And 'Iswolf,' 'Arewolf' in this same sense." "I know that--I'm no mental cripple--
The future form and participle
Are what I crave," the beast replied.
The scholar paused--again he tried: "A 'Will-be-wolf?' It's just too long:
'Shall-be-wolf?' 'Has-been-wolf?' Utterly wrong!
Such words are wounds beyond all suture--
I'm sorry, but you have no future." The Werewolf knew better--his sons still slept
At home, and homewards now he crept,
Happy, humble, without apology
For such folly of philology.

today in my website's access.log (wrapped for easier legibility):

64.1.215.166 - - [21/Nov/2008:19:03:02 +0100] "GET / ,blog/  HTTP/1.0" \
  404 6963 "-" "Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)"

the broken URL comes from my ~/.signature which since about 2 months contains the following fragment:

http://info.comodo.priv.at/ ,blog/ 

& probably that line was found in some web archive of a mailing list or newsgroup or somesuch.

now what is this highly intelligent Twiceler thingy? a look at the URL in the access.log entry redirects to cuil, & the page proudly claims in friendly, large, green letters

Cuil is the biggest search engine on the planet.

apart from the fact that I've never heard about them & that their search results are not impressive either, their "philosophy" (huh?) page claims:

Size matters

maybe it's just me but probably they should invest more in technical "cuil" [0] than in marketing speak that sounds less like search engines but more like another business branch on the intarwebs.

[0] "Cuil is an old Irish word for knowledge." (source: cuil again)

Currently there is a debate about censoring the Internet in Australia. Although debate might not be the correct word for a dispute where one party provides no facts and refuses to talk to any experts (Senator Conroy persistently refuses all requests to talk to anyone who knows anything about the technology or to have his office address any such questions). The failures of the technology are obvious to anyone who has worked with computers, here is an article in the Sydney Morning Herald about it [1] (one of many similar articles in the MSM). I don’t plan to mention the technological failures again because I believe that the only people who read my blog and don’t understand the technology are a small number of my relatives - I gave up on teaching my parents about IP protocols a long time ago. One of the fundamental problems with the current censorship idea is that they don’t seem to have decided what they want to filter and who they want to filter it from. The actions taken to stop pedophiles from exchanging files are quite different from what would be taken to stop children accidentally accessing porn on the net. I get the impression that they just want censorship and will say whatever they think will impress people. I have previously written about the safety issues related to mobile phones [2]. In that document I raised the issue of teenagers making their own porn (including videos of sexual assault). About four months after writing it a DVD movie was produced showing a gang of teenagers sexually assaulting a girl (they sold copies at their school). It seems that the incidence of teenagers making porn using mobile phones is only going to increase, while no-one has any plans to address the problem. The blog www.somebodythinkofthechildren.com has some interesting information on this issue. Two final reasons for opposing net censorship have been provided by the Sydney Anglicans [3]. They are:

1. Given anti-vilification laws, could religious content be deemed illegal and be filtered out? Could Sydneyanglicans.net be blocked as illegal if it carries material deemed at some point now or in the future as vilifying other religions? If it s illegal in Vic say, and there isn t state-based filtering (there wont be), will the govt be inclined to ban it nation wide?


2. Given anti-discrimination laws, if Sydneyanglicans.net runs an article with the orthodox line on homosexuality, will that be deemed illegal, and the site blocked? You can imagine it wouldn t be too hard for someone to lobby Labor via the Greens, for instance.

So the Sydney Anglicans seem afraid that their religious rights to discriminate against others (seriously - religious organisations do have such rights) will be under threat if filtering is imposed. I was a bit surprised when I saw this article, the Anglican church in Melbourne seems reasonably liberal and I had expected the Anglican church in the rest of Australia to be similar. But according to this article Peter Jensen (Sydney’s Anglican Archbishop) regards himself as one of the “true keepers of the authority of the Bible” [4]. It seems that the Anglican church is splitting over the issues related to the treatment of homosexuals and women (Peter believes that women should not be appointed to leadership positions in the church to avoid “disenfranchising” men who can’t accept them [5]). It will be interesting to see the fundamentalist Christians who want to protect their current legal rights to vilify other religions and discriminate against people on the basis of gender and sexual preference fighting the other fundamentalist Christians who want to prevent anyone from seeing porn. But not as interesting as it will be if the Anglican church finally splits and then has a fight over who owns the cathedrals. ;) A comment on my previous post about the national cost of slow net access suggests that Germany (where my blog is now hosted) has better protections for individual freedom than most countries [6]. If you want unrestricted net access then it is worth considering the options for running a VPN to another country (I have previously written a brief description of how to set up a basic OpenVPN link [7]).

• [1] http://www.smh.com.au/articles/2008/11/11/1226318639085.html?page=fullpage


• [2] http://doc.coker.com.au/safety/mobile-phone-safety/


• [3] http://www.sydneyanglicans.net/forums/viewthread/3673/


• [4] http://www.smh.com.au/articles/2008/06/22/1214073053814.html


• [5] http://www.abc.net.au/worldtoday/content/2004/s1213397.htm


• [6] http://etbe.coker.com.au/2008/11/17/national-cost-slow-net-access/#comment-16729


• [7] http://etbe.coker.com.au/2008/05/24/ipsec-is-pain/

Australia has slow Internet access when compared to other first-world countries. The costs of hosting servers are larger and the cost of residential access is greater with smaller limits. I read news reports with people in other countries complaining about having their home net connection restricted after they transfer 300G in one month, I have two net connections at the moment and the big (expensive) one allows me 25G of downloads per month. I use Internode, here are their current prices [1] (which are typical for Australia - they weren’t the cheapest last time I compared but they offer a good service and I am quite happy with them). Most people in Australia don’t want to pay $70 per month for net access, I believe that the plans which have limits of 10G of download or less are considerably more popular. Last time I investigated hosting servers in Australia I found that it would be totally impractical. The prices offered for limits such as 10G per month (for a server!) were comparable to prices offered by Linode [2] (and other ISPs in the US) for hundreds of gigs of transfer per month. I have recently configured a DomU at Linode for a client, Linode conveniently offers a choice of server rooms around the US so I chose a server room that was in the same region as my client’s other servers - giving 7 hops according to traceroute and a ping time as low as 2.5ms! Currently I am hosting www.coker.com.au and my blog in Germany thanks to the generosity of a German friend. An amount of bandwidth that would be rather expensive for hosting in Australia is by German standards unused capacity in a standard hosting plan. So I get to host my blog in Germany with higher speeds than my previous Australian hosting (which was bottlenecked due to overuse of it’s capacity) and no bandwidth quotas that I am likely to hit in the near future. This also allows me to do new and bigger things, for example one of my future plans is to assemble a collection of Xen images of SE Linux installations - that will be a set of archives that are about 100MB in size. Even when using bittorrent transferring 100MB files from a server in Australia becomes unusable. Most Australians who access my blog and have reasonably fast net connections (cable or ADSL2+) will notice a performance improvement. Australians who use modems might notice a performance drop due to longer latencies of connections to Germany (an increase of about 350ms in ping times). But if I could have had a fast cheap server in Australia then all Australians would have benefited. People who access my blog and my web site from Europe (and to a slightly lesser extent from the US) should notice a massive performance increase, particularly when I start hosting big files. It seems to me that the disadvantages of hosting in Australia due to bandwidth costs are hurting the country in many ways. For example I run servers in the US (both physical and Xen DomUs) for clients. My clients pay the US companies for managing the servers, these companies employ skilled staff in the US (who pay US income tax). It seems that the career opportunities for system administrators in the US and Europe are better than for Australia - which is why so many Australians choose to work in the US and Europe. Not only does this cost the country the tax money that they might pay if employed here, but it also costs the training of other people. It is impossible to estimate the cost of having some of the most skilled and dedicated people (the ones who desire the career opportunities that they can’t get at home) working in another country, contributing to users’ groups and professional societies, and sharing their skills with citizens of the country where they work. Companies based in Europe and the US have an advantage in that they can pay for hosting in their own currency and not be subject to currency variations. People who run Australian based companies that rent servers in the US get anxious whenever the US dollar goes up in value. To quickly investigate the hosting options chosen for various blogs I used the command “traceroute -T -p80” to do SYN traces to port 80 for some of the blogs syndicated on Planet Linux Australia [3]. Of the blogs I checked there were 13 hosted in Australia, 11 hosted independently in the US, and 5 hosted with major US based blog hosting services (Wordpress.com, Blogspot, and LiveJournal). While this is a small fraction of the blogs syndicated on that Planet, and blog hosting is also a small fraction of the overall Internet traffic, I think it does give an indication of what choices people are making in terms of hosting. Currently the Australian government is planning to censor the Internet with the aim of stopping child porn. Their general plan is to spend huge amounts of money filtering HTTP traffic in the hope that pedophiles don’t realise that they can use encrypted email, HTTPS, or even a VPN to transfer files without them getting blocked. If someone wanted to bring serious amounts of data to Australia, getting a tourist to bring back a few terabyte hard disks in their luggage would probably be the easiest and cheapest way to do it. Posting DVDs is also a viable option. Given that the Internet censorship plan is doomed to failure, it would be best if they could spend the money on something useful. Getting a better Internet infrastructure in the country would be one option to consider. The cost of Internet connection to other countries is determined by the cost of the international cables - which can not be upgraded quickly or cheaply. But even within Australia bandwidth is not as cheap as it could be. If the Telstra monopoly on the local loop was broken and the highest possible ADSL speeds were offered to everyone then it would be a good start towards improving Australia’s Internet access. Australia and NZ seem to have a unique position on the Internet in terms of being first-world countries that are a long way from the nearest net connections and which therefore have slow net access to the rest of the world. It seems that the development of Content Delivery Network [4] technology could potentially provide more benefits for Australia than for most countries. CDN enabling some common applications (such as Wordpress) would not require a huge investment but has the potential to decrease international data transfer while improving the performance for everyone. For example if I could have a Wordpress slave server in Australia which directed all writes to my server in Germany and have my DNS server return an IP address for the server which matches the region where the request came from then I could give better performance to the 7% of my blog readers who appear to reside in Australia while decreasing International data transfer by about 300MB per month.

• [1] http://www.internode.on.net/residential/internet/home_adsl/nakedextreme/pricing/


• [2] http://www.linode.com/


• [3] http://planet.linux.org.au/


• [4] http://en.wikipedia.org/wiki/Content_Delivery_Network

One month with Emacs and counting - Part 2 A while ago I've blogged about my switch from Vim to Emacs, promising a blog post series, quite a mouthful nevertheless, it's time to continue the series. The first part was about why I think that nowadays Emacs is ready to be switching to. This second part is about flawed Vim design choices which substantially contributed to my choice. (Of course all this post is under a IMO-/rant-disclaimer, it can't be otherwise, so take it cum grano salis) Flawed Vim design choices Vim started as a wonderful editor, following the path of Vi, which in turn has got right a good deal of design choices. One is the often debated modality. In spite of being a HCI PITA in general, modality is not a problem at all for software you are using every single day (and the editor is the 2nd program I use the most, with the terminal being the 1st), because you learn it as a habit. In the case of Vim, modality is good because it let you use powerful yet simple motions in command mode, which can be easily combined with operator-pending commands. This can be considered yet another incarnation of the UNIX philosophy, casted inside the editor itself. ... which brings us to the good old joke which describes Emacs as a great operating system, lacking only a decent editor. In fact, Vim is no longer the nice Unix philosophy player which it used to be, and Emacs plays much more nicely with external tools than Vim. As arguments, let's consider a few (not so) recent Vim evolutions:

• Spell checking. Starting with version 7, Vim has got support for on-the-fly spell-checking of text buffers. It is a super-nice feature, which I've advertised in the past, because spell checking is needed in many tasks we daily do with editors (mail/doc writing, code commenting), and having a single interface to do it comes handy. Unfortunately, how that is implemented in Vim is definitely not along the lines of the UNIX philosophy. We have countless different FOOspell implementations whereas Vim implemented yet another one, without relying on any pre-existing implementation. Even more so, the file format of spell files is basically ad-hoc, and not sharable with other tools. This is already a PITA for packaging (let's add to Debian another 20 binary packages vim-spellfiles-FOO), but is even worst than that: the spell files must be generated in ad-hoc ways which are seriously prone to failures (guess why in Debian we don't have yet the vim-spellfiles-FOO packages which used to exist in experimental a long time ago?). What would have been the UNIX-friendly way of achieving the same result? Well, obviously spawning FOOspell and interact with its process from the editor. Keep that in mind.
• Internal grep. A feature of Vim I've always loved is :grep (yet another instance of quick-fixing, which is also used for jumping to compile-time errors). In its original incarnation it was as simple as invoking grep and parse its output. KISS. Then, Vim started to have a problem: grep -r can take ages and in the meantime the editor was stuck. Hence, starting with Vim 7, vimgrep has born: an internal re-implementation of grep. OK, it is more portable (but how many of us do care? on how many system you use you are missing grep? damn, I used to install it also on Windoze!), and the editor retain control. Control that is exploited just to show a nice progress bar of what is being scanned ..., while the editor is still stuck. Well, to me it looks like the problem would have been solved much more nicely by adding the ability to interact with an external process (deja vu?). That process can then be grep -r. Full stop.
• Lack of debugging support (on purpose). Long standing missing feature of Vim: you can not "easily" drive a debugger from Vim, because a firm choice in Vim is to ... (OK, I confess, I'm starting to beat a dead horse) ... not have support for interaction with external processes. I found such a choice quite dumb: to me it is evident that it induces a trade-off between being feature-complete wrt programmer needs (as available external tools get available to face those needs) and not "re-implementing the wheel" inside the editor. Going back to the lack of debugging support in Vim, that has spawned projects like GdbVim, Clewn, Bram's Agide, and my own tiny teeny incarnation specific to ocamldebug called WOWcamldebug. Every single project of that list acts as an intermediary between Vim and an external process (a debugger of some sort), whereas that could have been implemented by an editor plugin, provided that the editor offer, guess what, support for interaction with external processes.
• Top-level. Similar to the debugging support issue, there is the interpreter issue, namely how to evaluate "phrases" of your interpreted language of choice directly from the editor. Nowadays a lot of languages have the so called top-levels which evaluate language phrases interactively (Python, Ruby, OCaml, many Lisps, ...) and show evaluation results. That ability can speed up testing considerably, but can become cumbersome to use for large code snippets without editor support (good luck with copy and paste). Yes, Vim has support for that, but the implementation choice is close to be ridicolous: only if you have linked Vim together the interpreted for a given language, then you will be able to interactive evaluate phrases of that language. What if you program in Python, Ruby, and Perl? Well, you need to link all of them. What if you also program in OCaml? Then you're screwed, because Vim currently does not support linking with it. (Yes, I know that there are other reasons for doing that linking, they are discussed below.) Do I need to tell you which feature will be enough to address top-level support? interaction with extern.. OK, I'll drop it, you know that already.

Well, I acknowledge that Emacs got this choice right, and that it's an important one: Emacs supports interaction with external processes, and plugin authors have been happy to exploit it to create really cool stuff (sticking to the UNIX philosophy). Please welcome: Flyspell, M-x grep (shameless smartass plug: same number of keystroke of :grep), Grand Unified Debugger, a sampling of language-specific major modes with top-level support. (But I recommend having a look at least once at the coolness of other stuff built-on top of external process support, like Flymake, mentioned by dancer not a long ago.)

• Its own scripting language. I've bored you enough with the external process topic, hence here is the second one: the saga of Vim and its scripting language, i.e., the programming language using which you can customize the editor. Note that the choice of scripting language impacts on 2 different targets: final users in need of fire and forget automations (when they start to get to complex to be implemented on top of, say, macros), and developer of editor extensions such as addons. The impact of scripting language on extension developer is crucial for the evolution of an editor aiming at being powerful. If the scripting language and the API towards the editor is flexible enough, then a lot of cool extensions and features will be developed by third party authors, otherwise the burden stays on editor upstream author.
  1. In its phase 1 (my term, Vim < 7.0) Vim pretended that it needed only a minimal scripting language (Vim script) and that everything else can be developed using external programming languages, linked in the editor. As a consequence Vim script was just a tiny teeny procedural language with editor interfacing capabilities, most notably lacking any "decent" data structures (no lists, no dictionaries, ...). In that phase, if you were a user it was fair enough. You just had to chose your external programming language (at compile time ...), and you were able to write your quick hacks with it. But if you were willing to develop an extension you were screwed, because you had either to impose your external programming language to the final user (that is what gave born to absurdities like vim-full, look at the deps!) or to enjoy a good deal of masochism to explode your 30 lines of Python extension into 200 lines of Vim script to make your Vim extension more "portable".
  2. In its phase 2 (Vim >= 7.0) it has been realized that something were wrong in the initial choices about Vim script, and data structures (lists, dictionaries, function references!) have been added to it, relieving a bit the pain of programming portable extensions. Nevertheless, and probably due to how it started, Vim script is far from being a programming language one can enjoy programming in. Consider for example the standard library of the language. It started as a random collection of unrelated functions being added on a "as-needed" basis, ... and continued along that path! Now the API reference lists together totally unrelated functions, organized inconsistently, and making really painful finding what you need (assuming it exists).

Even in this respect, I acknowledge that Emacs got this right. It has chosen a single, now full-fledged, programming language (Elisp), which is offered both to the final user (fact: the average Emacs user knows much better how to program her own editor than the average Vim user) and to extensions developer. The standard library of the language is organized with a consistent naming (even though not always intuitive) and documented in an organized manner. You might not like the language, but at least is has been taken seriously and managed as such. Finally, being a Lisp dialect, you might need what you learn with Elisp elsewhere. Vim script is too committing: it is useful only inside Vim. I believe that contributed significantly in its scarce diffusion among Vim users. The fact that Elisp was born together with Emacs and that it is also used internally for the editor implementation is not relevant here: I do care about what is offered to me as an user and as an extension developer. What I see on the Emacs side in this respect is much better that what I see on the Vim side. Moreover, from the point of view of the users, is the difference between what is inside the editor and what is outside (i.e., the extensions) that relevant? I think it is not, and that brings me to the final Vim choice which I consider flawed and which has the potential of seriously limit its future evolutions:

• Editor as a distribution. Emacs is managed as a intermediate software distribution layer, similarly to what happens in other softwares where many third party authors cooperates (e.g., TeXLive). The distribution architecture has advantages and contributes to the longevity and potentialities of a software project. What does this mean concretely in the Vim vs Emacs dispute? For example it means that Emacs get more feature-complete each passing release, still preserving uniformity in documentation and keybindings. On the contrary Vim sports many addons, which are very likely to step on each other feet when enabled simultaneously. Actually, that is one of the reason which brought me to the creation of vim-addons: you cannot enable all the content of vim-scripts together due to various kind of conflicts. As we know well in Debian, distributions have a fundamental role in blending together lightly-coupled software components, that's role is missing in Vim evolutionary path and I find that worrisome.

Where to from here in this blog post series? For sure some tips on how to migrate for hard-core Vim users, then we'll see ... If you did enjoy the read please let me know commenting in the discussion page (as you did in the last post, thanks!).

It has been reported that the roots of Monty Python's Dead Parrot Sketch are 1600 years old. The original is from a Greek joke collection called Philogelos (The Laugh Addict.) It goes like this:

A man goes up to a student dunce (scholastikos) and says "The slave you sold me died." "By the Gods," the dunce counters, "When he was with me, he never did any such thing!"

One of the speakers at OSCon this year — I forget which one — made a point that ran something like this, heavily paraphrased:

Education used to be an end in itself, not a means. It wasn’t about having a high-paying career. It was about knowing the world, about having knowledge and wisdom for its own sake. It was, quite bluntly, the accumulation of useless knowledge by the elite — those that could afford to spend time on such things, knowing that useless knowledge has a way of becoming useful in the most unexpected of ways. How fortunate we are to live in an age where the accumulation of useless knowledge is available to so many, and how sad it is that so few take advantage of it.

What a powerful statement, and it rings true to me. I remember in high school, when people from the local liberal arts college would come and talk. They’d talk about the value throughout a lifetime of knowledge in a broad range of disciplines: English, history, political science, religion, science, and the arts. They’d talk about how their graduates went on to lead distinguished lives, how this broad core of knowledge serves a person well through life. I guess I didn’t believe them, because due to their lack of a computer science major, I went elsewhere. That local school may not have been the best choice for me for other reasons, but as I look back on it, I think they had a much stronger message than I realized back then. Here I am, just two math classes, one computer science class, and one biology class away from a degree. Yet I have had not one class covering the history of east Asia, not one class on different world cultures or religions, and only a very basic understanding of one foreign language (German). This hits me in the face almost every day. Yesterday I was wondering about the history of slavery and racism in Europe. Today I’m curious about China’s history as an economic powerhouse. Last week I was curious about Roman law and daily life. The fact is, everything from philosophy to calculus is screamingly relevant to daily, modern life. We hear talk of “an American revolution” in Washington, of a shift of power in the Senate. It seems we forget that the notion of a Senate is considerably older than the United States is — and that we have such a thing because our founders were aware of this. Macroeconomic theory is thrust in our faces on an almost daily basis these days, yet I’ve never had a class on economics at all. We might feel fear of terrorist attacks, or see our fellow citizens lash out at “the Arabs.” Our own short memories fail to remind us of the light in which we are seen, fail to put the really quite minor terrorist threat in context of what London or Dresden endured in World War II. We demand our government to make us safer, and our government responds by making us less safe but making us *feel* safer at airports. In my own field, I see some universities buckling to pressure from Business to turn out large numbers of mediocre programmers that know the Java or .NET standard library well, but have no sense of the theory behind computer science, and would be utterly lost if asked to, say, write a recursive QuickSort. I find myself almost completely baffled that some companies that want to hire the world’s best programmers are only looking for people that are already fluent in $LANGUAGE — not ones that are good programmers, and so well-versed in computer science that they can easily pick up any language. I think there is a lot to the argument that a good, broad, classical education can serve a person well in any career. I wish I had realized that a little earlier.

A lot has been said about the elections already. I just want to share my personal experiences about it. During the weekend, Tom invited us to his place in lower Harlem (we live upper Harlem) on 123rd St and Adam Clayton Powell Jr Ave, for Tuesday night. As Tuesday arrived, we didn’t see too many people too excited about it. Living in Harlem, we were pretty used to black people supporting Obama, so getting closer and closer to the elections, we didn’t feel people anxious. On Tuesday, we commuted to work and some of our co-workers got to work late because they have been standing in lines for a couple of hours to vote in polls. The whole day was about talking on how much that day was representing for the US and the world, guessing how they results were going to end up, etc. Later at night, Greg, Phil, Raquel and I headed to Tom’s place for his watch-the-results party. His fridge was full of Blue Moon beer, but he also had some Red Stripe Since I like to go light on politics, I preferred some other American variantion, Coors Light Vermont, Connecticut and a few other New England states were claimed by the Democrats. Matty, a friend of Tom attending the party brought US maps with the electoral votes on each one for people to color them as they were called. It was pretty fun. At some point we forgot about the maps since it was pretty clear that Obama was gonna win. But when they called him at around 11 pm (and I was starting to feel nicely drunk), the whole neighborhood and people in the streets just went completely nuts. It’s something I hadn’t experienced at all in my life, it was like if Mexico would have won the f tbol World Cup (which yes, it will never happen anytime soon :P), but nicer. People went out to the Harlem State Office Building square were cars were honking, and a huge amount of people gathered to celebrate, play brass music, cheer “Yes We Can”. Ghetto guys, white hippies, curious latinos, Africans, African Americans, Puerto Ricans, just a lot, a lot of people, all kinds of them, just a clusterfuck. They projected Obama’s speech from Chicago and people went nuts. It was just, fucking amazing. The images they projected from Harlem to the outside world was in that same square we were in. Our Flickr set for the celebration here.

The Open Source Repository and Observatory (OSOR), a new site sponsored by the European Commission to foster the exchange of FOSS related information and software among European public administrations, recently published guidelines on the procurement of open source software. Public administrations in Europe have to follow public tender procedures and the new guidelines give practical and legal advice on how open source software and related services can be incorporated into the procurement process. Rishab Ghosh, who presented the guidelines at the Open Source World Conference in Malaga, argued that the procurement guidelines were needed because of two reasons. First, they studied recent tenders and found that many explicitly mentioned proprietary applications. 16% of 3615 software tenders explicitly asked for products from top 10 software vendors, such as Microsoft, SAP and Oracle. This practice may be illegal because public tenders generally have to describe functional requirements in a general way instead of specifying specific products. Second, many public administrations don't have any experience with the procurement of FOSS. In fact, they often don't know whether or under which circumstances they are allowed to adopt and ask for FOSS solutions. The guidelines are specifically designed in order to clearly and simply explain how public administrations can acquire open source and they don't assume that a country has adopted a specific policy regarding open source. The guidelines include a long section about open standards, open source and how they relate. Both open standards and open source align very well with the needs of public administrations who have an "obligation to support interoperability, transparency and flexibility, as well as economical use of public funds". The guidelines argue that the exit cost, i.e. the cost incurred in moving to another IT system, is also an important consideration but one that is often neglected. The adoption of a proprietary solution without open standards may limit the future choice, thereby increasing the long-time costs and giving the proprietary vendor an unfair advantage in future tenders. The procurement guidelines describe two ways of acquiring FOSS: it is possible to go the usual route of publishing a tender for the supply of software (possibly with related services). However, in the case of FOSS, it is also possible to download the software directly from the Internet. This is possible because the software is not only free of charge but comes with no contractual obligations. If there were any obligations involved with the download (such as fees, the agreement to an EULA or the requirement to purchase services in the future), software download is not an allowed method. What I like about the guidelines is that they explicitly say that downloading software has to be part of the formal procurement process. You have to think about your requirements, look at various alternatives, and so on, and not just blindly download something from the Internet. When it comes to the procurement of FOSS, the guidelines don't suggest that tenders should explicitly ask for FOSS. Instead, they should describe the functional requirements of the software as well as certain properties. For example, a tender could specify that the public administration as well as third parties must have the right to study, distribute and modify the software. In a sense, the guidelines suggest that tenders should include the principles of the Free Software Definition along with justifications for these requirements. Personally, I think there is a great need for these procurement guidelines. There are many public administrations that don't know how to acquire FOSS and these guidelines offer clear advice. Furthermore, I find the guidelines very balanced. They don't recommend that you should always ask for FOSS but incorporate FOSS principles into tenders where it makes sense. It remains to be seen whether the procurement guidelines will have an impact on the FOSS adoption in Europe, but I surely hope so. (Originally published on FOSSBazaar)

If you ask the old timers about how they got started in Debian they'll tell you about sending off an email saying they wanted to get involved and having an account setup the same day. By the time I got round to joining it was during tbm's blitz when NM opened up again and it took only a few days to get through the process. When I first AMed, back in 2002, it took anywhere from a few days to a few months to process an applicant. I've recently started AMing again and so far my NM and I have exchanged 9 mails over the course of 2 weeks and still aren't through Philosophy and Procedures.

This isn't due to any failing on the part of my NM. It's just that there's a lot more "paperwork" to get through these days. There's currently a lot of discussion going on about the idea of introducing new classes of Debian involvement. I think that's something that's definitely worth looking at; it's already in existence for Debian Maintainers vs Debian Developers. However anything new that's introduced really needs to make some effort to be lighter than NM; someone who just wants to handle a few packages is going to get put off by too many hoops and the current NM process is already largely constrained by available AM time rather than anything else. I don't pretend to know what the right solution is, but I think anyone attempting to produce one should be praised for that rather than attacked.

I am waiting in great anticipation for Phil Lapsley to publish his History of Phone Phreaking book. In the mean time, I get to enjoy tasty and funny morsels that emerge out of his research blog. Apparently, the FBI, even while invading privacy, also protect, I guess when some exceptional circumstances arise.

Alain Williams just sent me a link to the Atheist Bus Campaign which was started to raise £5,500.00 GBP in order to buy adverts on the sides of busses in London. As I write they have raised £111,832.43, so over 20 times what they needed for the London busses. I'm so impressed with this that I gave them £100.00 Update: I note the campaign's page, where among other things they link to a clip from the BBC's Have I got news for you which is rather amusing about this, particularly the last joke. Also, Justgiving's blog seems in awe about how this has taken off. As they mention, there's even a wikipedia page about this.