Six months ago I investigated the options for Xen virtual servers [1]. I ended up receiving an offer of free hosting and not needing that, but the research was useful. There is a good range of options for Xen servers with different amounts of CPU power, RAM, bandwidth, and disk space. There are a couple of things that seem to be missing, options to upgrade from virtual servers to physical servers, and information on dedicated disk and database performance - but I’ll explain that later after some history. About a week ago a client needed a Xen virtual server in a hurry, their main server (a Xen system that I run on hardware that they rent) was getting a bit overloaded and needed to have one of the largest DomUs moved off. I ended up recommending Linode [2] based on my research and comments I received. The Linode server is working quite well and the client is happy, one nice feature of Linode is the choice of server rooms that they offer. I was able to choose a room in the same region as the other servers that the client owns and thus get ping times that are sometimes less than 2ms! Due to a missing feature in a program that I’m maintaining for the client a large number of MySQL queries are being made. Due to a problem I’m having with MySQL it won’t let me create a slave database server so all the queries go over the VPN and use a large amount of data. This combined with the other traffic that should be going over that link means that about 600G per month is being used, fortunately that is rather cheap. Linode staff handled this very well, after the server had exceeded it’s quota by 120G they asked my client to confirm that the traffic was legitimate and then suggested an upgrade to a plan that could handle the traffic (which went smoothly). Now I have another week to add the feature in question before I meet the quota again. Shortly after getting the new virtual server running at full capacity David Welton wrote a detailed review of Linode and Slicehost for the issues that matter to his use [3]. His conclusion seems strongly in favor of Linode. But now I am looking at getting a Slicehost [4] virtual server for the same client (for a different project) because Slicehost is owned by Rackspace [5], and the new project if successful will need a set of powerful servers and Rackspace seems like a reasonable company to host that. The problem with Rackspace is that they (and every other ISP I’ve researched so far) seems to offer little in regard to customers who need serious disk IO. I am planning some servers that will have a write bottleneck on a MySQL database (or maybe multiple shards), so serious disk capacity is needed. At least I would like to be able to get disk storage by the tray (12-14 disks) with the controllers having RAID-6 support. Rackspace only offers RAID-5 (according to the “livechat” person), and we didn’t get as far as discussing how to add more trays. What would be ideal is if there was an ISP that had both virtual servers and physical servers (so I could start with a virtual server and move to a physical server when things are working well), and also serious storage options. They would offer internal disks, external RAID arrays, and NetApp Filers [6] (or some equivalent device). It would be really nice if I could just instruct the ISP to add another NetApp Filer to my back-end network and have it done for me (I’m certainly not going to visit the US to install new hardware). It’s been over a day since I submitted a sales request to NetApp asking whether they partner with any ISPs and I haven’t received a response. OpenSolaris with ZFS also sounds good for disk IO performance (they have similar features to NetApp). Unfortunately the support for OpenSolaris among ISPs is not that great (while everyone offers Linux and Windows), and I haven’t used any recent version of Solaris. So using OpenSolaris would require finding someone with the skills to manage it who can work for my client - as opposed to a NetApp device that would be just like any other NFS server, SAN, or iSCSI server. But I’m not ruling OpenSolaris out, if someone knows of a good ISP that hosts OpenSolaris machines and supports adding dozens of disks and decent amounts of NVRAM for ZFS then I would be interested to investigate it. Joyent has some interesting OpenSolaris virtual server plans [7], they are a little pricey but offer large amounts of data transfer. They don’t provide any information on disk IO capacity (other than saying that they use ZFS for good performance). I’ve just downloaded Nexenta (Debian with the OpenSolaris kernel) [8] and will test it out over the next few days. One of the reasons I’m tending towards Rackspace at the moment (with Slicehost as the entry point) is that they seem cooperative to customer requests. My discussions with them (on a web based “livechat” and on the phone) have indicated that they may be able to do something special for me.

• [1] http://etbe.coker.com.au/2008/05/28/xen-hosting/


• [2] http://www.linode.com/


• [3] http://journal.dedasys.com/2008/11/24/slicehost-vs-linode


• [4] http://www.slicehost.com/


• [5] http://www.rackspace.com/


• [6] http://www.netapp.com/


• [7] http://www.joyent.com/


• [8] http://www.nexenta.org/

jhead is a very nice and very powerful command line utility to mess with JPEG headers (esp. EXIF fields).

  $ apt-get install jhead

It can display/extract a great amount of metadata fields from JPEG files and also extract the thumbnails stored in JPEG files (if any). The following will list all known metadata fields from a sample photo:

  $ wget http://farm4.static.flickr.com/3173/3061542361_60acb0904b_o.jpg
  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1074172 bytes
  File date    : 2008:11:26 23:38:04
  Camera make  : Panasonic
  Camera model : DMC-FZ18
  Date/Time    : 2008:03:05 15:45:52
  Resolution   : 3264 x 2448
  Flash used   : No
  Focal length : 4.6mm  (35mm equivalent: 28mm)
  Exposure time: 0.0100 s  (1/100)
  Aperture     : f/3.6
  ISO equiv.   : 100
  Whitebalance : Auto
  Metering Mode: matrix
  Exposure     : program (auto)
  GPS Latitude : N %:.7fd %;.8fm %;.8fs
  GPS Longitude: E %;.8fd %:.7fm %;.8fs
  GPS Altitude : 174.00m
  Comment      : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.
  ======= IPTC data: =======
  Record vers.  : 4
  Headline      : Felsburg auf dem Felsberg
  (C)Notice     : www.froutes.de
  Caption       : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.

As you can see there's a huge amount of potentially privacy-sensitive metadata in your typical JPEG as generated by your camera (including camera type, settings, date/time, maybe even GPS coordinates of your location, etc). You can extract the thumbnail stored in all JPEGs in the current directory with:

  $ jhead -st "&i_t.jpg" *.jpg
  Created: '3061542361_60acb0904b_o.jpg_t.jpg'

Note that the JPEG thumbnail does not necessarily show the same picture as the JPEG itself. Depending on the image manipulation software that was used to create the edited/fixed/cropped JPEG, the thumbnail may still reflect the original JPEG contents (see sample image on the right-hand side). This is a huge potential privacy issue. There have been a number of articles about this some years ago, in case you missed them:

• disLEXia: Hidden data in Document Formats
• disLEXia: EXIF Thumbnail in JPEG images (many sample JPEGs and their differing thumbnails)
• Kevin Cohen: Digital Still Camera Forensics (PDF)

Thus, an important jhead command line to know is the following, which removes all metadata (including any thumbnails) from all JPEG images in the current directory:

  $ jhead -purejpg *.jpg
  Modified: 3061542361_60acb0904b_o.jpg

As you can see the result is that only very basic information can be gathered from the file afterwards:

  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1052506 bytes
  File date    : 2008:11:26 23:38:04
  Resolution   : 3264 x 2448
  $ jhead -st "&i_t.jpg" *.jpg
  Image contains no thumbnail

I recommend doing this for most photos you make publically available on sites like flickr etc. (unless you have a good reason not to). Finally, see the jhead(1) manpage for lots more options that the tool supports.

Some time ago Bill Joy (who is famous among other things for being a co-founder of Sun) [1] wrote an article for Wired magazine titled “Why the future doesn’t need us” [2]. He wrote many sensible things but unfortunately focussed on the negative issues and didn’t receive a good response. On reading it today I thought more highly of the article than I did in 2000 when it was printed, largely due to having done some background research on the topic. I’ve recently been reading Accelerating Future [3] which has a more positive approach. Now a talk by Bill Joy from 2006 has been published on the TED.com web site [4]. He starts by talking about “super empowered individuals” re-creating the 1918 flu. He also claims that “more technology super-empowers people more“. Such claims seem to be excessively hyped, I would be interested to see comments from someone who has a good knowledge of current bio-technology as to the merits of those claims. He talks briefly about politics and has some good points such as “the bargain that gives us civilisation is the bargain to not use power” and “we can’t give up the rule of law to fight an asymmetric threat - which is what we seem to be doing“. He mentions Moore’s law and suggests that a computer costing $1000 then (2006) might cost $10 in 2020. He seems to be forgetting the cost of the keyboard and other mechanical parts. I can imagine a high-end CPU which cost about $800 a couple of years ago being replaced by a $2 CPU in 2020, but I don’t expect a decent keyboard to be that cheap any time before we get fully automated nano-factories (which is an entirely separate issue). Even the PSU (which has a significant amount of government regulations for safety reasons) will have a floor cost that is a good portion of $10. Incidentally the keyboard on my EeePC 701 sucks badly, I guess I’m spoiled by the series of Thinkpad keyboards that I keep getting under warranty (which would cost me a moderate amount of money if I had to pay every time I wore one out). I will make a specific prediction, that by 2015 one of the better keyboards will comprise a significant portion of the entire cost of a computer system (more than a low end computer unit) - such that in some reasonable configurations the keyboard will be the most expensive part. It would be good if PCs could be designed to use external PSUs (such as the Compaq Evo models that took laptop PSUs). Then the PSU, keyboard, and monitor would be optional extras thus giving a small base price. Given that well built PSUs and keyboards tend not to wear out as fast as people want to replace computers, it seems that financial savings could be provided to most customers by allowing them to purchase the computer unit without the extra parts. People like me who type enough to regularly wear out keyboards and who keep using computers for more than 5 years because they still work are in a small minority and would of course be able to buy the same bundle of computer, PSU, and keyboard that new users would get. In 2020 a device the size of an iPaQ H39xx with USB (for keyboard and mouse) and the new DisplayPort [5] digital display interface (that is used in recent Lenovo laptops [6]) would make a great PDA/desktop. You could dock a PDA the way some people dock laptops now and carry all your data around with you. Bill cites an example of running the Mac interface on an Apple ][ (does anyone know of a reference for this) as an example of older hardware being more effective with newer software. It's a pity that often new software needs such powerful hardware. EG it's only recently that hardware developments have overtaken the developments of OpenOffice to make it deliver decent performance. He has an interesting idea of using insurance companies to replace government regulation of food and drugs. The general concept is that if you can convince an insurance company that your new drug is not a great risk to them so that they charge a premium you can afford then you could sell it without any further testing. Normally I'm in favor of government regulation of such things, but given the abject failures of the US government this idea has some merit. Of course there's nothing stopping insurance companies from just taking a chance and running up debts that they could never hope to repay (in a similar manner to so many banks). Finally I think it's interesting to note the camera work that the TED people used. My experience of being in the audience for many lectures (including one of Bill Joy's lectures in the late 90's) is that a speaker who consults their notes as often as Bill does gives a negative impression to the audience. Note that I'm not criticising Bill in this regard, often a great talk requires some significant notes - very few people can deliver a talk in a convincing manner entirely from memory. It seems to me that the choices of camera angle are designed to give a better impression than someone who was seated in the audience might receive - there's no reason why a video of a talk should be spoiled by seeing the top of the speaker's head while they consult their notes!

• [1] http://en.wikipedia.org/wiki/Bill_Joy


• [2] http://www.wired.com/wired/archive/8.04/joy.html


• [3] http://www.acceleratingfuture.com/michael/blog


• [4] http://blog.ted.com/2008/11/what_i_worry_ab.php


• [5]
  
• [6] http://lenovoblogs.com/insidethebox/?p=161

I love it when a lack-of-humor and lack-of-appropriateness-originated flamewar causes somebody to point me towards a very nice display of intelligent humor. Specially when it is so close to me, to my roots, to my family and my personal history. FWIW, for several years, while I was a BBS user, I used WereWolf as my nickname. Great thanks to Frank K ster - and, of course, to Christian Morgenstern.

The Werewolf - English translation by Alexander Gross A Werewolf, troubled by his name,
Left wife and brood one night and came
To a hidden graveyard to enlist
The aid of a long-dead philologist. "Oh sage, wake up, please don't berate me,"
He howled sadly, "Just conjugate me."
The seer arose a bit unsteady
Yawned twice, wheezed once, and then was ready. "Well, 'Werewolf' is your plural past,
While 'Waswolf' is singularly cast:
There's 'Amwolf' too, the present tense,
And 'Iswolf,' 'Arewolf' in this same sense." "I know that--I'm no mental cripple--
The future form and participle
Are what I crave," the beast replied.
The scholar paused--again he tried: "A 'Will-be-wolf?' It's just too long:
'Shall-be-wolf?' 'Has-been-wolf?' Utterly wrong!
Such words are wounds beyond all suture--
I'm sorry, but you have no future." The Werewolf knew better--his sons still slept
At home, and homewards now he crept,
Happy, humble, without apology
For such folly of philology.

I'm writing a piece of autobiography/alternate world fiction, using git. Whether it will get finished or be any good, or be too personal to share I don't know. The idea though is sorta interesting -- a series of descriptions of inflection points in a life, each committed into git at the time it describes. As the life paths diverge, branches form, but never quite merge. Reading this would not be quite like reading one of those choose your own adventure books. Rather you'd start at the end of a path and read back through the choices and events that led there. Or browse around for interesting nuggets in gitk. Or perhaps the point isn't that it be read at all, but is instead in the writing, and the committing.

Yesterday I exchanged my “old” Thinkpad T60 with a shiny new T500. This time I ordered a Thinkpad with integrated Intel graphics since it’s supposed to be way better supported than the graphic chips from ATI or NVIDIA under Linux. And of course it is supposed to be much more silent. I don’t regret this decision so far, the new T500 is barely hearable even under heavy load. Debian I installed Debian unstable (aka. frozen for waaayy to long) without a problem. Sound worked out of the box, graphics worked somewhat. The kernel automatically loaded the correct modules but I had to tell Xorg that I want the “intel” driver instead of Vesa. Unfortunately Xorg’s intel driver in unstable is already too old to allow me to use an external monitor via the docking station’s DVI port, so I had to to upgrade to Xorg from experimental, which is not very cool since this is the machine I depend on at work. Wifi also doesn’t work, since the the driver and firmware available in unstable are outdated. Suspend to RAM works sometimes, but most of the time the laptop refuses to wake up properly so I have to shut it down. I’ve not yet tested if bluetooth and Wimax works. So currently the T500 is usable under unstable but without external monitor and without Wifi. If you are brave enough to use packages from experimental, you get your external monitor to work but still don’t have Wifi. Suspend to RAM does not work reliably. The T500 should probably work fine with Lenny but of course without external Monitor and Wifi support. Vista After I’ve installed Debian and moved my /home from the old laptop to my new baby, I noticed that silly “Windows Vista” sticker on my laptop. Of course I tried to remove that thing, which I regretted painfully. After I’ve successfully removed the sticker — which was surprisingly hard — some glue remained on the surface of the laptop. No problem I thought, nothing some nail polish remover can’t handle. Unfortunately it only smeared the glue but did not resolve it as I had expected. So I’ve tried perfume, water, soap, rubber, tape and solvent for several hours, but was not able to remove the glue at all. All I achieved was to smear the compact but thick chunk of glue to a very thin but much more distributed layer of this nasty glue on my laptop. Now it looks like a 5×5cm square of dirt on top of my laptop — not sure if removing the Vista sticker was worth it.

I spent the second half of this morning delivering leaflets asking people to vote for the Weston Woods Path Project in today’s itv People’s Millions Big Lottery Fund competition. Just 50,000 gets a smooth, fairly flat, surfaced and edged path and takes some pedestrians off the bendy cliff-top road. Any surplus money from the call charges will be donated to charity. If you vote for it, you can comment on my blog to let me know if you want and you might like to watch WsM Forum for the reaction/news. It’s a beautiful day and North Somerset is a beautiful place but by hell is my jaw cold now! At least there’s no snow here.

I knew for a long time that rails is somewhat fun to work with. The first time I read something about it, I went to check it out, and kindof liked the idea. Web development really isn't my cup of tea, so I didn't do anything with rails; but really, the idea made somewhat sense to me. It's unfortunate that the implementation is such a piece of crap. Luckily I don't have to deal with it firsthand, but the stories I hear and read everywhere speak for themselves. I won't enumerate them all; but in case you're not familiar with them, the Debian Ruby team has a pretty clear explanation on their website. In addition to complaints by packagers, I also hear that many ruby developers recommend that you install gems inside the project you're running (or that you ship your software in such a way that it includes those libraries as part of your source). Oh, the horror. Now, however, they've really topped it. Via Koen's blog, I found some other page that contains this:

Now, in theory, to update to the latest version of RubyGems, you just need to run

sudo gem update --system

But - depending on what version of RubyGems you have installed, running that command may lie and tell you that you have nothing to update. If you re currently on RubyGems 1.1 or 1.2, you need to run a different set of commands to update RubyGems:

sudo gem install rubygems-update
sudo update_rubygems

Seriously. This is why we have distributions, packagers, and, really, people spending a whole shitload of time making sure everything works together nicely. This is why there is Debian, RedHat, CentOS, Mandrake, SuSE, and whatever. Me touching rails? Not with a 10-foot pole. Not if you pay me a million, in any currency. Stay away from my system!

My home workplace is slowly and steadily mutating into a never ending story. I do not remember blogging every aspect of it, but after three graphics cards, an even older mainboard and two DVB-S-Cards, my home workplace PC currently does what I expect it to do: Run Debian unstable, drive two 20 inch DVI TFT monitors with 1600x1200 pixels each and receiving DVB-S transmissions. I do not think that these are exaggerated expectations, but it took over three months to find a combination of hardware which will actually do what I want. The hardest part was finding a AGP graphics card which can drive two DVI monitors with 1600x1200 pixels each. After failing with two different Matrox cards (the G550 not being able to do 1600x1200 pixels if the monitors are connected via DVI), I finally settled on a used GeForce FX 5200. In the beginning, the binary nVidia module didn’t hurt as much as I expected. Unfortunately, this rapidly changed with the 2.6.27 Linux kernel. Unfortunately, the Linux kernel changed its interface with 2.6.27, which stopped the nvidia-kernel-source 173.14.09 from sid from compiling. Contrary to what is told in #500285, the driver doesn’t compile on my system even when I clean out /usr/src/modules/nvidia-kernel prior to unpacking the driver .tar.gz. I already thought that my problem was solved when I noticed nvidia-kernel-source 177.80 in Experimental. The module compiles just fine against kernel 2.6.27, but...

Nov 21 17:30:53 weave kernel: NVRM: The NVIDIA GeForce FX 5200 GPU installed in this system is
Nov 21 17:30:53 weave kernel: NVRM:  supported through the NVIDIA 173.14.xx Legacy drivers. Please
Nov 21 17:30:53 weave kernel: NVRM:  visit http://www.nvidia.com/object/unix.html for more
Nov 21 17:30:53 weave kernel: NVRM:  information.  The 177.80 NVIDIA driver will ignore
Nov 21 17:30:53 weave kernel: NVRM:  this GPU.  Continuing probe...
Nov 21 17:30:53 weave kernel: NVRM: No NVIDIA graphics adapter found!

Don’t closed source binary-only drivers just suck? I mean, granted, the FX 5200 is a five year old design, but it does its basic office job just fine. So I currently can choose to stick with a legacy kernel of the 2.6.26 series to be able to run the legacy nVidia driver, or can shell out money for a new graphics card and a new mainboard, a new CPU and new memory (since current graphics cards aren’t available for AGP any more). Just fine. Sucks. If I get around to buying a new box, which graphics card manufacturer is to be trusted these days? nVidia is out of the question after my current experiences, Intel doesn’t make graphics cards as far as I know, and ATI/AMD? I hear that they have recently seen the light, but are their open source drivers mature enough so that ATI/AMD products can already drive high resolution dual DVI setup? I currently do not care about 3D performance, I just want to push around windows on a flat KDE desktop and to some serious work. All I want is screen real estate and a picture of decent quality. I’d appreciate any hints and comments, and surely hope that there will be some kind of upstream support for the 2.6.26 kernels until I have refitted my home workplace hardware.

Yesterday I wrote that I'd ordered a new ASUS EEE PC, the 901 model, and today it arrived. The machine is gorgeous though I suspect in the long term I'll regret ordering the white model. This entry was written on the device, slowly due to my fat fingers, with its new Debian Lenny operating system. To save time I didn't even use the default system, I just immediately rebooted it into the installer via a 2Gb USB stick. Unfortunately I had to run through the installation twice, because I made some bad partitioning decisions and decided to fix them rather than live with them. Happily everything on the device appears to work perfectly, although there were a few hiccups along the way. The only niggle is that suspend to RAM seems a little flaky; 50% of the time I try to resume and just get a blank window not my X.org desktop. Happily suspend-to-disk works perfectly, and the bootup/restoration process looks very pretty with the splashy package installed. There are times when I really love using Debian, and this is definitely one of them. Together we've produced an operating system which just works on an amazing array of devices and systems and gets better and better as time goes on! For example in the past I'd always regarded Network Manager as "that thing we remove to stop breaking our system" - but now I see it working correctly with no effort on my part at all. Amazing! I guess I'll be returning my Nokia internet tablet in the near future. This device is bigger, but much more capable and versatile. ObFilm: St Trinians (the recent remake; not too bad. Bonus points for the Shampoo cover).

EMI (Experiments in Musical Intelligence) by David Cope changed my mind on the state of computer music generation software. I had come across numerous "music generation" programs, but it would have been a long stretch to call their results "music". They would produce, at best, strange patterns of beeps. I had heard of a live "jazz improvisation gig partner" machine (probably this one), but EMI is on a completely different level just listen to the MP3 samples. Sure, in the end it is mostly splicing and rehashing of existing pieces, but Cope claims that in blind tests even experienced listeners would sometimes confuse an original piece by a composer and a composition by EMI. In other words, EMI passed the equivalent of a Turing test for music generation software. If that is not impressive, I do not know what is. Unfortunately David Cope's site is skimpy on the details of how EMI works, even though there is an extensive bibliography (note the time span: 20 years!). At least some information can be gathered from the webpages of his numerous books.

Thanks to Koha, TTLLP is partly a library services worker cooperative and just now there’s a massive flame-storm about the largest library services consumer cooperative - OCLC, the Online Computer Library Center - because it just updated the terms for sharing its book data collection. OCLC seems to be treating it as a private asset that it can exploit, not the common resource that many librarians thought it was. I’ve been watching this fire with interest - because, at best, I don’t think it’s helping other cooperatives that sell to libraries - and here is my summary. OCLC and the Great Library Scandal is a good introduction and there’s a summary of the Talis Podcast about OCLC WorldCat Record Use Policy with Karen Clahoun and Roy Tennant which reveals some of the OCLC thinking. Just like CC and Open Source? Still No is a great illustration of how some librarians think OCLC is reaching the wrong conclusion, which then leads to asking Is OCLC truly cooperative? and What would it look like if OCLC was broken up? A commenter on Tom Watson: Library data asks: “Why would libraries play this game?” Well, the road to hell is paved with good intentions and this road has taken 40 years. First off, OCLC is essentially a good idea - libraries cooperating to act as a counter-weight to the large library service companies. However, I believe OCLC predates many recent company rule innovations (stronger common asset locks) and data licensing innovations (Share-Alike, Fiduciary Licensing and the general free and open source software movements), so there’s nothing to stop them trying to privatise the assets (library data) that the libraries have given to OCLC and it was only a matter of time before someone tried. I’m having a similar discussion with another organisation that has been given data by its members under vague terms and is now apparently about to exploit that data for the organisation’s benefit, to the detriment of some members. TTLLP is a member but has not contributed much data because I was suspicious of the lack of terms. In time, hopefully, asset locks and more awareness of data licensing will eliminate these problems, but there’s going to be a lot more people getting burnt first. Finally, a quick shout-out to the Open Library Environment as a possible emerging alternative to OCLC for practical services. Raw Thought: Stealing Your Library: The OCLC Powergrab covers some of its motivation (and includes a link to a petition to OCLC, if you want something to sign), but I share Stefano’s (now there’s someone I’d not read in 7 or so years) analysis of some of the challenges:

“any grass-root approach that will get big enough to take on OCLC on the metadata collection and redistribution service that libraries need will have to incorporate under the pressure of its users (if only for legal liability protection) and will have to find an answer to the same set of problems (policy, governance, financial sustainability) that OCLC has.”

This week (21-28 November) is National Maintenance Week. Do a bit of maintenance on your home this week yourself and save a bit of money, as well as improving the appearance of your local community. National Gutters Day is next Friday (28th). After this summer’s heavy rains led to a curtain of rain falling down, I paid the window cleaners to clear our building’s gutters. Unfortunately, they didn’t clear quite close enough to the end of one gutter and it still overflowed - but I didn’t find this out until the next rainstorm. I’ve bought a long handle and a gutter-cleaning tool for about the price of one gutter-cleaning, so I can deal with it myself in future! How about you? The gutter-cleaning tool I’d really like are the long tongs, but I found three types in web searches and no-one sells any of them in Europe. Why not?

This might be a good message to write in Spanish... But then again, a long time ago I decided this is an English-posting site. So be it, I'll only have to give more background information.
This day marks the date when, 98 years ago, Francisco I. Madero started the Mexican Revolution - About a decade of unrest, civil war and ideologies. The revolution is what created the violent, uncivil image of the Mexican, which accompanied us for long years in many foreigners' minds. The revolution brought to an end 30 years of a single-man rule, the Porfiriato. But that's only the major symptom - The Revolution had many, many other consequences. About one million (out of a 10 million population) people died. There was a very significative rearrangement of the society, a rearrangement that took about half a century to settle. But I won't write more background - You can always ask the wikipedia about our Revolution.
The reason I am posting this is that, as it usually happens in this time of year, several so-called analysts in the media have started asking, was the Revolution really worth it? Did it change anything at all? Did the Revolution in the end win, or was it defeated from within? Should we still celebrate it?
And there are, yes, reasons to doubt it. Renato Leduc, at the same time a great journalist and a delicious poet, says it as many - while at the same time, as nobody else: Tiempos en que era Dios omnipotente / y el se or Don Porfirio presidente / Tiempos, ay! tan iguales al presente, or ya se est n muriendo todos / Jes s qu desilusi n...! / se est volviendo gobierno / Ay dios...! La revoluci n.
Anyway... Our media overlords insist on us forgetting the struggles and the real changes that came from them, on rewriting the history... Probably they will push us later on to have the cristeros as the real fathers of the Nation?
Even if so many bits of reality didn't change after Porfirio Diaz's regime fell in 1910, I find it insulting to think that even 70 years of PRI -with very sharp differences between periods, with huge differences between the PRI-born governments- are comparable to 30 years of a one-man rule; even with brutal repressions such as the dirty war against so many subversive movements in the 50s-80s (as officially There Was No Armed Struggle Anymore, just some pesky communist subversives), it cannot be compared to the Porfirian Peace (ask Cananea and R o Blanco). Today we might have a shameful concentration of money and power in very few hands (including the world's richest man), but it certainly does not reach the point of 1910 where most of the Mexican soil was owned by less than 30 families, with latifundios as big as many states...
Anyway - So far, nothing new - just bits I heard here and there, and my reactions to them. But this morning, around 8:25, I tuned in to Noticias IMER, the news program of one of the few public, non-gubernamental, independent radio stations. An interview was under way, but I could not get the interviewed person's name (I guess, a historian - will write to ask for his data). His comments were very interesting, and very worth echoing. I'll try not to distort him.
The Revolution started off very organized, and with a very simple goal: Get Porfirio Diaz out, and call for real federal elections. Sufragio efectivo, no reelecci n. Of course, the fight was very short, and Madero became the president, with an overwhelming majority. Of course, also, the reactionary sectors set up a coup and killed Madero. Victoriano Huerta seized the power - and that's where the real revolution really began. Groups all over the country (some of which were at unrest since Madero, as they were not seeing the changes they needed - changes that would bring an end to the huge class differences and disrespect to the native Mexican population) rose in arms, and forced Huerta into exile. Then, they battled each other for many more years. It became known as la bola - When somebody joined the revolutionary forces, people said he went to fight with the crowd. But, inside the crowd, there were very different points of view. No, Carranza, Villa and Zapata (the foremost leaders in the hardest part of the fight) were not power-hungry barbarians - much to the contrary. They had very full, very complex views of the problem and possible solutions. I won't delve much into them, also, as I'm not an expert...
Villa and Zapata had the most compatible approaches, seeking an aggresive land redistribution, a communal property system (closest to most of the indigenous population's roots, what we would now call usos y costumbres). For the government, both favored going towards a Europe-like parliamentary system, where the parliament were the real force, and the president (or prime minister or whatever) would only be the designated person to implement the parliament's decision. Both Villa and Zapata feared the evil stemming from the unlimited power that the Presidential Chair symbolized (Fui soldado de Francisco Villa / de aquel hombre de fama mundial, / que aunque estuvo sentado en la silla / no envidiaba la presidencial). They met at the Aguascalientes convention, and were quite close to each other - but were defeated by the superior Venustiano Carranza (Constitucionalista) army.
Carranza, although vilified for his corruption (nowadays, carrancear is still a synonim for stealing), had an opposite view - also originating from a very deep analysis. Carranza saw that what brought down Madero was, in the end, the lack of power of the President to rule the country without support from the legislative power. So, he pushed a political program making the President the strongest man in Mexico. He and his people wrote and passed the 1917 Constitution, valid today. This constitution goes to great lengths pushing revolutionary ideals - Land and wealth redistribution, universal and free education, keeps a complete separation between state and church, ensures state control over strategic areas... The 1917 constitution is one of our history's greatest achievements.
But, of course, it is not perfect - it paved the way for a hegemonic party controlling the real power behind it all. PRI started as a very heterogeneous mixture of the whole revolutionary family, but slowly became a bureaucratic, stagnated monolith.
And in a somehow ironic twist of destiny, the forces that today push for deepest changes, and precisely in the same direction that Villa and Zapata wished, are... Ej rcito Zapatista de Liberaci n Nacional (EZLN) and Frente Popular Francisco Villa (FPFV). EZLN is far more successful and advanced in its social experiment. Again, I won't comment further in what I don't really understand.
As a last point, the commenter I'm quoting (and whose name I must get, to update this post!), said that practically in every country that has transited from any sort of dictatorship towards a more-or-less believable democracy (say, everywhere in South America, or Spain, or Eastern Europe, or...), one of the first steps has been to update or replace the constitution with a new one, preventing the mistakes overlooked by the previous one from being reinstated. In our country, we have long heard about the "Reforma del Estado", a very nice-sounding-term which nobody believes in. After the 2006 electoral mess (no matter who won in the end, everybody will agree it was a mess that should be prevented from happening again, we had high hopes of real changes being introduced. A parliamentary, or at least semi-presidentialist regime was strongly suggested as a way forward. Changing the electoral system towards having second-rounds if needed. _anything!_ But no, we were stuck with... The same as always.
So, did the Revolution win or lose? It is clear to me. It won, and it really shaped -for better- what would happen in the next 100 years. However, in a century, we have been able to twist the law to make it turn against itself. I have to agree with my EZLN-minded friends (I sympathize with EZLN's general goals, but don't think its way forward is the right way to go): Pushing the change from within the government is just wishful thinking, but a strong delusion. However, is there a way to push our country forward without repeating a violent cycle? I really hope so. Our current situation is simply pathetic.
I lack a good closing for this post... So I'll let good old Jefe Pluma Blanca, Renato Leduc, do it for me.
Tiempos de Pancho Villa
y de la guerra de mentadas y tiros en la sierra.
Tiempos de fe
no en Dios sino en la tierra Por el cerro de la Pila
fueron entrando a Torre n
mi general Pancho Villa
y atr s la revoluci n...
Ay jijos...! ya se nos hizo
cu nto diablo bigot n... Ya viene Toribio Ortega
subiendo y bajando cerros
y no te enredes ni enga es
que ah anda Pablito Se ez
haciendo ladrar los perros. Cu nto usurero barb n...!
Ay jijos... c mo les vuela
de la levita el fald n...!
Ay jijos... ya se nos hizo:
triunf la revoluci n...! Tenemos camino andado...
No hay que juntarse con rotos
siempre te juegan traici n
ya Madero est vengado
ya muri la usurpaci n. En su caballo retinto
lleg Emiliano Zapata
bonita su silla charra
y sus botones de plata
pero mucho m s bonito
su famoso Plan de Ayala... Este gallo es de navaja
y no es gallo de espol n
si quieres tierra trabaja
trabaja no seas huev n... Ya lleg don Venustiano
con sus anteojos oscuros
y Villa y Zapata gritan:
No s que tengo en los ojos...
porque ya en Pablo Gonz lez
se vislumbra la traici n
Ay reata no te revientes
que es el ltimo jal n...! ya se est n muriendo todos
Jes s qu desilusi n...!
se est volviendo gobierno
Ay dios...! La revoluci n.

It is often argued that companies have to work with the FOSS community and there are good reasons for doing so. I've tried to collect a number of case stories of good and bad community interaction that may help as a starting point for further exploration of this topic:

• NVidia is probably the best example of a company that is not interested in providing open source drivers for their graphics chips. There are lots of folks who won't buy hardware with NVidia chips for this reason. Intel is the number 1 choice at the moment because they work closely with the community. Related to this, the Linux Foundation and kernel community released a statement regarding open drivers.
• When Sun finally made Java available as open source, they argued that it would help inclusion and distribution of Java (e.g. inclusion into Debian). Sun also felt that the GPL provides the strongest protection "against misbehaviour by monopolists in the Java platform".
• The Novell/Microsoft deal was not perceived well by the community, which led to web sites like http://boycottnovell.com
• Nokia recently had an insightful blog posting about the problems when nobody knows about the open source contributions you make: for example, you cannot influence decisions in a project as easily.
• Sun is often criticized for dominating all open source projects they run. They can be used as a good example that you have to give up some control in order to get outside participation.
• The fact that Xen is not in the mainline kernel is a key reason why a number of companies (e.g. IBM, Red Hat) are looking at KVM (which is in the mainline kernel and actively developed in the community). In fact, "looking" is probably not the best word. IBM is actively working on KVM to make it a viable solution and Red Hat recently acquired Qumranet, the company behind KVM.
• Oracle got fairly bad press for copying the Red Hat distribution and changing Red Hat to Oracle, without engaging in the Red Hat community.
• There's a paper that looks at "empirical evidence on the incentives of firms that engage in OS activities." They look at 146 Italian companies supplying open source solutions, but unfortunately the study doesn't include any major company.

Can you think of other examples? (Originally published on FOSSBazaar where comments are possible)

My current desktop PC has an RS790-based Radeon on-board graphics controller. It also has a Radeon X1900 plugged in. Playing with my Watts Up, I found that the system was (at idle!) drawing around 35W more power with the X1900 than with the on-board graphics.

This is clearly less than ideal.

Recent Radeons all support dynamic clock gating, a technology where the clocks to various bits of the chip are turned off when not in use. Unfortunately it seems that this is generally already enabled by the BIOS on most hardware, so playing with that didn't give me any power savings. Next I looked at Powerplay, the AMD technology for reducing clocks and voltages. It turns out that my desktop hardware doesn't provide any Powerplay tables, so no joy there either. What next?

Radeons all carry a ROM containing a bunch of tables and scripts written in a straightforward bytecode language called Atom. The idea is that OS-specific drivers can call the Atom tables to perform tasks that are hardware dependent, even without knowledge of the specific low-level nature of the hardware they're driving. You can use Atom to do several things, from card initialisation through mode setting to (crucially) setting the clock frequencies. Jerome Glisse wrote a small utility called Atomtools that lets you execute Atom scripts and set the core and RAM frequencies. Playing with this showed that it was possible to save the best part of 5W by underclocking the graphics core, and about the same again by reducing the memory clock. A total saving of 9-10W was pretty significant.

The main problem with reducing the memory clock was that doing it while the screen is being scanned out results in memory corruption, showing up as big ugly graphical artifacts on the screen. I'm a fan of doing power management as aggressively as possible, which means reclocking the memory whenever the system is idle. Turning the screen off to reclock the memory would avoid the graphical corruption but introduce irritating flicker, so that wasn't really an option. The next plan was to synchronise the memory reclocking to the vertical refresh interval, the period of time between the bottom of a frame and the top of the next frame being drawn. Unfortunately setting the memory frequency took somewhere between 2 and 20 milliseconds, far too long to finish inside that time period.

So. Just using Atom was clearly not going to be possible. The next step was to try writing the registers directly. Looking at the R500 register documentation showed that the MPLL_FUNC_CNTL register contained the PLL dividers for the memory clock. Simply smacking a new value in here would allow changing the frequency of the memory clock with a single register write. It even worked. Almost. I could change the frequency within small ranges, but going any further resulted in increasingly severe graphical corruption. Unlike the sort I got with the Atom approach to changing the frequency, this corruption manifested itself as a range of effects from shimmering on the screen down to blocks of image gradually disappearing in an impressively trippy (though somewhat disturbing) way.

Next step was to perform a register dump before and after changing the frequencies via Atom, and compare them to the registers I was programming. MC_ARB_RATIO_CLK_SEQ was consistently different, which is where things got interesting. The AMD docs helpfully describe this register as "Magic field, please use the excel programming guide. Sets the hclk/sclk ratio in the arbiter", about as helpful as being told that the register contents are defined by careful examination of a series of butterflies kept somewhere in Taiwan. Now what?

Back to Atomtools. Enabling debugging let me watch a dump of the Atom script as it ran. The relevant part of the dump is here. The most significant point was:

MOVE_REG @ 0xBC09
src: ID[0x0000+B39E].[31:0] -> 0xFF7FFF7F
dst: REG[0xFE16].[31:0] <- 0xFF7FFF7F

, showing that the value in question was being read out of a table in the video BIOS (ID[0x0000+B39E] indicating the base of the ROM plus 0xB39E). Looking further back showed that WS[0x40] contained a number that was used as an index into the table. Grepping the header files gave 0x40 as ATOM_WS_QUOTIENT, containing the quotient of a division operation immediately beforehand. Working back from there showed that the value was derived from a formula involving the divider frequencies of the memory PLL and the source PLL. Reimplementing that was trivial, and now I could program the same register values. Hurrah!

It didn't work, of course. These things never do. It looked like modifying this value didn't actually do anything unless the memory controller was reinitialised. Looking through the Atom dump showed that this was achieved by calling the MemoryDeviceInit script. Reimplementing this from scratch was one option, but it had a bunch of branches and frankly I'm lazy and that's why I work on this Linux stuff rather than getting a proper job. This particular script was fast, so there was no real reason to do it by hand instead of just using the interpreter. Timing showed that doing so could easily be done within the vblank interval. This time, it even worked.

I've done a proof of concept that involved wedging this into the Radeon DRM code with extreme prejudice, but it needs some rework. However, it demonstrates that it's possible to downclock the memory whenever the screen is idle without there being any observable screen flicker. Combine that with GPU downclocking and we can save about 10W without any noticable degradation in performance or output. Victory!

I gave the code to someone with an X1300 and it promptly corrupted their screen and locked their machine up. Oh well. Turns out that they have a different memory controller or some such madness.

So, obviously, there's more work to be done on this. I've put some test code here. It's a small program that should be run as root. It should reprogram an Atom-based discrete graphics card[1] to half its memory clock. Running it again will halve it again. I don't recommend doing that. You'll need to reboot to get the full clock back. This isn't vblank synced, so it may introduce some graphical corruption. If the corruption is static (ie, isn't moving or flickering) then that's fine. If it's moving then I (and/or the docs) suck and there's still work to be done. If your machine hangs then I'm interested in knowing what hardware you have and may have some further debugging code to be run. Unless you have an X1300, in which case it's known to break and what were you thinking running this code you crazy mad fool.

Once this is stable it shouldn't take long to integrate it into the DRM and X layers. I'm also trying to get hold of some mobile AMD hardware to test what impact we can have on laptops.

[1] Shockingly enough, it's somewhat harder to underclock graphics memory on a shared memory system

There are a lot of companies out there that do web design work that looks fabulous. Unfortunately, a lot of these sites look fabulous only when viewed in IE6 build xxxx, with a 75dpi monitor, fonts set to the expected size, running on Windows XP SP2, with JavaScript enabled. Try looking at the site through Safari, Firefox, with larger-than-expected fonts, and things break down: text boxes overlap each other, buttons that should work don’t, and it becomes a mess. So, if your employer wanted a web design company that has a good grasp of Web standards and the appropriate use of them, where would you look? A company that can write good HTML, CSS, and JavaScript, and still make the site look appealing? A company that has heard of Apache and gets the appropriate nausea when someone mentions ColdFusion or Frontpage? So far, I’ve seen these places mentioned by others: WebDevStudios.com
Happy Cog
Crowd Favorite

Currently there is a debate about censoring the Internet in Australia. Although debate might not be the correct word for a dispute where one party provides no facts and refuses to talk to any experts (Senator Conroy persistently refuses all requests to talk to anyone who knows anything about the technology or to have his office address any such questions). The failures of the technology are obvious to anyone who has worked with computers, here is an article in the Sydney Morning Herald about it [1] (one of many similar articles in the MSM). I don’t plan to mention the technological failures again because I believe that the only people who read my blog and don’t understand the technology are a small number of my relatives - I gave up on teaching my parents about IP protocols a long time ago. One of the fundamental problems with the current censorship idea is that they don’t seem to have decided what they want to filter and who they want to filter it from. The actions taken to stop pedophiles from exchanging files are quite different from what would be taken to stop children accidentally accessing porn on the net. I get the impression that they just want censorship and will say whatever they think will impress people. I have previously written about the safety issues related to mobile phones [2]. In that document I raised the issue of teenagers making their own porn (including videos of sexual assault). About four months after writing it a DVD movie was produced showing a gang of teenagers sexually assaulting a girl (they sold copies at their school). It seems that the incidence of teenagers making porn using mobile phones is only going to increase, while no-one has any plans to address the problem. The blog www.somebodythinkofthechildren.com has some interesting information on this issue. Two final reasons for opposing net censorship have been provided by the Sydney Anglicans [3]. They are:

1. Given anti-vilification laws, could religious content be deemed illegal and be filtered out? Could Sydneyanglicans.net be blocked as illegal if it carries material deemed at some point now or in the future as vilifying other religions? If it s illegal in Vic say, and there isn t state-based filtering (there wont be), will the govt be inclined to ban it nation wide?


2. Given anti-discrimination laws, if Sydneyanglicans.net runs an article with the orthodox line on homosexuality, will that be deemed illegal, and the site blocked? You can imagine it wouldn t be too hard for someone to lobby Labor via the Greens, for instance.

So the Sydney Anglicans seem afraid that their religious rights to discriminate against others (seriously - religious organisations do have such rights) will be under threat if filtering is imposed. I was a bit surprised when I saw this article, the Anglican church in Melbourne seems reasonably liberal and I had expected the Anglican church in the rest of Australia to be similar. But according to this article Peter Jensen (Sydney’s Anglican Archbishop) regards himself as one of the “true keepers of the authority of the Bible” [4]. It seems that the Anglican church is splitting over the issues related to the treatment of homosexuals and women (Peter believes that women should not be appointed to leadership positions in the church to avoid “disenfranchising” men who can’t accept them [5]). It will be interesting to see the fundamentalist Christians who want to protect their current legal rights to vilify other religions and discriminate against people on the basis of gender and sexual preference fighting the other fundamentalist Christians who want to prevent anyone from seeing porn. But not as interesting as it will be if the Anglican church finally splits and then has a fight over who owns the cathedrals. ;) A comment on my previous post about the national cost of slow net access suggests that Germany (where my blog is now hosted) has better protections for individual freedom than most countries [6]. If you want unrestricted net access then it is worth considering the options for running a VPN to another country (I have previously written a brief description of how to set up a basic OpenVPN link [7]).

• [1] http://www.smh.com.au/articles/2008/11/11/1226318639085.html?page=fullpage


• [2] http://doc.coker.com.au/safety/mobile-phone-safety/


• [3] http://www.sydneyanglicans.net/forums/viewthread/3673/


• [4] http://www.smh.com.au/articles/2008/06/22/1214073053814.html


• [5] http://www.abc.net.au/worldtoday/content/2004/s1213397.htm


• [6] http://etbe.coker.com.au/2008/11/17/national-cost-slow-net-access/#comment-16729


• [7] http://etbe.coker.com.au/2008/05/24/ipsec-is-pain/

I love collecting quotes but I file most of them away in some file on my computer or distribute them in class. I thought I might start sharing some of my favorite passages/quotes. This one, which I read 3 years ago, reminded me of the craft-like spirit in Debian and Free Software in general. I like how the author brings together the themes of labor, time, quality, mistakes, intimacy and tools all in one swoop.

I refuse to be in such a hurry that I squeeze the aesthetic value out of everything just to gain a few minutes of time time which will then just be filled with more rushing and more mass-produced, soulless junk. In the drive to achieve instant gratification, we have spent a century trying to shorten the learning curve and eliminate the chance of error in every human activity. There is much good in this, but something has been lost in the process. The Galoots are the guardians of that which was almost lost: the challenge of trying to master a skill that can never be fully mastered, the creative freedom that comes from intimacy with a medium as complex as wood, the sense of self-sufficiency that comes with knowing that you can make a useful object with tools so simple that you can make the tools too, and the peaceful mediation of trying to bring eye, hand, and wood together into harmony through finesse and understanding rather than brute force.

Quoted from Hohn (2005), The Romance with Rust, a Harper’s article on tool collectors. A Galoot is a person with a deep passion for old hand tools, their ways, workings, and their history.