Here's a quick HOWTO for using Wine to play Starcraft on a Linux machine.

  $ apt-get install wine (as root)
  $ winecfg

The winecfg (graphical) utility will setup some config file defaults in your ~/.wine directory. Click on Graphics and activate Allow DirectX apps to stop the mouse leaving their window. Also, click on Audio (a dialog will pop up, just click OK). This will autodect your soundcard and setup Wine to use it. Under Drives click Add (this will add D:) and change the path to /media/cdrom, so that Wine knows about your CD-ROM drive. Finally click OK to close winecfg and save the settings. The next step is to insert the Starcraft CD-ROM into the drive and start the installer using Wine:

  $ mount /media/cdrom (as root)
  $ wine /media/cdrom/setup.exe

Follow the instructions in the installer until the Starcraft install is finished (you'll need your CD key number), then exit the installer (don't start playing Starcraft right away). The next step is to get the latest patch and get rid of the need to insert the CD-ROM every time.

  $ wget http://ftp.blizzard.com/pub/starcraft/patches/PC/SC-116.exe
  $ wine SC-116.exe

After the patch is installed click OK and Starcraft will be started (very annoying). Leave the game again. We'll get rid of the CD-ROM requirement now:

  $ cp /media/cdrom/install.exe ~/.wine/drive_c/Programme/Starcraft/StarCraft.mpq

That's a pretty big file, it may take a while. You might have to change "Programme" in the path (I have the German Starcraft version). That's it. You can now play Starcraft (without needing the CD-ROM) using:

  $ wine ~/.wine/drive_c/Programme/Starcraft/StarCraft.exe

A good thing is, it even works nice and fast with the open-source nv NVIDIA driver (no need to install the proprietary driver). I noticed one very annoying "bug" with the mouse behaviour at first. The mouse would sometimes just get stuck during the game (which is a total disaster of course, if you're in the middle of a fast-paced game). Left-clicking somewhere would "unstuck" the mouse, but it's still very bad. After many, many hours of reading bugreports and trying various patches I finally found out the root cause for the problem. It's somehow related to my window manager (IceWM); whenever you move the mouse to the bottom of the Starcraft screen (where the IceWM status bar is, even though it's not on top or even visible, and even though Wine/Starcraft runs in full-screen mode!), something funny happens with X11/IceWM and the mouse gets stuck. I haven't yet found out if/which IceWM option could fix this behavior, but I have a small work-around. Just start Wine directly on a second X11 server with Starcraft (without any window manager being involved):

  $ xinit -e '/usr/bin/wine ~/.wine/drive_c/Programme/Starcraft/StarCraft.exe' -- :1

No patches needed (stock Wine from Debian unstable works fine, that's version 1.0.1 right now). I hope this saves other people some debugging time...

Once again, a person died today murdered by terrorists in Spain. He was a businessman, who had a construction company in his town (Azpeitia) at Basque Country region. Close his house, he was murdered by gunshots. Just because his company was a contractor in the construction of AVE infraestructure in Basque Country, which will connect it with Madrid and France. Once again, a family has been hit forever. I was going to include a black strip with the article, but it’s better to see an image.

Some of you might be wondering where I ve gone (my last post was on 2008-11-18), but I have a short explanation for that: I got married! Yes It was on November 22nd, in a non-religious ceremony among the majority of my close friends and family. Brenda and I are together for more than 6 years already, so it was about time! After the party we went on honey moon to Bariloche (Argentina), undoubtfully one of the most beautiful parts of the planet, for one week. We were back just Monday and back to real life just now. Pictures of the party and the trip will follow

Today it made clear to me why Garvin Hicking is called "SuperGarv". I've had a little problem with my s9y installation and asked Garvin by mail, because I know him since good old Grafik.Ger times in FidoNet.
The problem has been described in #502954 as well: you can create directories in the Media Manager, but you can't see or use them afterwards, although they are there in the filesystem.

Usually, one could have solved this by yourself, but it seemed more apparent that the application is buggy. Garvin, instead, got the idea pretty soon:

Hast du vielleicht nur Schreib- aber keine Leserechte f r die Verzeichnisse?
Translation: "Do you have just write but no read permissions for those directories?

It's in fact somewhat unusual that you can create directories, but cannot use them afterwards. Especially when you see them on disk, you are likely to suspect a problem in the application itself, not on the disk.
But Garvin was right:

#ls -l /usr/share/serendipity/www/
...
lrwxrwxrwx 1 root root 28 2008-12-02 09:18 uploads -> /var/lib/serendipity/uploads
...

# ls -l /var/lib/serendipity/
total 4
drwx-wxr-x 2 root www-data 6 2006-10-18 16:10 archives
-rw-rw---- 1 root www-data 0 2008-12-02 09:19 htaccess
drwx-wxr-x 13 root www-data 4096 2008-12-02 07:48 uploads
root@vserv1:/usr/local/drupal6/modules# chmod g+r /var/lib/serendipity/uploads/

I've checked with Adrian von Bidder (#502954 submitter) on IRC and this seems to be a bug in the Debian package indeed.

So, thanks to Garvin for the hinting and Adrian for the bugreport and checking on IRC! :-)

A brief distraction from moving - I find it fascinating that people should vaguely think that a coalition appealing to the Governor General is undemocratic. As far as I can tell, this comes from watching too much American TV. As Canadians, we don't elect the government or the PM. We elect MPs to fill seats, and the GG chooses the government based on the available people who can form the most stable government.

This bit is important! The MP you voted for is *still* your MP. Now the people elected to run the country have to try and figure which group represents the best chance of running smooth government. For those of us not around to remember the last time this happened (1926, Lord Bynd as GG), it's perfectly reasonable for the GG to ask the opposition to try and form a working government.

By some measure, this is extraordinary, but I think of going for another election 7 weeks after the previous one to be the equivalent of seeking marriage counseling 7 weeks after starting dating someone. As in, it's perfectly okay to write it off and try the other available options.

Time to get some sleep for our first night in the new appartment. =)

The title of this post was a headline that was probably written by an overly sensationalistic editor at http://www.searchenterpriselinux.com. The actual article, though, was written by Pam Derringer, was a pretty balanced piece (although it obviously could have been more in-depth; given length and time constraints, she talked to as many people as I think could have been expected, including a Sun spokeswoman, two analysts from different analyst companies, the chair of Apache Software Foundation, a Sun customer, and so on). I’m not sure I’d agree with Jim Jagielski when he claims that Sun has the “best and brightest business people in the world,” though. As I said in the article, and as I’ve said before, I have nothing but the highest respect for many of Sun’s engineers, especially those who work on Solaris. But their business people? I’m not so sure they have the best and brightest… in fact, I think that’s one of Sun’s greatest problems right now. Compare that with the job that Mark Hurd has done with HP, or with the work done by the executives at my employer (IBM), and I’m not so sure a strong case can be made for the excellence of Sun’s business folk. (I mean, how many people still believe that the $2 billion dollars which Sun spent to acquire Cobalt Networks, or the $4 billion dollars which Sun spent to acquire StorageTek, or the $1 billion dollars which Sun spent MySQL, were all good ideas? Sun is currently valued by the market at $2.4 billion dollars!) My bigger concern at the moment, however, is what happens if Sun gets purchased by a rival or by a corporate raider and gets broken up for parts? Since November 19th, when Sun’s stock price crashed through $3.27 (and has hit a low of $2.60 recently), it has been trading for less than its per-stock cash value. I say that not out of a sense of Schadenfreude, but out of concern about what happens if someone decides to buy the company, lay everyone off, and even after paying all of its debts, the resulting cash left over would be more than cost of completely buying the company — all of the real estate, the inventory, and the patent portfolio would all be pure profit. It’s the last I’m most concerned about; Sun has a very large patent portfolio! What if it fell into the hands of a patent troll? What damage could be done to not just Linux and the OSS comunity, but the software industry as a whole? Suppose Sun really does have patents that read upon many of NetApp’s products? If those patents fall into the hands of a patent troll, NetApp would no longer be able to negotiate a peace settlement based on its patents (if they survive the on-going patent re-examination process) that read on ZFS, since the patent troll would no longer be making any products based on ZFS. Maybe somewhere in Sun’s patent portfolio, there are patents that read on the GNOME and KDE desktops, but since Sun is trying to ship those with its Project Indiana/Open Solaris dekstop, it’s not bothering to assert them at this time? What if a patent troll, perhaps one funded by Microsoft, gets their hands on those patents? One final scary thought. Sun’s current market cap: $2.40 billion USD. Sun’s cash on hand: $2.63 billion USD. Microsoft’s current cash on hand: $19.71 billion USD. I can just imagine Steve Ballmer saying, “Chris, take it out of petty cash. I have an itsy-bitsy little acquistion I want to make…. Now I can embrace and extend Java all I want! And you know those term-limited, non-Open Source TCK licenses Sun gave to OpenJDK developers — not any more! They’re gone, within a year, the next time Red Hat needs to get the license renewed!” Could it happen? What do you think?

I got an iPhone on the very first day it was available officially in my country. I have used it every day since then and am now ready to decide how good it really is. Summary: it is a great piece of technology and the only tradeback is the locked down platform. I have abandoned my Palm and am not looking back. Hardware: the hardware is just perfect. 4 buttons (Home, Power, Volume Up and Down) and 1 switch (Mute) are all very clear, easy to find and press and frankly are enough. The touchscreen is reasonably fast and very precise nicely corrected by software that guides your presses toward UI points of interest if such are within your finger. The casing is absolutely solid - there is no gaps, no wobble, nothing. All glossy surfaces attract fingerprints, but a couple passes with a microfiber cloth (included) clean it up easily. The loudspeaker is phenomenal - all sound comes from a single small hole, but it can easily fill a small room with music. Call sound quality is impeccable. You can plug in any headphones, but if you use the included headphones you get a microphone with a button that can pickup and hand up calls, start/pause music and skip to the next track. I also love that the only charger included is a very small power brick that provides a power-only USB port and a USB charging cable. That is a good way to reduce number of power bricks for gadgets. The iPhone 3G (that is what I have) also has a GPS build-in, but let us be clear - it is not good enough to replace a car GPS, the iPhone GPS is slow (takes several seconds to refresh the location and tens of seconds to find it first time) and the precision is measured in meters or even tens of meters. The WiFi is very good at getting a connection and doing it very fast. Oh and there is an acceptable camera. Out of the box software: the software included surely is better than on any other phone I have seen (including Palm). The UI widgets are nicely written specially for a touchscreen interface - the best example is having click wheels instead of drop down menus. It is much easier mentally to flick a wheel up or down then to click on a list, click to scroll the list and then click to select an entry. And the click wheel makes it much easier to correct off-by-one errors. Switches that look like regular physical switches instead of checkboxes is another very useful UI concept. I think a lot of this can be reused on the desktop as well. iTunes: and here comes the bad spot, the lock-in part. To get anywhere with the iPhone you need to associate it with an installation of iTunes. It fails to work in Wine or even in a virtual machine - apparently there are some errors related to avahi or something. Unfortunately, this means that you need wither Windows or MacOS to do nything with your iPhone. iTunes is the only way to backup, restore and upgrade your iPhone, it is the only way to get contacts and music on the iPhone (there is an iTunes Store app in the newest version that allows purchasing music over the air). Even after doing a jailbreak (see further) it is currently not possible to add music to the iPhone from any third party applications, because the iTunes database has a hashed checksum that the build-in iPhone music player (and iTunes) check before doing anything. If the checksum is wrong the only thing you can do is a full restore of the phone from backup. Crypto geeks - please help decrypt that hash, so that we can sync iPhones to amaroK. The iTunes itself is a bit weird - to get music onto the phone you need to create a playlist and then you can sync that playlist over. One of the playlist options is a smart playlist with a random feature and a limit on number of matches. That is the way to get random music from your collection onto the phone. I do dislike the fact that iTunes syncs all the contacts from Google to the phone and not only contacts for a certain group or only those that have a phone number, for example. Search and favorites do help here. Third party software (iTunes Store): it is very easy to install new software on the phone over the WiFi and there is plenty to choose from. You do need a Mac to develop this software which restricts the developer base a bit. The main bases that I needed (or did not know I needed) are covered: Last.fm player, TwitterFon, LockBox password storage, Stanza ebook reader, Shazam music identificator, Wordpress client, BigOven cookbook and lots of fun games. Have not used any paid-for apps due to the fact that iTunes store is not available in Latvia and the official documentation recommends registering with a fake address in UK or US. Most of the software is rather stable and useful, but there is also a lot of fluff. User reviews and ratings help a lot, can we have those for Debian packages? Jailbreak and third party software (Cydia): thanks to some anonymous hackers, we have the ability to break the software jail on the iPhone and install software that Apple did not approve. The process is made very simple, but again it does require you to boot into Windows or MacOS X. The main thing the jailbreak process does is install an installer on the phone. Currently the best one is Cydia which is built on top of … apt-get and dpkg! After that is done, you can install a terminal, a ssh server and BossPrefs advanced preferences. There are also other interesting programs there - a different dock, file manager, caching youtube player, quake, themed app launcher with categories and startup page (winterboard). Theoretically the ssh server allows you to manage your music over WiFi from any platform, but unfortunately until the iTuned database hash algorithm is decoded, we can only copy music from the phone to the PC and not back. It is possible to copy files over using ssh and then play them using third-party players, but those players are very crude and featureless. To summarize: an iPhone is a great phone and a decent application platform. To make using an iPhone with Linux easier we need to make iTunes run in Wine (and allow it to see the iPhone) and reverse engineer the iTunes database hashing algorithm so that we can get music on our phones with Linux tools. I also wish there was a way to develop iPhone apps in Linux, but that really is reaching for the stars.

In light of Ted’s post on copyright, it is clear that we are bogged down by a hostile terminology. Copyright infringement is not piracy - gunmen on seas killing people, looting ships and holding hostages is piracy. Copyright infringement is not theft. When my bike was stolen, I no longer had it. Stealing a bicycle is theft. Copyright infringement is more like sneaking into a concert. You enjoy the show, but did not pay for it. If the concert is held in a small venue then it is easy to spot someone who gets in without paying, but if you hold a concert in an open field, then expecting everyone in the surrounding area to be forced to pay you is rather absurd. If you have a concert in a field with a fence around it and someone makes a hole in that fence (posts a a recording to a P2P site), then the one making the hole might be punishable (property damage), but can any person that enters that hole really be prosecuted for theft, property damage and enabling further theft by not closing the hole (uploading that is inherent in the P2P protocols) ? I do not think so. A similar analogy is using public transport without a ticket. If there is a bus that goes from A to B and I get on it without buying a ticket from the driver, then in the current copyright enforcement world I would get arrested by the police, prosecuted and get a fine that is tens of thousand times more than the price of the ticket. This has multiple problems - 1. the police has no way of knowing if I have some kind of legal right to use the bus without a ticket (monthly ticket, free ride for seniors, …) before arresting me and bringing me to court, the police has no business wasting their time and taxpayer money until it is 100% certain that a crime has actually occured; 2. if a hacker disables the ticket composters in the bus and removes all signs about prices how am I to know that the bus is not free (like the park and ride buses in many locations); 3. the fines are excessive - I’ve not seen a public transport fine that is much more then 10 times the price of the ticket, and it only applies to one ride - you can not be retroactivelly fined for all free rides you took in the last year; 4. it actually is not possible to be 100% certain who is the person doing the act - you can only trace the IP which can be used by any number of computers and additionally the computers might be infected with a botnet acting as an unwilling proxy zombie. There is no way (except a confession) to prove that a particular person does a particular download. Now we just need a short and simple word or phrase that describes that. Any ideas?

This weekend I had the pleasure of visiting my friend Kike in Iru a, a city I really like but had not been able to visit in five years. I spent three days with him, after a really awful Bilman trip during Friday night. The first thing that happened on Friday morning was quite unexpected. I went out, crossed a pair of streets to get to Carlos III, and going past the corner I found myself surrounded by all kinds of policemen: red, blue, green and yellow. There were press reporters all over the place, with TV and photo cameras ready to record something big. What the hell? I looked around, and there it was: a huge blue sign read Populares de Navarra, and Mariano Rajoy was seconds away from getting out of the new PP headquarters in Nafarroa. With a few dozen policemen looking at me in suspicion, and realising my current hair dress wasn't the most appropriate for that scenario, I decided to disappear as quickly as possible. Populares aside, the weekend was really productive. I had a great time trying to find the places I visited 9 and 5 years ago. It was sad to see the fantastic Iru ako Gaztetxea turned into an iron building surrounded by old, traditional houses in the centre of the city. With Kike, I had enough time to learn more about the origins of Iru a and the three cities from which it originated, and visit the Ciudadela. On Saturday, Kike, Ana and I went to have some fun on a snowy day around Orreaga and had dinner in a small town around the area, maybe Lintzoain? I'm afraid I forgot due to the mix of basque names in my head. We had dinner in lo viejo, where I spotted a poster of Solidari@s con Itoiz for the Itoitz hustu arte campaign, a copy of which I unsuccessfully tried to get in several places in the city. On Sunday, Kike and I took a bus to Donosti, and I travelled across the most beautiful highway I've seen. But of course, that was from my perspective seated in a bus. I can imagine the Leitzaran highway project must have been greatly contested by the people of the tiny towns nearby. The spoiled valleys and views must have been really impressive in the past and seem now irreversibly ruined by a scar and holes through the mountains. This was my first visit to Donosti, which held its annual Marathon, and were lucky to meet our friend Rub n when he was around kilometre 25 of his race. He managed to finish under 2:50, which was a bit better than what he aimed for in his first Marathon. Well done! The weather was horrible, and our visit to La Concha, the old harbour and the Casco Viejo was short, we were freezing and getting our feet wet. While I visited Donosti for the first time, Mikel Laboa, one of the most respected singers in the basque culture, was ebbing away. Laboa's songs always give me goosebumps, even if I need to read a translation for the basque lyrics. Many will remember his music from Julio Medem's documentary, La pelota vasca, featuring Txoria txori, which has become a symbol of basque culture over the years. Youtube has quite a few videos, and I'd recommend watching Txoria txori, Gure bazterrak, Lili bat or Baga biga higa performed by the Orfe n Donostiarra to name a few. Mikel Laboa, goian bego.

When I have a moment, I’ll try to tally up the responses that I got to “An ethical question involving ebooks”and see if there are any interesting patterns based on self-identified generational markers. Obviously, this is not a properly controlled survey, so the results aren’t going to mean much, but it is interesting that some fairly passionately written comments came from folks who self-identified as coming from generations that broke with the common stereotypes of their respective demographic groups. If I were going to commission a study, one thing that I would almost certainly do is to ask pose a similar question about music and mp3’s, and do have the surveys asking the question about ebooks first, and half the surveys asking the questions about music first. It would be interesting to see if (a) there is a difference in attitudes between music and books, and (b) whether the order of the questions might influence the answers or not. A number of poeple have asked me about the author’s name and the title of the books/series involved. I deliberately didn’t include that information, for a number of reasons. First of all, I don’t believe idenifying the author/books/character involved is relevant to the question at hand, and in fact, might be distracting. Secondly, given the many comments, some of them quite passionate, I don’t think it would be fair to drag her name into the discussion without her permission first. I will say that the author does have a fairly extensive internet presence, and has apparently gotten a lot of questions about said character, and in fact whether those books would be made into ebooks. It’s been made quite clear that while those books were successful, they weren’t that successful, and so from an economic point of view, she chooses to write books that she (and her publishers) feel will be more economically viable. Because there will likely be no further books published containing this character, it is very unlikely that the publisher will reprint the original series of books — and when asked about whether they would be made available in ebook form, her response was effectively “it’s up to the publisher”, Apparently she has worked with a number of publishers, and while publisher X hasn’t been willing to publish her books in ebook form, publisher Y has. Furthermore, it seems that her contracts apparently delegate all decisions about how her books will be published, and whether a large Major Big City Law Firm with Fangs (aka MBCLFF) will go after copyright infringers to her publishers and her agent (who is a lawyer at said MBCLFF, and who could presumably inflict major Hurt on copyright infringers that curry the lawyer’s disfavor). I don’t know if this is true, or just her way of managing her relationship with her fans by disclaiming all responsibility about publication forms and enforcement decisions to others — but some authors do make such choices, if they are much more interested in the writing and storytelling end of things than the business side of things. Which brings up an interesting question with respect to copyright enforcement. It’s pretty obvious that many people will give different answers to the question relating to how much deference should be given to copyrights depending on whether they are owned by The Struggling Author versus whether they are owned by The Big Media Corporate Monolith, with many more allowances given if the question is framed as being primarily about the former rather than the latter. Another way in which how you frame the question radically changes the outcome depends on whether the focus is on making sure the author (and/or his surviving widow/widower/children) get paid or whether the focus is on control of one’s works. If you believe the primary justification is an economic one, then that leads to a series of ethical conclusions — the most obvious of which is that if it doesn’t result in a direct (or perhaps indirect) monetary loss to the author, there should not be a moral or ethical problem. There might be some question as to whether devaluing the secondary market might discourage the sale of new books, and hence indirectly harm the author sufficiently that this should be a concern, but those issues can be worked out. If however, you believe the primary issue at hand is one of control, a very different set of issues have to get factored into the conversation. For example, what if the author was ashamed of a book or series, and wants it to go quietly out of print, and hopefully disappear. How should that be weighed against fans who disagree with the author and who love the series? What is the right balance? For those who argue that the author’s wishes should be sacrosanct — should we move things more in that direction? What if all texts lived in DRM’ed, encrypted containers, and electronic readers had to ask permission of a central authorization server for the text could be displayed. This would allow the author to, after the fact, disable anyone from reading his or her works, if for some reason the author so desired it. Would that be a good thing? If not — and I hope most authors would agree this would be horrific power to give copyright holders — then it’s clear that author’s moral rights as creators should not be entirely sacrosanct, and that the society also has some claims on preserving its culture, and that once a book has been published and becomes part of the culture, society should have some claim on that book as part of culture. Whether that means that copyright terms should be 14 years or 20 years as opposed to whenever the Disney corporation feels like paying off more legislators to extend copyright terms is one way that question could be asked. Another is whether society should have the right to say that if after some number of years where a work has been abandoned for commercial exploitation, whether it should automatically enter the public domain. There are no obvious answers here. The final point that I want to make, which may be fairly controversial amongst the Open Source programmers in the room, is that if you believe that copyright should be fundamentally be about economic arguments of “no harm, no foul”, that this is in direct contradiction with the belief that lawsuits should be used in order to enforce the GPL. After all, the conditions imposed by the GPL are fundamentally about control, not about economic issues. Consider — if someone uses the Busybox project in an embedded device — especially if no changes has been made to the code — who has been harmed, economically? No harm, no foul, right? Or if someone uses GPLv3 code in a firmware which is protected by a digital signature — sure, it means that end users who want to modify the firmware and then use it to enhace/extend the device won’t be able to do so. But how does that economically harm the author of the GPLv3 code? Fundamentally, Copyleft schemes are all about extending control over how the code can be used. Hence, if you are an Free Software programmer who cheers on the activities of the SFLC, or who firmly believes that no one should be allowed to mix firmware which is not shipped with source with your GPL’ed software, it is completely and profoundly hypocritical to say, “F*ck the author’s wishes; if it’s not available in the from I want, I should be able to make a derived work to transfer the work into a form that I want.” What if the author is a luddite who hates eBooks and firmly believes and wants to enforce that their works should never be made available in eBook form. How is that fundamentally different from a Free Software Acolyte saying that because they abhor non-free firmware, and don’t want allow their code to be shipped alongside binary-only firmware?

This bottle was found in Mika's biosafety level 2 laboratory. To address any confusion, isoamyl alcohol is not drinking alcohol and this bottle was bought for use in a scientific lab from a scientific lab supply company. Explanations are welcome.

Interestingly, the just finished Extremadura QA + ftpmaster + i18n meeting also turned into a quite strong Bug "Smashing" party. Several participants were digging around the list of RC bugs affecting both lenny and unstable. Most often, it turned out to many bug downgrades (for many of which I'll take the blame). It's "funny" to see how sometimes our bug reporters tend to inflate their reports, most often because they feel that "bug should absolutely be fixed", but completely forgetting about the definition of 'serious' or 'grave'. I also feel that, in some way, several maintainers begin to be quite demotivated by the length of the process (which I personnally consider as being caused by several factors: decreasing (wo)manpower, haircutting about freeness of every single byte on Debian CD's or even shyness wrt correct actions needed to deal with bugs). And several of these bugs just needed a very simple action by the maintainer. And, therefore, just because we were about 15 people with a lot of time to devote to Debian for 3 days, we ended up digging in that pile of mud....removing most of the dust and sometimes very debatable RC bug reports, and finally having that count of bugs affecting both lenny and unstable go down again. This comforts me in the feeling that an aggressive approach is a good key to bug squashing. After all, if I'm sometimes too aggressive and inappropriately downgrade an RC bug, I'm sure that someone will correct me (and, anyway, I'm not that mad: sometimes I seek for advices). Ah, and "aggressive" also sometimes mean proposing to tag some bugs "lenny-ignore" (and, that needs to be very convincing with our release managers, believe me). So, what about you? Have you already smashed an RC bug today? If not, do it. Kill those nasty beasts, they deserve it. No mercy.

In a comment on my post about Slicehost, Linode, and scaling up servers [1] it was suggested that there is no real difference between a physical server and a set of slices of a virtual server that takes up all the resources of the machine. The commentator notes that it’s easier to manage a virtual machine. When you have a physical machine running at an ISP server room there are many things that need to be monitored, this includes the temperature at various points inside the case and the operation of various parts (fans and hard disks being two obvious ones). When you run the physical server you have to keep such software running (you maintain the base OS). If the ISP owns the server (which is what you need if the server is in another country) then the ISP staff are the main people to review the output. Having to maintain software that provides data for other people is a standard part of a sys-admin’s job, but when that data determines whether the server will die it is easier if one person manages it all. If you have a Xen DomU that uses all the resources of the machine (well all but the small portion used by the Dom0 and the hypervisor) then a failing hard disk could simply be replaced by the ISP staff who would notify you of the expected duration of the RAID rebuild (which would degrade performance). For more serious failures the data could be migrated to another machine, in the case of predicted failures (such as unexpected temperature increases or the failure of a cooling fan) it is possible to migrate a running Xen DomU to another server. If the server migration is handled well then this can be a significant benefit of virtualisation for an ISP customer. Also Xen apparently supports having RAM for a DomU balloon out to a larger size than was used on boot, I haven’t tested this feature and don’t know how well it works. If it supports ballooning to something larger than the physical size in the original server then it would be possible to migrate a running instance to a machine with more RAM to upgrade it. The question is whether it’s worth the cost. Applications which need exactly the resources of one physical server seem pretty rare to me. Applications which need resources that are considerably smaller than a single modern server are very common, and applications which have to be distributed among multiple servers are not that common (although many of us hope that our projects will become so successful ;). So the question of whether it’s worth the cost is often really whether the overhead of virtualisation will make a single large machine image take more resources than a single server can provide (moving from a single server to multiple servers costs a lot of developer time, and moving to a larger single server exponentially increases the price). There is also an issue of latency, all IO operations can be expected to take slightly longer so even if the CPU is at 10% load and there is a lot of free RAM some client operations will still take longer, but I hope that it wouldn’t be enough to compete with the latency of the Internet - even a hard drive seek is faster than the round trip times I expect for IP packets from most customer machines. VMware has published an interesting benchmark of VMware vs Xen vs native hardware [2]. It appears to have been written in February 2007 and while it’s intent is to show VMware as being better than Xen, in most cases it seems to show them both as being good enough. The tests involved virtualising 32bit Windows systems, this doesn’t seem an unreasonable test as many ISPs are offering 32bit virtual machines as 32bit code tends to use less RAM. One unfortunate thing is that they make no explanation of why “Intger Math” might run at just over 80% native performance on VMware and just under 60% native performance on Xen. The other test results seem to show that for a virtualised Windows OS either VMware or Xen will deliver enough performance (apart from the ones where VMware claims that Xen provides only a tiny fraction of native performance - that’s a misconfiguration that is best ignored). Here is an analysis of the VMware benchmark and the XenSource response (which has disappeared from the net) [3]. The Cambridge Xen people have results showing a single Xen DomU delivering more than 90% native performance on a variety of well known benchmarks [4]. As it seems that in every case we can expect more than 90% native performance from a single DomU and as the case of needing more than 90% native performance is rare it seems that there is no real difference that we should care about when running servers and that the ease of management outweighs the small performance benefit from using native hardware. Now it appears that Slicehost [5] caters to people who desire this type of management. Their virtual server plans have RAM going in all powers of two from 256M to 8G, and then they have 15.5G - which seems to imply that they are using physical servers with 16G of RAM and that 15.5G is all that is left after the Xen hypervisor and the Dom0 have taken some. One possible disadvantage of this is that if you want all the CPU power of a server but not so much RAM (or the other way around) then the Slicehost 15.5G plan might involve more hardware being assigned to you than you really need. But given the economies of scale involved in purchasing and managing the large number of servers that Slicehost is running it might cost them more to run a machine with 8G of RAM as a special order than to buy their standard 16G machine. Other virtual hosting companies such as Gandi and Linode clearly describe that they don’t support a single instance taking all the resources of the machine (1/4 and 1/5 of a machine respectively are the maximums). I wonder if they are limiting the size of virtual machines to avoid the possibility of needing to shuffle virtual machines when migrating a running virtual machine. One significant benefit of having a physical machine over renting a collection of DomUs is the ability to run virtual machines as you desire. I prefer to have a set of DomUs on the same physical server so that if one DomU is running slowly then I have the option to optimise other DomUs to free up some capacity. I can change the amounts of RAM and the number of virtual CPUs allocated to each DomU as needed. I am not aware of anyone giving me the option to rent all the capacity of a single server in the form of managed DomUs and then assign the amounts of RAM, disk, and CPU capacity to them as I wish. If Slicehost offered such a deal then one of my clients would probably rent a Slicehost server for this purpose as soon as their current contract runs out. It seems that there is a lot of potential to provide significant new features for virtual hosting. I expect that someone will start offering these things in the near future. I will advise my clients to try and avoid signing any long-term contracts (where long means one year in the context of hosting) so that they keep their options open for future offers.

• [1] http://etbe.coker.com.au/2008/11/27/slicehost-vs-linode/


• [2] http://www.vmware.com/pdf/hypervisor_performance.pdf


• [3] http://www.cmg.org/measureit/issues/mit41/m_41_1.html


• [4] http://www.cl.cam.ac.uk/research/srg/netos/xen/performance.html


• [5] http://www.slicehost.com/

Of course everybody in our crowd noticed the huge "Wifi zone powered by Debian" sticker with a Deban swirl at the restaurant door where we had dinner :-) And the Debian wallpaper among the other images on the screensaver on the restaurants computer was probably also noticed by many. What really freaked me out though (in a very positive sense) was that the owner/someone (? I have no idea, I talked with him, but not about restaurant ownership...) wrote the billing programm himself. When shown the features it looked to me like a good one, with a simple and nice looking interface and appearantly feature complete to manage the restaurant (including creditcard interface etc.). The cashier computer was running Debian and the app is written in gambas2, which is a BASIC interpreter with build-in database abstraction layers and supporting gtk and QT. And he plans to release this as free software. Yay. Yay Yay Yay!

Me looks forward to this to happen and also to get the .svg of the sticker

Six months ago I investigated the options for Xen virtual servers [1]. I ended up receiving an offer of free hosting and not needing that, but the research was useful. There is a good range of options for Xen servers with different amounts of CPU power, RAM, bandwidth, and disk space. There are a couple of things that seem to be missing, options to upgrade from virtual servers to physical servers, and information on dedicated disk and database performance - but I’ll explain that later after some history. About a week ago a client needed a Xen virtual server in a hurry, their main server (a Xen system that I run on hardware that they rent) was getting a bit overloaded and needed to have one of the largest DomUs moved off. I ended up recommending Linode [2] based on my research and comments I received. The Linode server is working quite well and the client is happy, one nice feature of Linode is the choice of server rooms that they offer. I was able to choose a room in the same region as the other servers that the client owns and thus get ping times that are sometimes less than 2ms! Due to a missing feature in a program that I’m maintaining for the client a large number of MySQL queries are being made. Due to a problem I’m having with MySQL it won’t let me create a slave database server so all the queries go over the VPN and use a large amount of data. This combined with the other traffic that should be going over that link means that about 600G per month is being used, fortunately that is rather cheap. Linode staff handled this very well, after the server had exceeded it’s quota by 120G they asked my client to confirm that the traffic was legitimate and then suggested an upgrade to a plan that could handle the traffic (which went smoothly). Now I have another week to add the feature in question before I meet the quota again. Shortly after getting the new virtual server running at full capacity David Welton wrote a detailed review of Linode and Slicehost for the issues that matter to his use [3]. His conclusion seems strongly in favor of Linode. But now I am looking at getting a Slicehost [4] virtual server for the same client (for a different project) because Slicehost is owned by Rackspace [5], and the new project if successful will need a set of powerful servers and Rackspace seems like a reasonable company to host that. The problem with Rackspace is that they (and every other ISP I’ve researched so far) seems to offer little in regard to customers who need serious disk IO. I am planning some servers that will have a write bottleneck on a MySQL database (or maybe multiple shards), so serious disk capacity is needed. At least I would like to be able to get disk storage by the tray (12-14 disks) with the controllers having RAID-6 support. Rackspace only offers RAID-5 (according to the “livechat” person), and we didn’t get as far as discussing how to add more trays. What would be ideal is if there was an ISP that had both virtual servers and physical servers (so I could start with a virtual server and move to a physical server when things are working well), and also serious storage options. They would offer internal disks, external RAID arrays, and NetApp Filers [6] (or some equivalent device). It would be really nice if I could just instruct the ISP to add another NetApp Filer to my back-end network and have it done for me (I’m certainly not going to visit the US to install new hardware). It’s been over a day since I submitted a sales request to NetApp asking whether they partner with any ISPs and I haven’t received a response. OpenSolaris with ZFS also sounds good for disk IO performance (they have similar features to NetApp). Unfortunately the support for OpenSolaris among ISPs is not that great (while everyone offers Linux and Windows), and I haven’t used any recent version of Solaris. So using OpenSolaris would require finding someone with the skills to manage it who can work for my client - as opposed to a NetApp device that would be just like any other NFS server, SAN, or iSCSI server. But I’m not ruling OpenSolaris out, if someone knows of a good ISP that hosts OpenSolaris machines and supports adding dozens of disks and decent amounts of NVRAM for ZFS then I would be interested to investigate it. Joyent has some interesting OpenSolaris virtual server plans [7], they are a little pricey but offer large amounts of data transfer. They don’t provide any information on disk IO capacity (other than saying that they use ZFS for good performance). I’ve just downloaded Nexenta (Debian with the OpenSolaris kernel) [8] and will test it out over the next few days. One of the reasons I’m tending towards Rackspace at the moment (with Slicehost as the entry point) is that they seem cooperative to customer requests. My discussions with them (on a web based “livechat” and on the phone) have indicated that they may be able to do something special for me.

• [1] http://etbe.coker.com.au/2008/05/28/xen-hosting/


• [2] http://www.linode.com/


• [3] http://journal.dedasys.com/2008/11/24/slicehost-vs-linode


• [4] http://www.slicehost.com/


• [5] http://www.rackspace.com/


• [6] http://www.netapp.com/


• [7] http://www.joyent.com/


• [8] http://www.nexenta.org/

jhead is a very nice and very powerful command line utility to mess with JPEG headers (esp. EXIF fields).

  $ apt-get install jhead

It can display/extract a great amount of metadata fields from JPEG files and also extract the thumbnails stored in JPEG files (if any). The following will list all known metadata fields from a sample photo:

  $ wget http://farm4.static.flickr.com/3173/3061542361_60acb0904b_o.jpg
  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1074172 bytes
  File date    : 2008:11:26 23:38:04
  Camera make  : Panasonic
  Camera model : DMC-FZ18
  Date/Time    : 2008:03:05 15:45:52
  Resolution   : 3264 x 2448
  Flash used   : No
  Focal length : 4.6mm  (35mm equivalent: 28mm)
  Exposure time: 0.0100 s  (1/100)
  Aperture     : f/3.6
  ISO equiv.   : 100
  Whitebalance : Auto
  Metering Mode: matrix
  Exposure     : program (auto)
  GPS Latitude : N %:.7fd %;.8fm %;.8fs
  GPS Longitude: E %;.8fd %:.7fm %;.8fs
  GPS Altitude : 174.00m
  Comment      : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.
  ======= IPTC data: =======
  Record vers.  : 4
  Headline      : Felsburg auf dem Felsberg
  (C)Notice     : www.froutes.de
  Caption       : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.

As you can see there's a huge amount of potentially privacy-sensitive metadata in your typical JPEG as generated by your camera (including camera type, settings, date/time, maybe even GPS coordinates of your location, etc). You can extract the thumbnail stored in all JPEGs in the current directory with:

  $ jhead -st "&i_t.jpg" *.jpg
  Created: '3061542361_60acb0904b_o.jpg_t.jpg'

Note that the JPEG thumbnail does not necessarily show the same picture as the JPEG itself. Depending on the image manipulation software that was used to create the edited/fixed/cropped JPEG, the thumbnail may still reflect the original JPEG contents (see sample image on the right-hand side). This is a huge potential privacy issue. There have been a number of articles about this some years ago, in case you missed them:

• disLEXia: Hidden data in Document Formats
• disLEXia: EXIF Thumbnail in JPEG images (many sample JPEGs and their differing thumbnails)
• Kevin Cohen: Digital Still Camera Forensics (PDF)

Thus, an important jhead command line to know is the following, which removes all metadata (including any thumbnails) from all JPEG images in the current directory:

  $ jhead -purejpg *.jpg
  Modified: 3061542361_60acb0904b_o.jpg

As you can see the result is that only very basic information can be gathered from the file afterwards:

  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1052506 bytes
  File date    : 2008:11:26 23:38:04
  Resolution   : 3264 x 2448
  $ jhead -st "&i_t.jpg" *.jpg
  Image contains no thumbnail

I recommend doing this for most photos you make publically available on sites like flickr etc. (unless you have a good reason not to). Finally, see the jhead(1) manpage for lots more options that the tool supports.

Wouter insists that Ruby Gems are enough of an argument to keep Rails at a distance. Even though I agree with the basic claim and think that Gems are basically insane and sick, this should be taken a bit more under perspective.
We are blessed. We are blessed to have Debian, such a rich OS with such a great package management system, and with superb integration between so many packages. Blessed are also the users of most Free Software based distributions, as they share the advantages of systems growing with full consciousness of their interaction's benefits. However, integration with the rest of the world is not seamless.
Most scripting languages have their own infrastructure for managing the modules/libraries/pacakges/whatevers dependencies. Perl has CPAN, PHP has PEAR, Ruby has Gems... I do see Gems as the most obnoxious of them all, but the basics are the same.
The Rails crowd started being Unix-centric, but the Windows (and MacOS - they are no better, believe me, at least in this regard) world has exerted its pressure. Gems caters very well to their needs, but we do suffer the integration at the distro side.
The only sane way the propietary-minded people have managed to stay clear of the well known "DLL hell" is to ship everything a given program requires bundled together - that's the main reason for the bloat of lots of applications, and for the sloppiness of security support. Every application packager is responsible for shipping updated versions to any library it bundles in, except for the very basic core that the OS itself provides. That seems so annoyingly backwards to us that... it is unbelievable.
So, yes, Rails application trees often include Rails itself. For $DEITY sake, even I have grown used to working that way, as things tend to break under your nose otherwise. My proposal (which we talked over at DebConf, but have not pushed so far) is to support simultaneous versions of Rails installed in a Debian system (of course, via different packages), more or less in the way that simultaneous versions of Ruby, PHP or Python (and, in some limited fashion, Perl - Although Perl does not suffer from this incompatible bumps. Yay for Perl!) can be installed.
...And, yes, together with the pkg-ruby-extras team, I have been trying to -slowly, yes- package whatever modules we often use so they don't have to be included in Rails applications.
So far, the best way (although by far not optimal) I have found to limit this explosion of trees is to include most libraries in my Rails application trees as git submodule trees - If they are not explicitly downloaded, the systemwide libraries will be used.
Yes, the "ship the whole thing as a bundle" approach is quite annoying. However, at least I must acknowledge that it works better than the approach I took with previous (mod_perl-based) webapps I wrote... As relatively few people grok mod_perl, I ended up with quite some apps which were not installed by anybody but me. Rails is obnoxious... but seems to be parsable by the humanity at large.

Based on feedback from my last couple of posts, I've written a guide to some good power management practices. It's aimed at people implementing desktop environments and power management interfaces for the most part, rather than end users or driver developers. Any suggestions for additions or corrections happily received.

I'm not aware of any OS that currently gets all of these correct.