This weekend I had the pleasure of visiting my friend Kike in Iru a, a city I really like but had not been able to visit in five years. I spent three days with him, after a really awful Bilman trip during Friday night. The first thing that happened on Friday morning was quite unexpected. I went out, crossed a pair of streets to get to Carlos III, and going past the corner I found myself surrounded by all kinds of policemen: red, blue, green and yellow. There were press reporters all over the place, with TV and photo cameras ready to record something big. What the hell? I looked around, and there it was: a huge blue sign read Populares de Navarra, and Mariano Rajoy was seconds away from getting out of the new PP headquarters in Nafarroa. With a few dozen policemen looking at me in suspicion, and realising my current hair dress wasn't the most appropriate for that scenario, I decided to disappear as quickly as possible. Populares aside, the weekend was really productive. I had a great time trying to find the places I visited 9 and 5 years ago. It was sad to see the fantastic Iru ako Gaztetxea turned into an iron building surrounded by old, traditional houses in the centre of the city. With Kike, I had enough time to learn more about the origins of Iru a and the three cities from which it originated, and visit the Ciudadela. On Saturday, Kike, Ana and I went to have some fun on a snowy day around Orreaga and had dinner in a small town around the area, maybe Lintzoain? I'm afraid I forgot due to the mix of basque names in my head. We had dinner in lo viejo, where I spotted a poster of Solidari@s con Itoiz for the Itoitz hustu arte campaign, a copy of which I unsuccessfully tried to get in several places in the city. On Sunday, Kike and I took a bus to Donosti, and I travelled across the most beautiful highway I've seen. But of course, that was from my perspective seated in a bus. I can imagine the Leitzaran highway project must have been greatly contested by the people of the tiny towns nearby. The spoiled valleys and views must have been really impressive in the past and seem now irreversibly ruined by a scar and holes through the mountains. This was my first visit to Donosti, which held its annual Marathon, and were lucky to meet our friend Rub n when he was around kilometre 25 of his race. He managed to finish under 2:50, which was a bit better than what he aimed for in his first Marathon. Well done! The weather was horrible, and our visit to La Concha, the old harbour and the Casco Viejo was short, we were freezing and getting our feet wet. While I visited Donosti for the first time, Mikel Laboa, one of the most respected singers in the basque culture, was ebbing away. Laboa's songs always give me goosebumps, even if I need to read a translation for the basque lyrics. Many will remember his music from Julio Medem's documentary, La pelota vasca, featuring Txoria txori, which has become a symbol of basque culture over the years. Youtube has quite a few videos, and I'd recommend watching Txoria txori, Gure bazterrak, Lili bat or Baga biga higa performed by the Orfe n Donostiarra to name a few. Mikel Laboa, goian bego.

So in my first year Human Culture and Communication class we will soon be having a class on disability and communication. I wanted to show a video on eugenics to provide some historical context but I just found out that all decent videos (or ones that I have seen and thought were decent) have been pulled down.
The one I am looking for in particular is a 10 minute video by Liam Dunaway. I find it beyond despicably pathetic that an educational video, under 10 minutes long, is not available for people to watch. For x’s sake, if one cannot easily circulate this type of video, whose whole purpose is to educate, why even bother make it? These are times when I find copyright completely totally, fully, and also absolutely ludicrous. If you are going to make a 10 minute video on a political/educational subject like Eugenics, and you don’t consider freeing it up, then there is something really contradictory driving your creative desires. update: I wrote the filmmaker and got the word that someone had uploaded the video on their account and inappropriately connected himself to the video and so the film maker has uploaded the same video here

When I have a moment, I’ll try to tally up the responses that I got to “An ethical question involving ebooks”and see if there are any interesting patterns based on self-identified generational markers. Obviously, this is not a properly controlled survey, so the results aren’t going to mean much, but it is interesting that some fairly passionately written comments came from folks who self-identified as coming from generations that broke with the common stereotypes of their respective demographic groups. If I were going to commission a study, one thing that I would almost certainly do is to ask pose a similar question about music and mp3’s, and do have the surveys asking the question about ebooks first, and half the surveys asking the questions about music first. It would be interesting to see if (a) there is a difference in attitudes between music and books, and (b) whether the order of the questions might influence the answers or not. A number of poeple have asked me about the author’s name and the title of the books/series involved. I deliberately didn’t include that information, for a number of reasons. First of all, I don’t believe idenifying the author/books/character involved is relevant to the question at hand, and in fact, might be distracting. Secondly, given the many comments, some of them quite passionate, I don’t think it would be fair to drag her name into the discussion without her permission first. I will say that the author does have a fairly extensive internet presence, and has apparently gotten a lot of questions about said character, and in fact whether those books would be made into ebooks. It’s been made quite clear that while those books were successful, they weren’t that successful, and so from an economic point of view, she chooses to write books that she (and her publishers) feel will be more economically viable. Because there will likely be no further books published containing this character, it is very unlikely that the publisher will reprint the original series of books — and when asked about whether they would be made available in ebook form, her response was effectively “it’s up to the publisher”, Apparently she has worked with a number of publishers, and while publisher X hasn’t been willing to publish her books in ebook form, publisher Y has. Furthermore, it seems that her contracts apparently delegate all decisions about how her books will be published, and whether a large Major Big City Law Firm with Fangs (aka MBCLFF) will go after copyright infringers to her publishers and her agent (who is a lawyer at said MBCLFF, and who could presumably inflict major Hurt on copyright infringers that curry the lawyer’s disfavor). I don’t know if this is true, or just her way of managing her relationship with her fans by disclaiming all responsibility about publication forms and enforcement decisions to others — but some authors do make such choices, if they are much more interested in the writing and storytelling end of things than the business side of things. Which brings up an interesting question with respect to copyright enforcement. It’s pretty obvious that many people will give different answers to the question relating to how much deference should be given to copyrights depending on whether they are owned by The Struggling Author versus whether they are owned by The Big Media Corporate Monolith, with many more allowances given if the question is framed as being primarily about the former rather than the latter. Another way in which how you frame the question radically changes the outcome depends on whether the focus is on making sure the author (and/or his surviving widow/widower/children) get paid or whether the focus is on control of one’s works. If you believe the primary justification is an economic one, then that leads to a series of ethical conclusions — the most obvious of which is that if it doesn’t result in a direct (or perhaps indirect) monetary loss to the author, there should not be a moral or ethical problem. There might be some question as to whether devaluing the secondary market might discourage the sale of new books, and hence indirectly harm the author sufficiently that this should be a concern, but those issues can be worked out. If however, you believe the primary issue at hand is one of control, a very different set of issues have to get factored into the conversation. For example, what if the author was ashamed of a book or series, and wants it to go quietly out of print, and hopefully disappear. How should that be weighed against fans who disagree with the author and who love the series? What is the right balance? For those who argue that the author’s wishes should be sacrosanct — should we move things more in that direction? What if all texts lived in DRM’ed, encrypted containers, and electronic readers had to ask permission of a central authorization server for the text could be displayed. This would allow the author to, after the fact, disable anyone from reading his or her works, if for some reason the author so desired it. Would that be a good thing? If not — and I hope most authors would agree this would be horrific power to give copyright holders — then it’s clear that author’s moral rights as creators should not be entirely sacrosanct, and that the society also has some claims on preserving its culture, and that once a book has been published and becomes part of the culture, society should have some claim on that book as part of culture. Whether that means that copyright terms should be 14 years or 20 years as opposed to whenever the Disney corporation feels like paying off more legislators to extend copyright terms is one way that question could be asked. Another is whether society should have the right to say that if after some number of years where a work has been abandoned for commercial exploitation, whether it should automatically enter the public domain. There are no obvious answers here. The final point that I want to make, which may be fairly controversial amongst the Open Source programmers in the room, is that if you believe that copyright should be fundamentally be about economic arguments of “no harm, no foul”, that this is in direct contradiction with the belief that lawsuits should be used in order to enforce the GPL. After all, the conditions imposed by the GPL are fundamentally about control, not about economic issues. Consider — if someone uses the Busybox project in an embedded device — especially if no changes has been made to the code — who has been harmed, economically? No harm, no foul, right? Or if someone uses GPLv3 code in a firmware which is protected by a digital signature — sure, it means that end users who want to modify the firmware and then use it to enhace/extend the device won’t be able to do so. But how does that economically harm the author of the GPLv3 code? Fundamentally, Copyleft schemes are all about extending control over how the code can be used. Hence, if you are an Free Software programmer who cheers on the activities of the SFLC, or who firmly believes that no one should be allowed to mix firmware which is not shipped with source with your GPL’ed software, it is completely and profoundly hypocritical to say, “F*ck the author’s wishes; if it’s not available in the from I want, I should be able to make a derived work to transfer the work into a form that I want.” What if the author is a luddite who hates eBooks and firmly believes and wants to enforce that their works should never be made available in eBook form. How is that fundamentally different from a Free Software Acolyte saying that because they abhor non-free firmware, and don’t want allow their code to be shipped alongside binary-only firmware?

Space and place has long been important to hackers. Whether it was/is the university lab, the workplace, the hacker con, or the particularly high-tech city, hackers congregate and meet face to face, often and everyday. One semi-new development has been the explosion and proliferation of hacker labs and spaces, such as Noisebridge located in the Mission district of San Francisco Foulab in Montreal. I recently got back from San Francisco and was able to spend a few nights at Noisebridge and was jaw droppingly impressed. The space is, well, spacious and nice (and located right by the Bart, a real +++++), but more important is that it is a thriving collective with all sorts of geeky participants and they have just souped up the space with all sorts of equipment, from the usual suspects (lots of computers) to lots of electronic gear such as oscilloscopes. What I also was impressed by was not only the blizzard of events but the open and accessible nature of the organization, which seemed to sit in some contrast to NYC’s hacker collective, NYC Resistor. Like so many organizations in this metropolis, they apparently are lacking in space to grow and the word on the streets (which I cannot confirm or deny as I have had very little contact with them but have heard this repeatedly) is that the organization has had a tough time letting in new members. Some folks are understanding of this given their space limitation, others have been less kind, and have referred to the group as a clique. There are already a few other initiatives under way to find a larger space so as to accommodate a more open, participatory atmosphere for a hacker space (sign up for the email list here but I imagine that over time the culture and developments of NYC Resistor will also change as new spaces develops and do hope that this creates the conditions for more access rather than less (and again, I know next to nothing of the situation though I suspect space plays a real limiting role as it does with nearly everything else in NYC). And thankfully Rose White, a NYC-based sociology graduate student at CUNY, is paying close attention to the rise and development of these hacker spaces. She is well underway doing her dissertation dissertation on these hack spaces and I really look forward to her work. update: The Axis of Awesome is a hacker space in L.A. and as far as I am concerned, dons the best name.

Interestingly, the just finished Extremadura QA + ftpmaster + i18n meeting also turned into a quite strong Bug "Smashing" party. Several participants were digging around the list of RC bugs affecting both lenny and unstable. Most often, it turned out to many bug downgrades (for many of which I'll take the blame). It's "funny" to see how sometimes our bug reporters tend to inflate their reports, most often because they feel that "bug should absolutely be fixed", but completely forgetting about the definition of 'serious' or 'grave'. I also feel that, in some way, several maintainers begin to be quite demotivated by the length of the process (which I personnally consider as being caused by several factors: decreasing (wo)manpower, haircutting about freeness of every single byte on Debian CD's or even shyness wrt correct actions needed to deal with bugs). And several of these bugs just needed a very simple action by the maintainer. And, therefore, just because we were about 15 people with a lot of time to devote to Debian for 3 days, we ended up digging in that pile of mud....removing most of the dust and sometimes very debatable RC bug reports, and finally having that count of bugs affecting both lenny and unstable go down again. This comforts me in the feeling that an aggressive approach is a good key to bug squashing. After all, if I'm sometimes too aggressive and inappropriately downgrade an RC bug, I'm sure that someone will correct me (and, anyway, I'm not that mad: sometimes I seek for advices). Ah, and "aggressive" also sometimes mean proposing to tag some bugs "lenny-ignore" (and, that needs to be very convincing with our release managers, believe me). So, what about you? Have you already smashed an RC bug today? If not, do it. Kill those nasty beasts, they deserve it. No mercy.

Hacking the PTS using SOAP I've finally found some time, thanks to the Extremadura QA + ftpmaster + i18n meeting, to release the first draft of the SOAP interface to the PTS. You probably already got the idea, which is quite simple after all. The PTS, as always, gathers information about (source) packages from various sources and melds them together into web pages. With a SOAP interface you just gain the ability of accessing such information from your programs via SOAP. A proof of concept is overdue:

    $ cat ./test.py 
    #!/usr/bin/python
    import SOAPpy
    url = 'http://packages.qa.debian.org/cgi-bin/soap-alpha.cgi'
    ws = SOAPpy.SOAPProxy(url)
    print ws.versions(source="ocaml")['unstable']
    print ws.uploaders(source="ocaml")[1]['name']
    $ ./test.py 
    3.10.2-3
    Stefano Zacchiroli

Everything is still in alpha version, but already working. Some links which you might find useful:

• (fancy) API reference (blame epydoc if you think it's too fancy)
• general information about the SOAP interface (from wiki.d.o), including pointers to other doc / tools

Please let me know if / how you are using of the SOAP interface, it will help for future developments. How it works Just a few comments on how it works. You might remember that a while ago I've made all PTS pages XHTML-valid. Well, on top of it I've implemented something along the lines microformats, that just make a clever use of ingredients already available in XHTML like classes and unique identifiers. Having that, a "reshuffling" of the information already available on the web pages (which are now kinda "semantically" tagged) can be obtained by evaluating a handful of XPaths on the (not anymore) final XHTML pages. That's precisely what the CGI implementing the SOAP API is currently doing. This way one can avoid implementing two different access paths to the information collected by the PTS: one for rendering, and one for SOAP (no, reusing the rendering one for SOAP was not an option, given that it was originally written in XSLT). The only annoyance I've encountered is that XPath is completely unaware of the "CSS-like" semantics of XHTML classes, which states that classes are space-separated list of class names, to be interpreted as sets. That means that to check whether an element belongs to a given class you need to fiddle with substring matches on the class attribute (which is quite crappy).

Let's start with a caution: this is a work-in-progress and is aimed at Squeeze (for the tools) and Squeeze+1 for the actual TDebs. Nothing about TDebs will happen within Debian any time soon. DebConf9 could be an important step in the work - as long as we've released Lenny by then.
Various steps and decisions have been discussed and tested during the meeting, to the point where we now have an early draft specification for tdebs.
See also details of the ftpmaster meeting and progress on i18n and tdebs.
The draft spec includes links to the current resources, including links to git repositories for the changes being tested for dpkg and debhelper.
These changes will supersede the existing experimental scripts in the emdebian-tdeb package.
The spec will continue to be updated during the rest of this meeting and I'll continue working on the dpkg and debhelper support.
(A trivial patch for CDBS support is already done, dependent on the debhelper support.)
Next on my ToDo list is Emdebian Grip - including a few tweaks to em_installtdeb.

I've always installed the smartmontools package on my servers because I figured it'd maybe give me some early warning about an impending disk failure. Getting proper SMART access was one of the motivating factors in moving my home-brew SAN from IDE-to-USB disks to native SATA. Since then, I've had two drives fail on me. One of the original four that I bought, and the replacement one that Seagate shipped to me. (Incidentally, I love how your only recourse to a replacement drive failing with Seagate is they'll ship you another (reconditioned) replacement drive. I can see the shipping expenses racking up very quickly). I decided to do the expedited return when the second drive failed, just because it meant I didn't have to be without a disk in my RAID-10 for a week while I waited for them to receive my drive and turn around and ship me a replacement drive, and it also meant I didn't have to mess around with packaging it myself. (Incidentally, I wonder what happens in Seagate's reconditioning process? Will someone else get my failed drive (i.e. the same serial number) after they replace the platters or whatever it was that failed? Would it be interesting to set up a website where people could register the serial numbers of drives they returned to Seagate, and how they died, and at how many power-on hours? How many times does a drive get "reborn"?) Whilst I was waiting for the replacement drive to arrive, I messed around with the failed one (I removed it from the RAID array), fiddling with the SMART self-tests that smartctl can initiate for you. The faulty drive failed the long self-test at the same LBA every time I ran it. It was around this time that I realised that the default configuration that comes with the Debian smartmontools package isn't all that useful with SATA disks. It ships with

DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner

which, as it turns out, doesn't detect SATA drives properly. I got

Nov 24 22:00:13 minotaur smartd[2990]: Device /dev/sda: ATA disk detected behind SAT layer 
Nov 24 22:00:13 minotaur smartd[2990]:   Try adding '-d sat' to the device line in the smartd.conf file. 
Nov 24 22:00:13 minotaur smartd[2990]:   For example: '/dev/sda -a -d sat' 

logged. It seems to me that if the software can detect this situation, it should be able to just try the "-d sat" behaviour automatically. So it seems that this whole time, I haven't been getting any SMART monitoring of my SATA drives. hddtemp has been happily logging the temperatures for me though. So I took the advice in the comments of /etc/smartd.conf and removed the DEVICESCAN entry in favour of explicitly naming the disks I wanted to monitor. I also discovered that I can have smartd kick off scheduled short and long self-tests on whatever schedule floats my boat. So now I've got something like this:

/dev/sda -d sat \
         -M exec /usr/share/smartmontools/smartd-runner \
         -s (L/../.././01 S/../.././(00 12))

This runs a long self-test at 1am every day, and a short one at midnight and midday. Hopefully this will help predict the next drive a bit earlier. I'm now debating whether to set up an active Nagios monitor for SMART, or to see if I can write a passive one that runs as part of the smartd-runner infrastructure that the smartmontools package provides (and is pretty neat). I guess it's already setup to email me, so it doesn't really matter whether it's Nagios emailing me, or smartd itself.

In a comment on my post about Slicehost, Linode, and scaling up servers [1] it was suggested that there is no real difference between a physical server and a set of slices of a virtual server that takes up all the resources of the machine. The commentator notes that it’s easier to manage a virtual machine. When you have a physical machine running at an ISP server room there are many things that need to be monitored, this includes the temperature at various points inside the case and the operation of various parts (fans and hard disks being two obvious ones). When you run the physical server you have to keep such software running (you maintain the base OS). If the ISP owns the server (which is what you need if the server is in another country) then the ISP staff are the main people to review the output. Having to maintain software that provides data for other people is a standard part of a sys-admin’s job, but when that data determines whether the server will die it is easier if one person manages it all. If you have a Xen DomU that uses all the resources of the machine (well all but the small portion used by the Dom0 and the hypervisor) then a failing hard disk could simply be replaced by the ISP staff who would notify you of the expected duration of the RAID rebuild (which would degrade performance). For more serious failures the data could be migrated to another machine, in the case of predicted failures (such as unexpected temperature increases or the failure of a cooling fan) it is possible to migrate a running Xen DomU to another server. If the server migration is handled well then this can be a significant benefit of virtualisation for an ISP customer. Also Xen apparently supports having RAM for a DomU balloon out to a larger size than was used on boot, I haven’t tested this feature and don’t know how well it works. If it supports ballooning to something larger than the physical size in the original server then it would be possible to migrate a running instance to a machine with more RAM to upgrade it. The question is whether it’s worth the cost. Applications which need exactly the resources of one physical server seem pretty rare to me. Applications which need resources that are considerably smaller than a single modern server are very common, and applications which have to be distributed among multiple servers are not that common (although many of us hope that our projects will become so successful ;). So the question of whether it’s worth the cost is often really whether the overhead of virtualisation will make a single large machine image take more resources than a single server can provide (moving from a single server to multiple servers costs a lot of developer time, and moving to a larger single server exponentially increases the price). There is also an issue of latency, all IO operations can be expected to take slightly longer so even if the CPU is at 10% load and there is a lot of free RAM some client operations will still take longer, but I hope that it wouldn’t be enough to compete with the latency of the Internet - even a hard drive seek is faster than the round trip times I expect for IP packets from most customer machines. VMware has published an interesting benchmark of VMware vs Xen vs native hardware [2]. It appears to have been written in February 2007 and while it’s intent is to show VMware as being better than Xen, in most cases it seems to show them both as being good enough. The tests involved virtualising 32bit Windows systems, this doesn’t seem an unreasonable test as many ISPs are offering 32bit virtual machines as 32bit code tends to use less RAM. One unfortunate thing is that they make no explanation of why “Intger Math” might run at just over 80% native performance on VMware and just under 60% native performance on Xen. The other test results seem to show that for a virtualised Windows OS either VMware or Xen will deliver enough performance (apart from the ones where VMware claims that Xen provides only a tiny fraction of native performance - that’s a misconfiguration that is best ignored). Here is an analysis of the VMware benchmark and the XenSource response (which has disappeared from the net) [3]. The Cambridge Xen people have results showing a single Xen DomU delivering more than 90% native performance on a variety of well known benchmarks [4]. As it seems that in every case we can expect more than 90% native performance from a single DomU and as the case of needing more than 90% native performance is rare it seems that there is no real difference that we should care about when running servers and that the ease of management outweighs the small performance benefit from using native hardware. Now it appears that Slicehost [5] caters to people who desire this type of management. Their virtual server plans have RAM going in all powers of two from 256M to 8G, and then they have 15.5G - which seems to imply that they are using physical servers with 16G of RAM and that 15.5G is all that is left after the Xen hypervisor and the Dom0 have taken some. One possible disadvantage of this is that if you want all the CPU power of a server but not so much RAM (or the other way around) then the Slicehost 15.5G plan might involve more hardware being assigned to you than you really need. But given the economies of scale involved in purchasing and managing the large number of servers that Slicehost is running it might cost them more to run a machine with 8G of RAM as a special order than to buy their standard 16G machine. Other virtual hosting companies such as Gandi and Linode clearly describe that they don’t support a single instance taking all the resources of the machine (1/4 and 1/5 of a machine respectively are the maximums). I wonder if they are limiting the size of virtual machines to avoid the possibility of needing to shuffle virtual machines when migrating a running virtual machine. One significant benefit of having a physical machine over renting a collection of DomUs is the ability to run virtual machines as you desire. I prefer to have a set of DomUs on the same physical server so that if one DomU is running slowly then I have the option to optimise other DomUs to free up some capacity. I can change the amounts of RAM and the number of virtual CPUs allocated to each DomU as needed. I am not aware of anyone giving me the option to rent all the capacity of a single server in the form of managed DomUs and then assign the amounts of RAM, disk, and CPU capacity to them as I wish. If Slicehost offered such a deal then one of my clients would probably rent a Slicehost server for this purpose as soon as their current contract runs out. It seems that there is a lot of potential to provide significant new features for virtual hosting. I expect that someone will start offering these things in the near future. I will advise my clients to try and avoid signing any long-term contracts (where long means one year in the context of hosting) so that they keep their options open for future offers.

• [1] http://etbe.coker.com.au/2008/11/27/slicehost-vs-linode/


• [2] http://www.vmware.com/pdf/hypervisor_performance.pdf


• [3] http://www.cmg.org/measureit/issues/mit41/m_41_1.html


• [4] http://www.cl.cam.ac.uk/research/srg/netos/xen/performance.html


• [5] http://www.slicehost.com/

Russell I know this is odd but I have an email message for you (from someone else who can’t seem to successfully get it to you). If you would like it, drop me a line and I will pass it along. (I have no idea why they wrote me but..)

Yay, long time since the last ftpmaster blog post. Time to write a new one. It’s not that I haven’t done anything since my last post, it just wasn’t only ftpmaster related. Right now I’m somewhere in Extremadura, Spain, at the Ftpmaster / I18N / QA Meeting. This meeting is near to the location of DebConf9, so yesterday we visited that venue. Still have to upload the pictures I took. It’s pretty nice, but I fear we might have a problem with wheelchairs. While the venue is very well prepared for wheelchair users, like having ramps or electric elevator platforms for every little staircase, I see one big problem in the size of the elevator going from the ground floor up to the first level. We don’t have exact measurements currently, just our rough guess from looking at it, but this is something we definitely have to look in. Something similar applies to the sleeping place. They do have a room for disabled people, but we couldn’t look into it when we where there, as it was in use (And it is pretty unpolite to just move into an occupied room). So we need to get an exact floor plan and hopefully also pictures for it later on. Luckily, the sleeping place is a much smaller issue, if it doesn’t fit we could think about placing disabled people in a Hotel. More ftpmaster related we had a lot of discussions about one of our main agenda points for this meeting, tdebs. The more we discuss it, the more we all hate it. Its a topic that touches nearly everything in Debian and will redefine quite a lot of places/actions/behaviours, so it needs a lot of thought. Which is what we start here. We wont be able to get anything substantly implemented here, but I think we can get to a common understanding of a specification and a Todo. After all the schedule currently has “lenny + 2” for this, its not a light weight step. But thats not all, would be boring if we would have only one topic. Us ftpteam also intends to/does work on other issues, like a redesign of dak and the database below it. And today we issued a statement about the AGPL. Not ftpmaster related, but also a nice work done here in Extremadura, is the setup for mirrors of packages.debian.org. This enables us to mirror the part of packages.debian.org to other .debian.org hosts thats responsible for the web view, and as such allows us to have redundancy in case of errors, load balancing and all those nice things. Comments: 2

What started with a little host naming theme riddle, and spread as a meme across various planets, has come full circle today, as I installed a new machine, khyber.madduck.net in the racks hosted at the LayerOne colocation site in Z rich. Init Seven, my favourite Swiss ISP, are sponsoring the rack space and connectivity: a routed /28 IPv4 subnet and a /64 IPv6 subnet. The machine is a Transtec Calleo 331 1U server with two quad-core AMD Opteron 2354 processors, 16Gb of RAM, and 4Tb of disk space, configured into a 2Tb RAID1 with LVM on top. It will be used to host a couple of KVM instances for a number of projects I currently operate off my main, personal server, pulse.madduck.net. Unfortunately, Transtec (as well as most other vendors I checked out) uses nVidia s MCP55Pro chipset, which had me sceptical long before I sent the order. nVidia is known to suck at thinking open-source, and the Linux kernel driver for their network chips, forcedeth, had to be reverse-engineered, because nVidia failed to cooperate. Apart from that, nVidia s chips are also often simply crap. Many of their motherboards come with broken APICs and the network chips often wreak havoc on subnets. I find it curious that a search for MCP55 or MCP55Pro on the nVidia website yields no results. Given these expectations, I insisted on a try-and-buy contract, which are not default in Switzerland (as opposed to the EU). My fears proved right, it wasn t long until I encountered the first problems with nVidia s crap: the network chip is unable to deal with segmentation offloading for larger packets, e.g. those with IPv6 payload in a bridging context, and generic segmentation offloading has to be turned off on the interface:

ethtool -K eth0 gso off

This fixes the issue, and so I stuffed it into /etc/network/interfaces and decided to keep the machine, especially after having invested the time into the server to figure out how to work the watchdog and the IPMI card. The folks behind Transtec s support hotline are helpful, but it took me a bit of time to get the information I wanted out of them. At least they deemed it perfectly natural that I wanted to run Debian, and never questioned that. Let s document some of the more internal specs here for posterity: The BIOS gives the impression of two separate watchdogs, one on the IPMI card (a SIMSO+) and one on the mainboard itself (a SuperMicro H8DMU+). Transtec told me that SuperMicro disabled the IPMI watchdog (called BMC watchdog in the BIOS) with firmware 1.29, leaving only the one on the motherboard. It took a bit of time to find out that this was a W83627THF and supported by the w83627hf_wdt driver, which I configured with module options timeout=120 nowayout=1. I left the BIOS setting at disabled since that would otherwise interfere with booting e.g. the awesome grml live system by rebooting the machine after the set amount of time. It also took a bit of time to get SOL working with the IPMI card. In the end, I found it to work best by configuring the console redirection in the BIOS to use COM2 with 38400/n/8, using hardware flow control, redirection-after-POST always, and vt100 emulation ANSI and vt-utf8 emulation were horrific. I instructed the bootloader to pass console=tty0 console=ttyS1,38400n8 to the kernel (order matters), and configured agetty in /etc/inittab like this:

T0:2345:respawn:/sbin/getty -8hL ttyS1 38400n8 vt100

To guard against the worst, I also left a grml CD in the drive, to be able to boot a rescue system via SOL in case everything else was broken. The following kernel boot parameters enable me to use the rescue system via IPMI:

vga=normal noeject noprompt console=tty0 console=ttyS1,38400n8 forensic

Michael Prokop of the grml project tried to create a customised ISO image with these options for me, but I kept getting medium errors trying to read it, independent of the burner, CD-writing-software, operating system used to burn, medium, and drive used to read, so it s possible that the ISO image is faulty. Michael used the following command line to create it (supporting files are here):

grml-live -t ~/templates-madduck -c GRMLBASE,GRML_MEDIUM,AMD64 -o \
  /grml-live/grml64-medium_2008.11-madduck -a amd64 -s sid -v \
  2008.11-madduck -g grml64-medium -r Schluchtenscheisser -V -b

Until we figure out the problem with this, the grml64 0.2 CD in the drive will do. The machine is now operational, but the IPv6 prefix I have still arrives at vera.madduck.net, a Xen-Instance hosted by Init Seven, which I will no longer need, once I find the time to migrate all the services it hosts. NP: This Will Destroy You: Young Mountain

Cooperatives-SW is working in partnership with the Regional Infrastructure for Social Enterprise (RISE) to hold a Trade Fair with support from CDA (BRAVE Ltd.), Social Enterprise Works, Co-opportunity and Watershed Media Centre. It will be held on Monday 1 December at the Watershed in Bristol. TTLLP will be there and I should post lots of news about it on Tuesday. I’ll try to microblog the event, but that depends how busy it is. Thanks to the VAT rate cut on the same day, I’ll be the only one there, leaving workers in the office just in case any of our customers have problems with their OSCommerce or Drupal online shops. (It seems there shouldn’t be any, but we hope for the best and prepare for the worst.) If you’d like to meet up on Monday, send me a comment however you prefer. Update: If you want to attend the event, registration details are on this page - it looks like registration is required, but I’ll be disappointed if they turn people away from the exhibition space between 1 and 2pm if there’s room.

Netatia has an interesting series of articles about running a computer for two people [1]. It is a bit of a kludge, they have a single X server that covers both displays and then use Xephyr to divide it into two virtual screens. The positive aspecct of this is that it shuld allow a single wide monitor to be used by two sessions as displays are getting wider regardless of the wishes of manufacturers and consumers [2] this should be useful. It’s a pity that no-one has solved the problem of having multiple video cards, sound cards, and input devices to allow a single desktop system to be used for 6 or more people. It seems that the problems that need to be solved are only the support for multiple video cards, mouse-wheel support, and sound support. Paul Ewald gave an interesting TED talk about changing the conditions for diseases so that they evolve to be benign [3]. The first example is Cholera which if spread by water will benefit from being as toxic as possible (to cause the greatest amount of diorrhea - killing the host not being a problem), but if spread by human contact benefits from leaving it’s host well enough to walk around and meet people. This and the other examples he cites seem like strong reasons for universal health-care provided by the government. If clean water is provided to all the poor people then cholera will evolve to be less harmful, and if a rich person (such as myself) is unlucky enough to catch it then the results won’t be so bad. He also notes that less harmful bacteria will often result in the victim not seeking anti-biotics and therefore less pressure for the disease to evolve resistance to anti-biotics. Therefore the people who really need them (the elderly, the very young, and people who are already sick) will find them to be more effective. Paul Stamets gave a great TED talk about fungus [4]. One of his discoveries was that fungi can be used for breaking down petro-chemicals (they can eat oil). It would be interesting to see this tested on a large scale with one of the oil spils or with the polluted land around an ooil refinery. Also he has patented a method for using fungus to kill wood-eating ants (such as the ones that briefly infested his home). Robert Full gave an interesting TED talk on robot feet [5]. I found the bit about leg spikes particularly interesting (I had always wondered why insects have spikey legs). Alan Kay gave a very interesting presentation on using computers to teach young children about science [6]. An OLPC is referenced. It makes me want to buy an OLPC for everyone I know who has young children. The start of the talk is a little slow. Dan Barber gave a very interesting TED talk about organic and humane production of foie gras in Extramuda [7]. Apparently it tastes a lot better too. Incidentally I don’t list all the TED talks I watch, only the better ones. Less than half the TED talks that I see announced seem interesting enough to download, and of those less than half are good enough that I will recommend them. The ones that I don’t recommend don’t suck in any way, it’s just that I can’t write a paragraph about every talk. Of recent times my video watching has been divided about equally between “The Bill” and TED talks. Here’s an interesting article about Sarah Palin and “anti-elitism”: The prospects of a Palin administration are far more frightening, in fact, than those of a Palin Institute for Pediatric Neurosurgery. Ask yourself: how has “elitism” become a bad word in American politics? There is simply no other walk of life in which extraordinary talent and rigorous training are denigrated. We want elite pilots to fly our planes, elite troops to undertake our most critical missions, elite athletes to represent us in competition and elite scientists to devote the most productive years of their lives to curing our diseases. And yet, when it comes time to vest people with even greater responsibilities, we consider it a virtue to shun any and all standards of excellence. When it comes to choosing the people whose thoughts and actions will decide the fates of millions, then we suddenly want someone just like us, someone fit to have a beer with, someone down-to-earth in fact, almost anyone, provided that he or she doesn’t seem too intelligent or well educated.[8] Sarah will be representing the Republican party in 2012, the desire for leaders of average intelligence (or less) will still be around then. It will be interesting to see how many votes she gets and amusing to see her interviewed. The proceedings of the “Old Bailey” - London’s Central Criminal Court have been published [9]. It’s interesting to read some of the historical information about the legal system at the time. It made me appreciate how civilised the UK (and other countries that I have visited) are now. Bruce Schneier writes about the feture of ephemeral communication [10]. He concludes with the point “until we have a Presidential election where both candidates have a complete history on social networking sites from before they were teenagers we aren’t fully an information age society“. Of course as he notes the rules are written by the older people, currently I don’t think that any candidate for high office (cabinet minister or above) anywhere in the world can have a good history on the Internet. During the course of a decade or more on the net it’s impossible not to write something that can be used against you and no reasonable person could avoid changing their views on some issues in such a time period. That’s enough to lose an election with the way things currently work.

• [1] http://netpatia.blogspot.com/2008/02/multiseat-computer-with-ubuntu-804.html


• [2] http://lenovoblogs.com/insidethebox/?p=155


• [3] http://www.ted.com/index.php/talks/paul_ewald_asks_can_we_domesticate_germs.html


• [4] http://www.ted.com/index.php/talks/paul_stamets_on_6_ways_mushrooms_can_save_the_world.html


• [5] http://www.ted.com/index.php/talks/robert_full_on_animal_movement.html


• [6] http://www.ted.com/index.php/talks/alan_kay_shares_a_powerful_idea_about_ideas.html


• [7] http://www.ted.com/index.php/talks/dan_barber_s_surprising_foie_gras_parable.html


• [8] http://www.newsweek.com/id/160080/page/1


• [9] http://www.oldbaileyonline.org/


• [10] http://www.schneier.com/blog/archives/2008/11/the_future_of_e.html

It's interesting to be here in Casar de Caceres for the joined i18n+QA+FTPmasters Extremadura meeting. Today, I've been sitting all day long in the biblioteca, which is used as hacklab, and just watched all great people around playing around with the concept of "tdebs". That weird beast can be described in two words as a concept of deb packages that would be able to distribute translation data in a more independent way then "regular" deb packages. Don't smash me as the concept is currently being twisted around in all directions by the nice folks here, to bring it to a great concept well suited for "side" material of our packages. To get an idea, you can read the original concept that popped up during the 2006 i18n Extremadura meeting, but got transformed drastically by ideas exchanged at Debconf 8 All this is made possible because the concept is now a very important issue for Emdebian and is strongly pushed by Neil Williams, whose nickname shall now be "MisterT", by general agreement of the folks here. What's really interesting right now is to see live discussions happening all the way round and see the concept slowly emerging from this. Something we could probably never do in regular mailing lists discussions or on IRC. We should really grant tons of rewards to the Extremadura region, and particularly out tireless C sar who again organised that meeting. It's still hard to say what will emerge from this and we willprobably need some more such live meetings to have real implementations of these tdeb beasts, but this is really how I like to see free software development going on. And now people begin to think about coding stuff and prepare patches, which is a good sign, in general.

Six months ago I investigated the options for Xen virtual servers [1]. I ended up receiving an offer of free hosting and not needing that, but the research was useful. There is a good range of options for Xen servers with different amounts of CPU power, RAM, bandwidth, and disk space. There are a couple of things that seem to be missing, options to upgrade from virtual servers to physical servers, and information on dedicated disk and database performance - but I’ll explain that later after some history. About a week ago a client needed a Xen virtual server in a hurry, their main server (a Xen system that I run on hardware that they rent) was getting a bit overloaded and needed to have one of the largest DomUs moved off. I ended up recommending Linode [2] based on my research and comments I received. The Linode server is working quite well and the client is happy, one nice feature of Linode is the choice of server rooms that they offer. I was able to choose a room in the same region as the other servers that the client owns and thus get ping times that are sometimes less than 2ms! Due to a missing feature in a program that I’m maintaining for the client a large number of MySQL queries are being made. Due to a problem I’m having with MySQL it won’t let me create a slave database server so all the queries go over the VPN and use a large amount of data. This combined with the other traffic that should be going over that link means that about 600G per month is being used, fortunately that is rather cheap. Linode staff handled this very well, after the server had exceeded it’s quota by 120G they asked my client to confirm that the traffic was legitimate and then suggested an upgrade to a plan that could handle the traffic (which went smoothly). Now I have another week to add the feature in question before I meet the quota again. Shortly after getting the new virtual server running at full capacity David Welton wrote a detailed review of Linode and Slicehost for the issues that matter to his use [3]. His conclusion seems strongly in favor of Linode. But now I am looking at getting a Slicehost [4] virtual server for the same client (for a different project) because Slicehost is owned by Rackspace [5], and the new project if successful will need a set of powerful servers and Rackspace seems like a reasonable company to host that. The problem with Rackspace is that they (and every other ISP I’ve researched so far) seems to offer little in regard to customers who need serious disk IO. I am planning some servers that will have a write bottleneck on a MySQL database (or maybe multiple shards), so serious disk capacity is needed. At least I would like to be able to get disk storage by the tray (12-14 disks) with the controllers having RAID-6 support. Rackspace only offers RAID-5 (according to the “livechat” person), and we didn’t get as far as discussing how to add more trays. What would be ideal is if there was an ISP that had both virtual servers and physical servers (so I could start with a virtual server and move to a physical server when things are working well), and also serious storage options. They would offer internal disks, external RAID arrays, and NetApp Filers [6] (or some equivalent device). It would be really nice if I could just instruct the ISP to add another NetApp Filer to my back-end network and have it done for me (I’m certainly not going to visit the US to install new hardware). It’s been over a day since I submitted a sales request to NetApp asking whether they partner with any ISPs and I haven’t received a response. OpenSolaris with ZFS also sounds good for disk IO performance (they have similar features to NetApp). Unfortunately the support for OpenSolaris among ISPs is not that great (while everyone offers Linux and Windows), and I haven’t used any recent version of Solaris. So using OpenSolaris would require finding someone with the skills to manage it who can work for my client - as opposed to a NetApp device that would be just like any other NFS server, SAN, or iSCSI server. But I’m not ruling OpenSolaris out, if someone knows of a good ISP that hosts OpenSolaris machines and supports adding dozens of disks and decent amounts of NVRAM for ZFS then I would be interested to investigate it. Joyent has some interesting OpenSolaris virtual server plans [7], they are a little pricey but offer large amounts of data transfer. They don’t provide any information on disk IO capacity (other than saying that they use ZFS for good performance). I’ve just downloaded Nexenta (Debian with the OpenSolaris kernel) [8] and will test it out over the next few days. One of the reasons I’m tending towards Rackspace at the moment (with Slicehost as the entry point) is that they seem cooperative to customer requests. My discussions with them (on a web based “livechat” and on the phone) have indicated that they may be able to do something special for me.

• [1] http://etbe.coker.com.au/2008/05/28/xen-hosting/


• [2] http://www.linode.com/


• [3] http://journal.dedasys.com/2008/11/24/slicehost-vs-linode


• [4] http://www.slicehost.com/


• [5] http://www.rackspace.com/


• [6] http://www.netapp.com/


• [7] http://www.joyent.com/


• [8] http://www.nexenta.org/

jhead is a very nice and very powerful command line utility to mess with JPEG headers (esp. EXIF fields).

  $ apt-get install jhead

It can display/extract a great amount of metadata fields from JPEG files and also extract the thumbnails stored in JPEG files (if any). The following will list all known metadata fields from a sample photo:

  $ wget http://farm4.static.flickr.com/3173/3061542361_60acb0904b_o.jpg
  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1074172 bytes
  File date    : 2008:11:26 23:38:04
  Camera make  : Panasonic
  Camera model : DMC-FZ18
  Date/Time    : 2008:03:05 15:45:52
  Resolution   : 3264 x 2448
  Flash used   : No
  Focal length : 4.6mm  (35mm equivalent: 28mm)
  Exposure time: 0.0100 s  (1/100)
  Aperture     : f/3.6
  ISO equiv.   : 100
  Whitebalance : Auto
  Metering Mode: matrix
  Exposure     : program (auto)
  GPS Latitude : N %:.7fd %;.8fm %;.8fs
  GPS Longitude: E %;.8fd %:.7fm %;.8fs
  GPS Altitude : 174.00m
  Comment      : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.
  ======= IPTC data: =======
  Record vers.  : 4
  Headline      : Felsburg auf dem Felsberg
  (C)Notice     : www.froutes.de
  Caption       : Aufgenommen auf dem <a href="http://www.froutes.de/TT00000014_Ars_Natura">Kunstweg Ars Natura</a>.

As you can see there's a huge amount of potentially privacy-sensitive metadata in your typical JPEG as generated by your camera (including camera type, settings, date/time, maybe even GPS coordinates of your location, etc). You can extract the thumbnail stored in all JPEGs in the current directory with:

  $ jhead -st "&i_t.jpg" *.jpg
  Created: '3061542361_60acb0904b_o.jpg_t.jpg'

Note that the JPEG thumbnail does not necessarily show the same picture as the JPEG itself. Depending on the image manipulation software that was used to create the edited/fixed/cropped JPEG, the thumbnail may still reflect the original JPEG contents (see sample image on the right-hand side). This is a huge potential privacy issue. There have been a number of articles about this some years ago, in case you missed them:

• disLEXia: Hidden data in Document Formats
• disLEXia: EXIF Thumbnail in JPEG images (many sample JPEGs and their differing thumbnails)
• Kevin Cohen: Digital Still Camera Forensics (PDF)

Thus, an important jhead command line to know is the following, which removes all metadata (including any thumbnails) from all JPEG images in the current directory:

  $ jhead -purejpg *.jpg
  Modified: 3061542361_60acb0904b_o.jpg

As you can see the result is that only very basic information can be gathered from the file afterwards:

  $ jhead *.jpg
  File name    : 3061542361_60acb0904b_o.jpg
  File size    : 1052506 bytes
  File date    : 2008:11:26 23:38:04
  Resolution   : 3264 x 2448
  $ jhead -st "&i_t.jpg" *.jpg
  Image contains no thumbnail

I recommend doing this for most photos you make publically available on sites like flickr etc. (unless you have a good reason not to). Finally, see the jhead(1) manpage for lots more options that the tool supports.

Some time ago Bill Joy (who is famous among other things for being a co-founder of Sun) [1] wrote an article for Wired magazine titled “Why the future doesn’t need us” [2]. He wrote many sensible things but unfortunately focussed on the negative issues and didn’t receive a good response. On reading it today I thought more highly of the article than I did in 2000 when it was printed, largely due to having done some background research on the topic. I’ve recently been reading Accelerating Future [3] which has a more positive approach. Now a talk by Bill Joy from 2006 has been published on the TED.com web site [4]. He starts by talking about “super empowered individuals” re-creating the 1918 flu. He also claims that “more technology super-empowers people more“. Such claims seem to be excessively hyped, I would be interested to see comments from someone who has a good knowledge of current bio-technology as to the merits of those claims. He talks briefly about politics and has some good points such as “the bargain that gives us civilisation is the bargain to not use power” and “we can’t give up the rule of law to fight an asymmetric threat - which is what we seem to be doing“. He mentions Moore’s law and suggests that a computer costing $1000 then (2006) might cost $10 in 2020. He seems to be forgetting the cost of the keyboard and other mechanical parts. I can imagine a high-end CPU which cost about $800 a couple of years ago being replaced by a $2 CPU in 2020, but I don’t expect a decent keyboard to be that cheap any time before we get fully automated nano-factories (which is an entirely separate issue). Even the PSU (which has a significant amount of government regulations for safety reasons) will have a floor cost that is a good portion of $10. Incidentally the keyboard on my EeePC 701 sucks badly, I guess I’m spoiled by the series of Thinkpad keyboards that I keep getting under warranty (which would cost me a moderate amount of money if I had to pay every time I wore one out). I will make a specific prediction, that by 2015 one of the better keyboards will comprise a significant portion of the entire cost of a computer system (more than a low end computer unit) - such that in some reasonable configurations the keyboard will be the most expensive part. It would be good if PCs could be designed to use external PSUs (such as the Compaq Evo models that took laptop PSUs). Then the PSU, keyboard, and monitor would be optional extras thus giving a small base price. Given that well built PSUs and keyboards tend not to wear out as fast as people want to replace computers, it seems that financial savings could be provided to most customers by allowing them to purchase the computer unit without the extra parts. People like me who type enough to regularly wear out keyboards and who keep using computers for more than 5 years because they still work are in a small minority and would of course be able to buy the same bundle of computer, PSU, and keyboard that new users would get. In 2020 a device the size of an iPaQ H39xx with USB (for keyboard and mouse) and the new DisplayPort [5] digital display interface (that is used in recent Lenovo laptops [6]) would make a great PDA/desktop. You could dock a PDA the way some people dock laptops now and carry all your data around with you. Bill cites an example of running the Mac interface on an Apple ][ (does anyone know of a reference for this) as an example of older hardware being more effective with newer software. It's a pity that often new software needs such powerful hardware. EG it's only recently that hardware developments have overtaken the developments of OpenOffice to make it deliver decent performance. He has an interesting idea of using insurance companies to replace government regulation of food and drugs. The general concept is that if you can convince an insurance company that your new drug is not a great risk to them so that they charge a premium you can afford then you could sell it without any further testing. Normally I'm in favor of government regulation of such things, but given the abject failures of the US government this idea has some merit. Of course there's nothing stopping insurance companies from just taking a chance and running up debts that they could never hope to repay (in a similar manner to so many banks). Finally I think it's interesting to note the camera work that the TED people used. My experience of being in the audience for many lectures (including one of Bill Joy's lectures in the late 90's) is that a speaker who consults their notes as often as Bill does gives a negative impression to the audience. Note that I'm not criticising Bill in this regard, often a great talk requires some significant notes - very few people can deliver a talk in a convincing manner entirely from memory. It seems to me that the choices of camera angle are designed to give a better impression than someone who was seated in the audience might receive - there's no reason why a video of a talk should be spoiled by seeing the top of the speaker's head while they consult their notes!

• [1] http://en.wikipedia.org/wiki/Bill_Joy


• [2] http://www.wired.com/wired/archive/8.04/joy.html


• [3] http://www.acceleratingfuture.com/michael/blog


• [4] http://blog.ted.com/2008/11/what_i_worry_ab.php


• [5]
  
• [6] http://lenovoblogs.com/insidethebox/?p=161

I love it when a lack-of-humor and lack-of-appropriateness-originated flamewar causes somebody to point me towards a very nice display of intelligent humor. Specially when it is so close to me, to my roots, to my family and my personal history. FWIW, for several years, while I was a BBS user, I used WereWolf as my nickname. Great thanks to Frank K ster - and, of course, to Christian Morgenstern.

The Werewolf - English translation by Alexander Gross A Werewolf, troubled by his name,
Left wife and brood one night and came
To a hidden graveyard to enlist
The aid of a long-dead philologist. "Oh sage, wake up, please don't berate me,"
He howled sadly, "Just conjugate me."
The seer arose a bit unsteady
Yawned twice, wheezed once, and then was ready. "Well, 'Werewolf' is your plural past,
While 'Waswolf' is singularly cast:
There's 'Amwolf' too, the present tense,
And 'Iswolf,' 'Arewolf' in this same sense." "I know that--I'm no mental cripple--
The future form and participle
Are what I crave," the beast replied.
The scholar paused--again he tried: "A 'Will-be-wolf?' It's just too long:
'Shall-be-wolf?' 'Has-been-wolf?' Utterly wrong!
Such words are wounds beyond all suture--
I'm sorry, but you have no future." The Werewolf knew better--his sons still slept
At home, and homewards now he crept,
Happy, humble, without apology
For such folly of philology.