In a Java framework I'm working on, 'pairs' arise everywhere. Unfortunately in contrast to e.g. C++, Java doesn't include a predefined 'pair' class. C++ templates are really nice because of the way they are complied and optimized (in particular they also handle what Java calls 'native datatypes'); Java generics aren't up to par with that. (But yes, Java offers other benefits, such as being much easier to parse and thus refactor). Anyway, this is not going to be a rant on Generics.So I have this interface in Java called Pair<FIRST,SECOND> along with two implementations SimplePair<FIRST,SECOND> and ComparablePair<FIRST extends Comparable<FIRST>,SECOND extends Comparable<SECOND>>.For performance reasons, SimplePair is declared 'final', and so is ComparablePair. It's written everywhere that making classes final can make a large difference in Java, and since these objects will be used in a lot of places, it seems reasonable to care about this here.However, it would often be nice to have better readable code, that is assuming I'm using SimplePair<Monkey,Banana>, it would then be nice to make a derived class BananaPreference extends SimplePair<Monkey,Banana>, with added methods getMonkey() and getPreferredbanana() to make the resulting code more readable.Having readable code is also often quite as important as having performant code, after all ...If someone with solid experience in Java optimization has some ideas to share, please do so! Email: erich AT debian DOT org - no comments in blog.Right now, I have one idea on how it could be possible to achieve both (seriously, I could use some feedback from Java Gurus on that): make SimplePair and ComparablePair abstract, all methods there final, then derive final classes as needed. Does that combine the benefits?[Update: I received from Joachim Sauer the following helpful link: JavaOne presentation on performance tuning and various VMs. Basically this seems to indicate that in all these common situations, any modern Java VM should be able to figure out the inlining options automatically and optimize appropriately, so it won't benefit from any "final" hint by the developer. Note that a C++ compile doesn't do runtime optimization, but allows compile time optimization at a much lower level, so this rule doesn't apply to C++.]

The recent update to udev 0.140 lead to a system without network access to me. The error messages were:

Running 0dns-down to make sure resolv.conf is ok...done.
Setting up networking....
Configuring network interfaces...
ioctl[SIOCGIFFLAGS]: No such device
Could not get interface  ath0  flags
ioctl[SIOCSIWPMKSA]: No such device
ioctl[SIOCSIWMODE]: No such device
Could not configure driver to use managed mode
ioctl[SIOCGIWRANGE]: No such device
ioctl[SIOCGIFINDEX]: No such device
ioctl[SIOCSIWENCODEEXT]: No such device
ioctl[SIOCSIWENCODE]: No such device
ioctl[SIOCSIWENCODEEXT]: No such device
ioctl[SIOCSIWENCODE]: No such device
ioctl[SIOCSIWENCODEEXT]: No such device
ioctl[SIOCSIWENCODE]: No such device
ioctl[SIOCSIWENCODEEXT]: No such device
ioctl[SIOCSIWENCODE]: No such device
ioctl[SIOCSIWAP]: No such device
ioctl[SIOCGIFFLAGS]: No such device
wpa_supplicant: /sbin/wpa_supplicant daemon failed to start
/etc/network/if-pre-up.d/wpasupplicant exited with return code 1
SIOCSIFADDR: No such device
ath0: ERROR while getting interface flags: No such device
SIOCSIFNETMASK: No such device
SIOCSIFBRDADDR: No such device
ath0: ERROR while getting interface flags: No such device
ath0: ERROR while getting interface flags: No such device
Failed to bring up ath0.
done.

It seems, the update added rules to /etc/udev/rules.d/70-persistent-net.rules by increasing the device number and applying the last applicable NAME directive instead of the first one. This lead to the following file here:

SUBSYSTEM=="net", DRIVERS=="?*", ATTRS address =="XX:XX:XX:XX:XX:a7", ATTRS type =="1", NAME="ath0"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS address =="XX:XX:XX:XX:XX:18", NAME="eth0"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS address =="XX:XX:XX:XX:XX:XX", NAME="eth1"
# PCI device 0x10ec:0x8139 (8139too)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR address =="XX:XX:XX:XX:XX:18", ATTR type =="1", KERNEL=="eth*", NAME="eth2"
# PCI device 0x168c:0x0013 (ath_pci)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR address =="XX:XX:XX:XX:XX:a7", ATTR type =="1", KERNEL=="ath*", NAME="ath1"

So the devices created were eth2 and ath1 but /etc/network/interfaces contained entries for eth0 and ath0. So fixed /etc/udev/rules.d/70-persistent-net.rules and rebooted. <rant>Is it really necessary to break network access with an update???</rant> Update I forgot this one: #521521.

Flame has written some ideas about building Linux Powered Battle Droids and demonstrating them [1]. It seems to me that the biggest problem with having a battle of a similar nature to Robot Wars [2] is creating a safe arena. Getting an area the size of a basketball court fenced off with bullet-proof plastic is not going to be cheap. The first solution to this problem that occurred to me was to have a battle held underwater. Water really slows down projectiles. The disadvantages of this are that it s slightly more difficult to view. The viewing options are to using a web-cam, having a port-hole in the side of the pool (impossible if you don t own the pool), and having a transparent viewing platform on the surface of the water (which would also be difficult and maybe expensive). Another disadvantage of submarine warfare is that having a droid spring a leak and quietly sink is not very dramatic. It might be a better option to use aerial warfare via helicopters and balloons. The amount of weight that such craft can carry is not particularly great so there will be no real armor and fairly weak weapons. A motorbike helmet and a heavy coat should be all the protective equipment that is needed if the usual rules prohibiting projectile weapons from such contests are implemented. A petrol powered model aeroplane could do some serious damage if it hit someone at full speed. But I don t think that an indoor basketball court or any other enclosed arena that might be available would be large enough for fighter-planes. So it would be just balloons and helicopters.

• [1] http://cyko-circus.blogspot.com/2009/02/linux-powered-battle-droids.html


• [2] http://en.wikipedia.org/wiki/Robot_Wars_(TV_series)

Two days ago I read about the Ada Lovelace day, which was an event were men and women were asked to blog about the women in technology which made an impact on them, in an effort to present girls (and women) all over the world with female role models in technology.

In similar though more personal motivation I choose my machine naming scheme at home two years ago: my (virtual) machines at home are named after women who made a difference, mostly in IT, but also some in more general science. Even though this list is not 100% in the spirit of pointing out the one woman who made a personal difference to one self, I thought I share my list, as I found it very interesting to learn about these great women myself.

goldberg is named after Adele Goldberg, who pioneered the smalltalk programming language in the 70s. According to a sympathetic annecdote she refused to show the system to Steve Jobs and only did so after her superiors ordered her to do so. Thats were Apple got many ideas which made them famous for being innovative...

goldwasser is named after Shafi Goldwasser, who is a RSA professor at MIT, doing research in complexity theory, cryptography and computational number theory. She is the co-inventor of zero-knowledge proofs.

lovelace is named after Ada Lovelace, and is widely recognized as the first programmer, who wrote a program (something that manipulated symbols according to rules) for a machine which has not even been build at that time. Of course, some men on this planet object to her being the first programmer, as work by women is always insignificant... </irony or maybe sarcasm>

bunten is named afer Danielle Bunten Berry, who was a game designer and programmer best known for the games M.U.L.E. and Seven Cities of Gold.

holberton is named after Betty Holberton, who was one of the six original programmers of ENIAC, the first general-purpose electronic digital computer, during and after World War II.

hopper is named after Grace Hopper who wrote the first compiler ever, the A compiler. She was also heavily involved in creating COBOL and standardizing COBOL and FORTRAN.

koller is named after Daphne Koller who does AI research at MIT and who seems to behind the idea of bayesian machine learning, which you might know from your spam filters... (I can only state her involvement in bayesian learning so vaguely as I'm basically clueless about the topic :)

leveson is named after Nancy Leveson who is also an MIT professior and is an expert in systems and software safety.

Last but not least I like to remark that none of the women above is even nearly covered with what I wrote here or what is written in wikipedia, which I used as my main source for this blog post. It's simple impossible to sum up one life that brief and be close to accurate.

And then there is another group of women whose achievements are noteworthy as well and who choose to present themselves on that webpage to show others that Debian is not a mens club. Thank you (a lot) for that and for your other contributions to Debian! If you want to learn more about these giants, I recommend this talk. Guess who's achievements I admire most in this group ;-)

Somewhat unrelated I wish to shout cheers to Yvonne, who tought me many things and who died six years ago today. You will also not be forgotten, though the pain is mostly gone now (well, it's gone, except for days like today). Thank you, among other stuff, for teaching me to celebrate and cheer every day, even if not everything is cheerful. Wenn die Nacht am tiefsten, ist der Tag am n chsten! :-)

A couple of things I loved in the past few days:

• While talking about IP over DNS and aircrack techniques with a friend, being given some credentials to use a near-by hotspot directly (without tunneling).
• Attending a very nice concert with an old friend: das Hager-Trio in Villa Falkenhorst [de]. It featured Claude Debussy, Silvio Lazzari, and Felix Mendelssohn Bartholdy. Of course I'm going to lack words to describe the wonderful moment I had in that very tiny room (approx. 60-70 seats, which probably helped obtain a very warmish atmosphere). Reminds me I should go out more often. :)

: Unfortunately, I can't seem able to find a related webpage. Its member: Inge Hager (violin), Elke Hager (cello), and Enrico Pompili (piano).

Copy of debian-devel-announce. We re happy to announce that Debian has been accepted again as one of the mentoring organizations for this year s Google Summer of Code. According to the timeline for this year s program, we will only have a few days to get set up and ready. Student applications will be accepted between the 23rd of March and 4th of April, but we re already being contacted by students interested in working with us. That means that if you want to get involved as a mentor or a student, you don t have very long. We re also still actively looking for more project ideas - what would you like somebody to work on? You don t need to be able to mentor a project to propose it. Both students and mentors will have to register on the new Melange application. Once you ve edited your profile, you will have a linkid . Always keep it at hand. We will need it to process registrations and applications to match it with who you are. If you d like to mentor, you will need to apply to become a mentor and click on Debian in the list of orgs. Also please add yourself to the Debian wiki page and keep in touch with us (see below). Once you ve done that, the admins will be able to approve you so you can be added into the official Debian mentors list. Please add any specific ideas that you may have, or if you have nothing specific then list the areas where you think you could mentor a student. Existing Debian developers are preferred as mentors, as that will make things easier on some fronts (e.g. sponsored package uploads, data setup on Debian-hosted machines). But that s not a hard and fast rule. If you re a student hoping to work on Debian this summer in SoC, please get in contact with us (see below) to talk about project ideas, either what we already have listed or your own. It s in everybody s interests to work on applications as much as possible before the deadline closes to improve their chances of acceptance. Ideally we want to encourage new blood to join in and contribute, but we ll be happy to accept high quality applications from anybody that meets Google s eligibility requirements. We can be contacted at:

• the Debian wiki page
• the debian-soc mailing list
• on IRC (irc.debian.org, #debian-soc)
• on twitter (DebianGSoC) or identi.ca (debiangsoc)

Good luck to all involved; may all your code be merged in squeeze!

As I already told you, I am using XMPP4R-Simple in an internal project. For this, I d have to add PubSub capabilities to it, and I did when I decided to get serious with git. Today I committed a version with integration of various patches by other forks I found on GitHub plus a lot of improvements on PubSub functions. With this version it s simple to use PubSub like this:

require 'xmpp4r-simple'
# Simple function just to parse an event
require 'time'
def parse_event(event)
  item = event.children[0]
  node = item.node
  time = nil; item.each_element("//published")    e  time = Time.parse(e.text)  
  text = nil; item.each_element("//body")    e  text = e.text  
  return   :item => item, :node => node, :time => time, :text => text  
end
# Create the clients
im1 = Jabber::Simple.new "im1@example.com", "password" 
im2 = Jabber::Simple.new "im2@example.com", "password" 
# im1 creates a node
im1.create_node("/some/node")
# im2 subscribe to that node
im2.pubsubscribe_to("/some/node")
# We'll start a simple thread to get the events coming from that node to im2
Thread.new   loop  
  sleep 1 while ! im2.received_events?
  im2.received_events    event 
    h = parse_event(event)
    puts ">>> Got an event from node # h[:node]  published at # h[:time]  with text # h[:text] " 
   
 
# Now im1 just publishes anything to that node.
im1.publish_atom_item("/some/node", "This is my node", "This is the content of my node")
# The thread should capture the event and run puts on the hash from parsing.

PubSub is great, isn t it?

Bruce Schneier writes about the risks involving children abandoned in cars and cites an article about the tragic deaths of children in hot cars [1]. One unfortunate error that he made was to not cite the following from the end of the last page of the Washington post article he cited [2]:
In hyperthermia cases, he believes, the parents are demonized for much the same reasons. We are vulnerable, but we don t want to be reminded of that. We want to believe that the world is understandable and controllable and unthreatening, that if we follow the rules, we ll be okay. So, when this kind of thing happens to other people, we need to put them in a different category from us. We don t want to resemble them, and the fact that we might is too terrifying to deal with. So, they have to be monsters. I believe that similar thought processes are used in relation to many other situations, and that such thought processes prevent people from taking appropriate actions to minimise the risk. If someone considers that forgetting a child in the back seat to be an accident that could happen to anyone then they would be inclined to take action to minimise the risk (such as spending some money on a sensor). If however they consider such forgetfulness to be proof of being a bad parent , then as they are a good parent they would have to avoid buying a monitor. I m surprised that Bruce didn t draw an analogy between this and the forgetful losses of laptops and guns by people who work for law enforcement agencies (which he has written about before). I wonder how expensive it would be to make a sensor for heart-rate, breathing, and temperature integrated with a GSM modem and a GPS? If it could be small enough to be attached to clothes then the child could wear it at all times. If such a sensor was to detect a sign of a problem it wouldn t matter whether the child was forgotten in a car, at day-care, or even being actively supervised. The data would be sent to the monitoring agency along with GPS data. The monitoring agency could then phone the parents. If the parents don t answer or don t know where the child is then the police could track down the GPS location. Probably most calls would be due to parents leaving a child too close to an air-conditioner or playing outside in the sun in summer which are unlikely to give a fatal result and a phone call would get a quick fix for what would only be a minor health problem. If the device was marketed as monitoring for sleep apnia then parents could buy it without admitting to the possibility that they might do anything wrong. The causes of SIDS are a topic of ongoing research and parents can admit to being worried about their children suffering from it without admitting any possibility that they might make a mistake.

• [1] http://www.schneier.com/blog/archives/2009/03/leaving_infants.html


• [2] http://www.washingtonpost.com/wp-dyn/content/article/2009/02/27/AR2009022701549.html

From the FFII:

EPO seeks to validate software patents without the European Parliament

Benjamin Henrion, President of the association, says: "The current plan of the patent lobby is very clear: avoid a new software patent directive, validate the EPO practice via a central patent court, and guide the hand of the courts via a decision of the Enlarged Board of Appeal. They want to avoid the intervention of the European Parliament in substantive patent law."

In other words, seeing as the elected representatives have blocked a change in the law, the unelected quango will simply seek a change in the rules to make it unnecessary to change the law and therefore avoid asking the elected representatives at all.

This is crazy, dangerous and downright sinister.

"Considering that the EPO is an institution acting as judge and party, where the attributions and procedures have to be revised. [...] Demand the revision of rules of function of the EPO in order to guarantee that this institution can publicly justify the accountability in the exercise of its functions [...]."

Judge, jury and executioner more like.

Interested parties have up to the last day of April to send their comments to the Enlarged Board of Appeal.

Information for third parties wishing to file written statements

Communication from the Enlarged Board of Appeal concerning case G 3/08

(my comments added in bold)

In accordance with Article 112(1)(b) EPC, the President of the European Patent Office has referred the following points of law concerning the limits of patentability of programs for computers within the meaning of Article 52(2)(c) and (3) EPC to the Enlarged Board of Appeal. The case is pending under ref. No. G 3/08.

The questions referred are:

1. Can a computer program only be excluded as a computer program as such if it is explicity claimed as a computer program?

No, any computer program is excluded no matter how it is claimed because a computer program is not patentable. The patent claim must contain material other than a computer program and the computer program itself cannot be claimed as part of any patent claim nor can any computer program ever be deemed to have infringed any patent claim even if that claim was previously based entirely in hardware.

2.(a) Can a claim in the area of computer programs avoid exclusion under Art. 52(2)(c) and (3) merely by explicity mentioning the use of a computer or a computer-readable data storage medium?

No patent claim can be made against any method expressed as a computer program. It is the software that matters, not the paraphernalia of computers. Therefore, whether a computer or computer-readable data storage media are mentioned, any software is automatically excluded from patentability. If the claim makes no sense without that software, then the claim is excluded.

(b) If question 2(a) is answered in the negative, is a further technical effect necessary to avoid exclusion, said effect going beyond those effects inherent in the use of a computer or data storage medium to respectively execute or store a computer program?

Any effect is excluded from patentability if that effect is or can be executed by software. If the effect becomes possible via software at some point in the future, the software cannot be deemed to have infringed the patent claim because software is not patentable.

3.(a) Must a claimed feature cause a technical effect on a physical entity in the real world in order to contribute to the technical character of the claim?

Effects on the "real world" are irrelevant - if the effect is or can be performed in software, the effect is excluded and any software implementing that effect cannot be deemed to have infringed the patent claim. The real world still includes the electrons that implement the effect of software but software is not patentable.

(b) If question 3(a) is answered in the positive, is it sufficient that the physical entity be an unspecified computer?

No, the effect is not the issue, the method is the issue.

(c) If question 3(a) is answered in the negative, can features contribute to the technical character of the claim if the only effects to which they contribute are independent of any particular hardware that may be used?

Portability infers software and software is not patentable. A claim must be excluded if it is wholly or partially implemented as software or can be wholly implemented as software and no software implementation can be deemed to have infringed the patent claim either in whole or in part now or in the future because software is not patentable.

4.(a) Does the activity of programming a computer necessarily involve technical considerations?

No. Programming is a form of speech, it uses languages and different languages have different requirements for technical expertise. Some languages require little or no technical expertise to generate an effect within a computer by means of software.

(b) If question 4(a) is answered in the positive, do all features resulting from programming thus contribute to the technical character of a claim?

No software programming can have any technical character for patentability because software is not patentable.

(c) If question 4(a) is answered in the negative, can features resulting from programming contribute to the technical character of a claim only when they contribute to a further technical effect when the program is executed?

No, if the features result from programming then the software to effect those features is not patentable and the feature itself is not patentable. No effect of a computer program can be wholly or partially patentable.

The text of the referral in the English language is available under Referrals pending before the Enlarged Board of Appeal.

The Enlarged Board of Appeal considering the referral will be composed as follows:

* P. Messerli (CH) (Chairman)
* M. Vogel (DE)
* D. Rees (GB)
* M. Dorn (DK)
* K. H rmand (EE)
* A. Klein (FR)
* J.-P. Seitz (FR)

It is expected that third parties will wish to use the opportunity to file written statements in accordance with Article 10 of the Rules of Procedure of the Enlarged Board of Appeal (OJ EPO 2007, 303 ff). To ensure that any such statements can be given due consideration they should be filed together with any new cited documents by the end of April 2009 at the Registry of the Enlarged Board of Appeal, quoting case number G 3/08. An additional filing of the statement and documents in electronic form would be appreciated (Dg3registry_eba@epo.org ).

---

Feel free to add your own answers to the questions as comments.

Today, I was sitting on the couch. Jacob crawled up, and pushed me forward, saying I go there. He crawled behind me. Then, ready for one of his favorite games, yelled out: QUISH, daddy! So I leaned back gently a bit, and said squish! Jacob yelled, laughing, QUISH! Quish again, daddy! So I d lean back a bit, gently, again, this time reaching behind me to tickle him a bit as I leaned. Squash! Louder laughing. QUASH!!! Right in my ear ouch, but I didn t mind. Do it again, daddy! Right now, being a dad seems complicated enough. You ve got to have the right touch to squish a 2-year-old without really squishing him. Or have the presence of mind to realize that when Jacob was happily playing outside, then suddenly comes running over, very upset, saying Go inside! it means he probably needs to use the potty urgently, or just had an accident. (Or both, as it happened today.) Or to recognize that little walk that means he really does need to use the potty even though he d rather not. And, of course, there s figuring out what he s saying, when his words can still be a bit garbled. But these all seem simple to me, compared to what will come. How will we help Jacob to grow as a person of good character? How will we meet his need to be challenged intellectually? Will we be able to maintain a good relationship, and yet still have the judgment to have the right set of rules, when he gets to high school? Will I have a good relationship with him as an adult? And how am I going to react when the day comes when he tells me I ought to move into the nursing home? Jacob, of course, doesn t care about any of that right now. Each night, when I put him down for the night, he wants me to cover him up with blankets. Once I ve done that, he peeks out and says, Have a good night, dad! I always reply with You have a good night too, Jacob! If we can get along that well for the next 60 years, I guess we ll do all right.

Earlier today I stumbled across a recording of a work I had not heard in a few years by the obscure Swiss-German composer Joachim Raff. The composition is an orchestral transcription of the Chaconne from Bach's Partita in D minor, without doubt the finest work ever written for solo violin. This piece has been transcribed countless times; the source material is clearly a masterpiece, but what makes the movement really attractive is its implicit nature - dispite the violin's limited facilities for counterpoint and polyphony in general, Bach succeeds in alluding to a multitude of voices within the same line. Larry Solomon provides a brief overview of what to look out for. Raff's transcription thus endeavours to render the implicit explicit; for example, where there is the implication of a held note, it can played for its implied duration. Indeed, Raff believed that Chaconne was in itself a "reduction" from an orchestrated original. Whilst I don't share that literal belief, it is certainly good working model for a transcription. And what a transcription! I've often thought of the romantic transcriptions such as these to be a guilty pleasures; they clearly violate any sensibilities I have about performing early music. However, what is most surprising (and guilt-relieving) about Raff's effort is the sheer amount of novel material involved. Here is an example starting at bar 216 - I have overlayed Bach's original line with a melody of Raff's own invention:

There are countless other examples which, following Bach's example, vary greatly in their subtlety. No prizes for guessing why I chose this excerpt though. However, dispite these additions I feel some crucial aspects are actually lost in translation. Firstly, we can muse over whether Bach would have approved of such a rendering - not wishing to dwell too deeply in well-tread arguments, many would point to Bach's re-use of his own material (as well as his transcriptions of other composers) to illustrate that he was not against the practice, but I find that argument difficult to apply to the Chaconne - the dualism between the implicit and the explicit would have greatly appealed to Bach, so to rob the work of it would seem to be doing him a serious discourtesy. Furthermore, the implications in Bach's work are certainly not followed to the same conclusions by all - a truly solo musician has the advantage of being able to make their own decisions about the music, perhaps even on the spur of the moment. In contrast, an ensemble is effectively forced to obey Hr. Raff's inferences. This lack of spontaneity is particularly damning; I am sure one would tire of differing interpretations of Raff's transcription quicker than one would of Bach's original. Lastly (and certainly most subjectively) there is something disarmingly solitary about the original work which is only amplified by being played solo - dispite my inability to provide a satisfactory rendition I would find performing this piece with (or even to) others rather discomforting. In conclusion, I would recommend listening to this work (and the rest of the CD). I have found it extremely illuminating, if only about Bach's original than of Raff's.

Edited to add: Sorry, it turns out that ext4 does now have a heuristic to deal with this case. However, it seems that btrfs doesn't. My point remains, though - designing a filesystem in such a way that a useful behaviour is impossible is unhelpful for the vast majority of application writers, even if it does improve performance in some other use cases.

Further edit: It's mentioned in the comments that btrfs will force a flush of data when a rename is performed as of 2.6.30. That prevents the data loss, but it means that we're still stuck with disk access when we'd be happy with that being batched up for the future. It feels like we're optimising for the wrong case here.

Original text:

There's been a certain amount of discussion about behavioural differences between ext3 and ext4[1], most notably due to ext4's increased window of opportunity for files to end up empty due to both a longer commit window and delayed allocation of blocks in order to obtain a more pleasing on-disk layout. The applications that failed hardest were doing open("foo", O_TRUNC), write(), close() and then being surprised when they got zero length files back after a crash. That's fine. That was always stupid. Asking the filesystem to truncate a file and then writing to it is an invitation to failure - there's clearly no way for it to intuit the correct answer here. In the end this has been avoided by avoiding delayed allocation when writing to a file that's just been truncated, so everything's fine.

However, there's another case that also breaks. A common way of saving files is to open("foo.tmp"), write(), close() and then rename("foo.tmp", "foo"). The mindset here is that a crash will either result in foo.tmp being zero length, foo still being the original file or foo being your new data. The important aspect of this is that the desired behaviour of this code is that foo will contain either the original data or the new data. You may suffer data loss, but you won't suffer complete data loss - the application state will be consistent.

When used with its (default) data=ordered journal option, ext3 provided these semantics. ext4 doesn't. Instead, if you want to ensure that your data doesn't get trampled, it's necessary to fsync() before closing in order to make sure it hits disk. Otherwise the rename can occur before the data is written, and you're back to a zero length file. ext4 doesn't make guarantees about whether data will be flushed before metadata is written.

Now, POSIX says this is fine, so any application that expected this behaviour is already broken by definition. But this is rules lawyering. POSIX says that many things that are not useful are fine, but doesn't exist for the pleasure of sadistic OS implementors. POSIX exists to allow application writers to write useful applications. If you interpret POSIX in such a way that gains you some benefit but shafts a large number of application writers then people are going to be reluctant to use your code. You're no longer a general purpose filesystem - you're a filesystem that's only suitable for people who write code with the expectation that their OS developers are actively trying to fuck them over. I'm sure Oracle deals with this case fine, but I also suspect that most people who work on writing Oracle on a daily basis have very, very unfulfilling lives.

But anyway. We can go and fix every single piece of software that saves files to make sure that it fsync()s, and we can avoid this problem. We can probably even do it fairly quickly, thanks to us having the source code to all of it. A lot of this code lives in libraries and can be fixed up without needing to touch every application. It's not the end of the world.

So why do I still think it's a bad idea?

It's simple. open(),write(),close(),rename() and open(),write(),fsync(),close(),rename(), are not semantically equivalent. One is "give me either the original data or the new data"[2]. The other is "always give me the new data". This is an important distinction. fsync() means that we've sent the data to the disk[3]. And, in general, that means that we've had to spin the disk up.

So, on the one hand, we're trying to use things like relatime to batch data to reduce the amount of time a disk has to be spun up. And on the other hand, we're moving to filesystems that require us to generate more io in order to guarantee that our data hits disk, which is a guarantee we often don't want anyway! Users will be fine with losing their most recent changes to preferences if a machine crashes. They will not be fine with losing the entirity of their preferences. Arguing that applications need to use fsync() and are otherwise broken is ignoring the important difference between these use cases. It's no longer going to be possible to spin down a disk when any software is running at all, since otherwise it's probably going to write something and then have to fsync it out of sheer paranoia that something bad will happen. And then probably fsync the directory as well, because what if someone writes an even more pathological filesystem. And the disks sit there spinning gently and chitter away as they write tiny files[4] and never spin down and the polar bears all drown in the bitter tears of application developers who are forced to drink so much to forget that they all die of acute liver failure by the age of 35 and where are we then oh yes we're screwed.

So. I said we could fix up applications fairly easily. But to do that, we need an interface that lets us do the right thing. The behaviour application writers want is one which ext4 doesn't appear to provide. Can that be fixed, please?

[1] xfs behaves like ext4 in this respect, so the obvious argument is that all our applications have been broken for years and so why are you complaining now. To which the obvious response is "Approximately anyone who ever used xfs expected their data to vanish if their machine crashed so nobody used it by default and seriously who gives a shit". xfs is a wonderful filesystem for all sorts of things, but it's lousy for desktop use for precisely this reason.

[2] Yes, ok, we've just established that it actually isn't that in the same way that GMT isn't UTC and battery refers to a collection of individual cells and so you don't usually put multiple batteries in your bike lights, but the point is that this is, for all practical intents and purposes, an unimportant distinction and not one people should have to care about in their daily lives.

[3] The disk is free to sit there bored for arbitrary periods of time before it does anything, but that's fine, because the OS is behaving correctly. Sigh.

[4] Dear filesystem writers - application developers like writing lots of tiny files, because it makes a large number of things significantly easier. This is fine because sheer filesystem performance is not high on the list of priorities of a typical application developer. The answer is not "Oh, you should all use sqlite". If the only effective way to use your filesystem is to use a database instead, then that indicates that you have not written a filesystem that is useful to typical application developers who enjoy storing things in files rather than binary blobs that end up with an entirely different set of pathological behaviours. If I wanted all my data to be in oracle then I wouldn't need a fucking filesystem in the first place, would I?

It seems python-dateutil is the Python module that can bring some sanity to time handling in Python. I ve finally found a module that can these two tasks:

• parse dates in several formats
• provide machinery to make the standard datetime talk timezones

For the first item I was using mx.DateTime, but it seems dateutil s parser is very good too. The second item has been bugging me for a long time. You have the datetime module in the standard library providing nice object-oriented interfaces to date and time, but what good it is if you can t even use it to print a date in RFC822 format! At least now it s possible:

  from datetime import datetime
  from dateutil.tz import gettz
  print datetime.now(tz=gettz()).strftime('... %z %Z')

(Oh, sure, that simple task can be done directly with the time module. But if you re dealing with HTML templates, you really want to pass date objects around, and let the templates print them one or more times in whatever format they need, including %z or %Z. And with dateutil you can use any timezone, of course, not just the current one.) The dateutil module also offers interfaces to do relative time calculations (what date will it be in 47 days?), and some stuff for recurrence rules. Check it out!

One of the recent poor trends in mailing list discussions is to reply to a message with a comment such as FAIL or EPIC FAIL . The FAIL meme has been around for a while and actually does some in some situations, slate has a good article about it [1]. The first example cited in that article is that when Ben Bernanke and Henry Paulson testified before the Senate banking committee last month about Paulson s proposed bailout bill, a demonstrator in the audience held up an 8.5-by-11 piece of paper with one word scrawled on it in block letters: FAIL. . This is an effective form of political demonstration, short words generally work well on placards (if only because the letters can be larger and therefore read from a greater distance) and anyone can understand the meaning of FAIL in that context. There are some blogs dedicated to publicising supposed failures, failblog.org and ShipmentOfFail.com are two examples. I cite these as supposed failures because some of the pictures that they contain are obviously staged. It s basically an Internet equivalent of the Funniest Home Videos shows that I never watched because they were not particularly funny. So using the word FAIL on it s own can be an effective form of political protest and can be used for mildly amusing web sites. But where it falls down is when it s applied to a discussion that involves people who are from different cultures or have different levels of background knowledge - which covers most mailing list discussions. Something that might be obviously wrong to some people is often not obvious at all to others. For example being forced to reboot a computer for any reason other than a kernel upgrade seems obviously wrong to me (and to most people who use Linux or other Unix systems) but Windows users seem happy to reboot machines after applying patches or upgrades. So writing a message with FAIL as the only word in a discussion with Windows users would not be productive. It could however be reasonable to forward a link to a page on a Microsoft web site to Linux people for their amusement with FAIL as the only comment - anyone who would find the link in question amusing would require no more explanation. Sometimes when in a debate someone will write a message that only says FAIL , this is a very unconvincing argument that will not convince the opposition or any onlookers. Generally it seems that using FAIL in a discussion with other like-minded people when talking about someone outside your group for the purpose of amusement can be effective. But any other use is going to be a FAIL . As a more general rule single-word messages seem to have little value apart from certain limited situations. I have identified the following seven scenarios where a single word message is useful. Can anyone think of any others?

1. Code review - someone posts code (or design for code) and people who like it will write ACK or something similar.


2. Arranging a meeting - the question who wants to meet for lunch tomorrow has me as a valid answer.


3. Voting - yes and no are valid answers for a poll, but a mailing list or forum probably isn t the best place for it.


4. Citing an example to refute a claim - often a single word won t be a great response but may be adequate to prove a point.


5. Answering a request for a recommendation - if asked to recommend a laptop I might say Thinkpad or if asked to recommend a server I might say HP . Both those answers are poor (I recommend EeePC for netbooks and Dell for small/cheap servers), so while such an answer would be useful it would be below my usual quality standards for email (I prefer to write at least two paragraphs explaining why I recommend something).


6. Informing people that something has been done by replying to a request with the word Done .


7. Agreeing to a contract or proposal with OK or Yes .

Update: I added another two reasonable uses of single word messages,

• [1] http://www.slate.com/id/2202262/pagenum/all/

I finally decided that it's time for me to upgrade my Xen installation. It used to run etch with backported Xen, because the etch version was increasingly difficult to work with. I also acknowledge that some of the issues I've been having are simply caused by yours truly, but even still the Debian Xen installation is way too fragile to my taste. I've already considered installing XenServer Express locally and running the hosts on it. The big drawback has been that XenCenter (the tool that is used to manage XenServer) is windows only and it doesn't work with wine. So you can imagine my desperation... Anyway, the latest upgrade from etch to lenny was painful as usual. The first part went smoothly, bit of sed magic on sources.list and a few upgrade commands (carefully picking the Xen packages out of the upgrade set). So in the end I had a working lenny installation with backported Xen. Next I made sure that there was nothing major going on in my network (one of the virtual machines acts as my local firewall) and took a deep breath before upgrading the rest of the packages. I knew to be careful about xendomains -script which has reliably restored my virtual machines after reboot to a broken host so I had always ended up restarting my virtual machines after reboot. I carefully cleared XENDOMAINS_AUTO and set XENDOMAINS_RESTORE to false in /etc/default/xendomains so that the virtual machines would be saved but not restored or restarted on reboot. After the normal pre-boot checks I went for it. Oddly enough everything worked normally and the system came up after a bit of waiting. I checked the bridges and everything appeared normal, so it was time to try and restore a single domain to see that everything actually did work as planned.

Hydrogen:~# xm restore /var/lib/xen/save/Aluminium
Error: Device 0 (vif) could not be connected. Hotplug scripts not working.

Oof, Googling for the issue revealed that there were others that had suffered from the same problem on various different platforms the problems were caused by different things. One would assume that the problem is in the vif-bridge script that is mentioned in the xend-config.sxp file as the script that brings up the vif, but after many hours of tial and error and pointless googling (over gprs connection), I couldn't find any solution to the problem. It was time to call it a day (it was almost 3 am already...) During the night I had a new idea about the possible cause. What if the problem isn't in xend, but somewhere else. I fired up udevadm monitor to see what udev saw and it wasn't much. I'm not an expert with udev, but from previous encounters I had a vague feeling that there was supposed to be more events flying around. I wasn't able to pinpoint what was wrong so I decided to purge xen-utils, of which I had 2 versions installed: 3.2-1 and 3.0.2. I also removed everything related to xenstore. After reinstalling the current versions and restoring my configuration files the first host came up just fine. I still had problems resuming the virtual machines and I ended up rebooting them again, which was nothing new, but at least they were running again. In the end I don't know what was the actual cause for udev not handling the devices properly, but I'm happy to have them all running again. And I learned a valuable lesson of all this: udev is an important part of Xen, make sure it works properly.

I have a bit of money invested in stocks and funds, passively managed by my bank. I ve made a bit with risky speculations, and I ve lost about the same amount with the markets dive of the last 18 months. Net benefit: zero cash, a bit of experience. Time to move on I am not an interesting customer, so I was all the more surprised to get a call from an employee of the bank the other day. He identified himself as Crisis Manager or something similarly ridiculous, mentioned at least 20 times that he was a regular employee and wouldn t receive provisions, and he also kept assuring me that he s doing himself what he s suggesting to me: leveraging the cost-average effect. I turned him down, probably a bit too briskily. Let s investigate why. Banks are good at devising ingenious tricks to fool people; it s their business after all. One of those tricks is the so-called cost-average effect. It s awesome, they say, because it is the first product that adapts to the ever-changing market: when prices are up, it buys smaller numbers of shares, and in low times, it stocks up on them by buying larger quantities, automatically. Fancy, no? It isn t, because the entire idea is to get your signature on a contract that obliges you to pay a fixed sum each month, with which shares are then bought. Obviously, when prices are up, numbers you get for your money are low, and vice versa. Observe how well the banks manage to turn that basic rule into a selling point. They are good at that. In fact, that s probably all they re good at. Obviously , inflation is also factored in, so the monthly fixed sum actually increases by a projected 5% each year, which they call inflation protection . While this strategy investing fixed-sum installments rather than lump sums certainly has its benefits, and can level out the risk of throwing a large sum of money at the market, it is commonly considered a suboptimal strategy, especially when employed over longer spans of time. For more information, I refer you to the frequently cited article Nobody gains from dollar cost averaging analytical, numerical and empirical results , published in the 1992/1993 issue of the Financial Services Review journal. So why would the bank approach me with this offer ? Let s go hypothetical and check out Ashley s investment practices: Ashley bought a fund five years ago at 100 per share, and it s now down to 80 . The loss is thus 20%, a -4% factor (20% / 5 years) that affects the overall performance of the portfolio. Ashley doesn t really care much about the percentages. After all, the only thing that counts is the worth of the shares when sold right now. Everything else is history and only influences the size of the tears one weeps in times like this, which is a psychological malfunction, not a function of the financial markets crisis. The bank, however, does not like negative performance values. Positive percentages yield happy (blinded?) customers, who get much more enthusiastic (greedy?) to put more money into the giant machine that makes the banks tick which is still in the process of blowing up into their faces. But negative performance values cause investors to become conservative and leery. They d rather hold on to their hard-earned cash for fear of losing more. That s the flip-side of aforementioned psychological malfunction, and this is the core of the reason why some of the big investors get bigger even in times like this they do not succomb to the feeling . The banks, however, need cash, and loads of it, and while the state is busy printing notes, they take every chance to get more. Private clients are wonderful prey, as they are less informed about the system than investors. So, to get cash from the clients, the banks have to circumvent the psychological malfunction. They cannot boost the market value share, but there are two values that factor into a percentage, and the cost-average effect, sold as the best thing since sliced bread, is nothing but a way to affect the other number: the sale price, the divisor. If the fund stays below the original sale price (100 ), each time Ashley pumps more cash into the machinery each month, this divisor decreases, effectively decreasing the percentage value. In the long run, this yields happier customers, who are willing to put more money in, especially if the performance turns positive. The baseline is now lower, so a return to previous heights would correspond to larger performance values than ever before. Obviously, a positive market causes the baseline to grow, but the cost-average deals continues to pay off for the banks. Since Ashley put that signature on the contract, the bank has a guaranteed cash flow for the contract s duration, while Ashley wonders why the percentage remains dampened. Cost-average means decreasing the percentage, in good as well as in bad times. Just now, the bank doesn t have to work so hard to get even more money from you. It gets worse: usually, those contracts are embedded in some sort of life insurance deals with grand promises in 40 years to come, and over all that time, it ll be quite clear who wins and who loses. Scary, huh? I was going to end with the advice to stay away from the cost-average effect, but I think I can just as well make it more general: if you don t understand a deal, don t sign it. If you abide by that rule, you would stay away from cost-average effect deals all by yourself. Maybe my little writeup has helped, nonetheless. NP: DJ Ti sto: Live at Innercity

The Glest FAQFuckYou says (emphasis is mine):

[..] as a rule of thumb we don t integrate any patches in our mainline, since that would require us to check them first and then maintain them, however feel free to post them at the board.

Glest development sucks. I really hope Glest Advanced Engine effort takes speed and replaces Glest.

This is the main reason I don't feel any incentive to maintain this package in Debian.

I had an existential experience in my local drug store last night while pondering this sign. What does it mean for pain to be truly external to the person feeling it? Have I ever felt external pain? Is external pain merely another term for empathy? What might products to help with empathy entail? Would my local drug store stock them?

ROTFL quote (especially for geeks emigrated to France, like yours truly): normaliser unix c'est comme pasteuriser le camembert Found at the bottom of a pleasant announcement.

Some necessary background: George R.R. Martin is a science fiction and fantasy author who's been writing (and editing) in the field for quite a while. In 1996, he started a new epic fantasy series entitled The Song of Ice and Fire, a hugely complex and sprawling series telling the story of an entire medieval world, with four volumes published and something on the order of 19 viewpoint characters to date. The fourth and last book to date in the series was published in 2005. It contained only half the viewpoints expected (and in my opinion and that of at least some other readers, the least interesting half). There was an afterword saying basically that the book was cut somewhat artificially in half for length reasons (it was 753 hardcover pages for what was there), the next book would have the other viewpoints, and it was basically done. Three years and some later, and after a few other optimistic updates, it's not yet done. Some of Martin's fans are increasingly getting antsy and sometimes abusive about this. This has increasingly been getting to Martin, and he's responded in various ways in his personal blog, most recently February 19th. See also John Scalzi and Charlie Stross reacting to his post. Now, I'm not one of the people who is rabid about this series. I'm a little disappointed that I haven't gotten to read the next bits with the characters I really like in the series, and I didn't much care for A Feast for Crows because of how the book was split. But I have a thousand other books I want to read (literally I've read less than half the books I own) and if I don't get to see more of the series, oh well. I am, however, very interested in the fan reaction to this delay and Martin's reaction to that, because it has started reminding me of similar things in my own life. Some of the fans are just batshit insane, which is one of the things that happens to a best-selling author; I feel for Martin there but there isn't much anyone can really do about that. But among the sane people, there are a bunch who say something like "I understand why the constant nagging doesn't help, but on the other hand, when you start a big story with unresolved plots, you have an obligation to your readers to finish it." I call bullshit on that. This same effect happens in the free software world. If you write some program, you're perceived to have an obligation to support it, fix it, release new versions, and so forth. I've also had this happen in other volunteer contexts, such as Usenet. Often that obligation is based on some comments that one makes about future plans, things that will go into the next release, e-mail messages where you say that you'll look at something, and so forth. It's not entirely groundless, and it's not groundless with Martin either. However. As any software developer knows, time estimates for tasks are horribly difficult and almost always too short. This is clearly what happened to Martin, and bitching about the time estimates doesn't help at all. Some problems turn out to be so much harder than you expect that time estimates fail entirely. With creative work (including some programming) it's particularly hard to know how much work is remaining. You may know roughly how much you have left to write, but you don't know how hard each chunk will be until you start it, and sometimes until you finish it. With some projects, the only thing you can do is say "sorry, my estimates were completely off and I don't know how to estimate this" and then stop promising dates. Which Martin has already done. He was "wrong" in some sense for setting bad expectations, but estimation is insanely hard and the world usually doesn't let you get away with just refusing to estimate anything. So much for time. What about whether Martin has an obligation to write the next book at all? The equivalent question in the free software world is whether one can just walk away from a project, or away from a volunteer position, leaving a bunch of stuff undone that some people who were using the software were counting on having finished. It's a bit easier to deal with in free software, since usually just promises are at stake, whereas with Martin he's sold a commercial product and people get weird about money. There is a reasonable expectation that a series of this type will have a conclusion (although I'll point out that the SFF world is filled with unfinished series). Maybe you wouldn't have bought any of the series if you had known that it wouldn't end, so now you're out $100 for the hardcovers that you feel like you bought on false pretenses. Let's examine that. First, I have a serious problem with people who use the economic transaction as leverage in this sort of argument. This is a really piddly amount of money for each individual person, generally less than I get paid for one hour of work. Writing a new novel is way, way more than one hour of work. Martin's fans are demanding that he do work that's worth many orders of magnitude more than they're conceivably out monetarily. (And the reactions are individual reactions of anger; the level of upsetness of each individual shouldn't change based on the total size of the fan base.) Furthermore, their economic transaction was with the publisher, and they had no control and no say over how Martin got paid. From multiple different directions, they're not his employer, have no contractual control over how he spends his time, and have no legitimate economic basis for asserting any such control. They can make a decision to not buy his future work because they're disappointed, and that's it. Second, it is absolutely mandatory to give other people room to be wrong or change their mind. Human beings cannot keep every promise that they make. We fail at things that we fully intend to do, particularly around estimates, and particularly around one's dedication to decade-long major life projects. People are wrong about marriages and children, which involve far more serious consequences than a fiction series. I think it's a good reality check here to note that his fans are placing way more restrictions on what he can do with his life than would be legal for any employer. Third, when someone doesn't meet your expectations, the response should be proportional and rational. This is the one that really gets to me. Choosing not to read the next book in the series because it's been too long is entirely reasonable. Trying to make Martin feel like crap is not. I would go further: nothing that an artist could promise to their general public could warrant intentionally making them feel like crap for failing to do it. That level of emotional consequence should be reserved for things like marriage, parenting, or life-long friendship, not disappointment that you didn't get to hear the end of a story you were enjoying. I speak from personal experience here: this type of punishment by guilt is a nasty, nasty business. It burns people out faster than any other possible reaction, it makes it much less likely that Martin will continue the series at all, and it's probably poisoning his feelings about the entire world. It's burning down the bridge you're standing on. It always hurts and never helps. People already create feelings of obligation around things they've promised, implicitly or explicitly, to do, and generate plenty of their own guilt. That guilt is one of the major obstacles that one has to overcome to be productive; if you don't believe me, read any good productivity book, such as Allen's Getting Things Done, and notice how much it talks about getting past guilt. Intentionally going out of your way to contact someone you do not know personally and pile on more guilt is vicious, cruel, and tactically counterproductive. Trust me, Martin didn't fail to notice that he missed his own deadlines. I think Martin's post and the subsequent discussion is a great opportunity for all of us who create or use creative and technical works to think about how we apply the Golden Rule. There's a natural tendency to hold people tightly to promises that we'll get something that we want. There's a natural tendency for all of us to make those promises, since people are grateful for them, and then pile guilt on ourselves for not meeting them. All of the emotion wrapped up in both sides of that tension of expectation and obligation can be exhausting and poisonous. It makes relationships brittle because we don't let people renegotiate. We don't let people say they were wrong. We don't let people change their minds. And we don't let ourselves change our minds. Here's my promise: the next time someone doesn't meet an expectation I had, I will remember this discussion, give them space to be wrong, and not pile on more guilt. Like all promises, I'll try to keep it, but I might not be able to.