A recent Ubuntu bug has gotten slashdotted, and has started raising a lot of questions about the safety of using ext4. I ve actually been meaning to blog about this for a week or so, but between a bout of the stomach flu and a huge todo list at work, I simply haven t had the time. The essential problem is that ext4 implements something called delayed allocation. Delayed allocation isn t new to Linux; xfs has had delayed allocation for years. Pretty much all modern file systems have delayed allocation, according to the Wikipedia Allocate-on-flush article, this includes HFS+, Reiser4, and ZFS; btrfs has this property as well. Delayed allocation is a major win for performance, both because it allows writes to be streamed more efficiently to disk, and because it can reduce file fragmentation so that later on they can be read more efficiently from disk. This sounds like a good thing, right? It is, except for badly written applications that don t use fsync() or fdatasync(). Application writers had gotten lazy, because ext3 by default has a commit interval of 5 seconds, and and uses a journalling mode called data=ordered. What does this mean? The journalling mode data=ordered means that before the commit takes place, any data blocks are
associated with inodes that are about to be committed in that transaction will be forced out to disk. This is primarily done for security reasons; if this is not done (which would be the case if the disk is mounted with the mount option data=writeback), then any newly allocated blocks might still contain previous data belonging to some other file or user, and after a crash, accessing that file might result in a user seeing unitialized data that had previously belonged to another user (say, their e-mail or their p0rn), which would be a Bad Thing from a security perspective. However, this had the side effect of essentially guaranteeing that anything that had been written was guaranteed to be on disk after 5 seconds. (This is somewhat modified if you are running on batteries
and have enabled laptop mode, but we ll ignore that for the purposes of this discussion.) Since ext3 became the dominant filesystem for Linux, application writers and users have started depending on this, and so they become shocked and angry when their system locks up and they lose data even though POSIX never really made any such guarantee. This become especially noticeable on Ubuntu, which uses many proprietary, binary-only drivers, which caused some Ubuntu systems to become highly unreliable, especially for Alpha releases of Ubuntu Jaunty, with the net result that some Ubuntu users have become used to their machines regularly crashing. (I use bleeding edge kernels, and I don t see the kind of unreliability that apparently at least some Ubuntu users are seeing, which came as quite a surprise to me.) So what are the solutions to this? One thing is that the applications could simply be rewritten to properly use fsync() and fdatasync(). This is what is required by POSIX, if you want to be sure that data has gotten written to stable storage. Some folks have resisted this suggestions, on two grounds; first, that it s too hard to fix all of the applications out there, and second, that fsync() is too slow. This perception that fsync() is too slow was most recently caused by a problem with Firefox 3.0. As Mike Shaver put it:

On some rather common Linux configurations, especially using the ext3 filesystem in the data=ordered mode, calling fsync doesn t just flush out the data for the file it s called on, but rather on all the buffered data for that filesystem.

Fundamentally, the problem is caused by data=ordered mode. This problem can be avoided by mounting the filesystem using data=writeback or by using a filesystem that supports delayed allocation such as ext4. This is because if you have a small sqllite database which you are fsync(), and in another process you are writing a large 2 megabyte file, the 2 megabyte file won t be be allocated right away, and so the fsync operation will not force the dirty blocks of that 2 megabyte file to disk; since the blocks haven t been allocated yet, there is no security issue to worry about with the previous contents of newly allocated blocks if the system were to crash at that point. Another solution is a set of patches to ext4 that has been queued for 2.6.30 merge window. These three patches (with git id s bf1b69c0, f32b730a, and 8411e347) will cause a file to have any delayed allocation blocks to be allocated immediately when a file is replaced. This gets done for files which were truncated using ftruncate() or opened via O_TRUNC when the file is closed, and when a file is renamed on top of an existing file. This solves the most annoying set of problems where an existing file gets rewritten, and thanks to the delayed allocation semantics, that existing file gets replaced with a zero-length file. However, it will not solve the problem for newly created files, of course, which would have delayed allocation semantics. Yet another solution would be to mount ext4 volumes with the nodelalloc mount option. This will cause a significant performance hit, but apparently some Ubuntu users are happy using proprietary Nvidia drivers, even if it means that when they are done playing World of Goo, quitting the game causes the system to hang and they must hard-reset the system. For those users, it may be that nodelalloc is the right solution for now personally, I would consider that kind of system instability to be completely unacceptable, but I guess gamers have very different priorities than I do. A final solution which might not be that hard to implement would be a new mount option, data=alloc-on-commit. This would work much like data=ordered, with the additional constraint that all blocks that had delayed allocation would be allocated and forced out to disk before a commit takes place. This would probably give slightly better performance compared to mounting with nodelalloc, but it shares many of the disadvantages of nodelalloc, including making fsync() to be potentially very slow because it would force all dirty blocks to be forced out to disk once again. What s the best path forward? For now, what I would recommend to Ubuntu gamers whose systems crash all the time and who want to use ext4, to use the nodelalloc mount option. I haven t quantified what the performance penalty will be for this mode of operation, but the performance will be better than ext3, and at least this way they won t have to worry about files getting lost as a result of delayed allocation. Long term, application writers who are worried about files getting lost on an unclena shutdown really should use fsync. Modern filesystems are all going to be using delayed allocation, because of its inherent performance benefits, and whether you think the future belongs to ZFS, or btrfs, or XFS, or ext4 all of these filesystems used delayed allocation. What do you think? Do you think all of these filesystems have gotten things wrong, and delayed allocation is evil? Should I try to implement a data=alloc-on-commit mount option for ext4? Should we try to fix applications to properly use fsync() and fdatasync()? Related posts (automatically generated):

1. Don t fear the fsync! After reading the comments on my earlier post, Delayed allocation...
2. SSD s, Journaling, and noatime/relatime On occasion, you will see the advice that the ext3...
3. Ext4 is now the primary filesystem on my laptop Over the weekend, I converted my laptop to use the...

If you code for long enough on x86-64, you'll eventually hit an error such as: Here's a little example that might help you figure out what you've done wrong. Consider the following code: In case it's not clear, that would look something like: Let's build that code, and see what it looks like We can see that the mov instruction has only allocated 4 bytes (00 00 00 00) for the linker to put in the address of foovar. If we check the relocations: The R_X86_64_32S relocation is indeed only a 32-bit relocation. Now we can tickle this error. Consider the following linker script, which puts the foo section about 5 gigabytes away from the code. This now means that we can not fit the address of foovar inside the space allocated by the relocation. When we try it: What this means is that the full 64-bit address of foovar, which now lives somewhere above 5 gigabytes, can't be represented within the 32-bit space allocated for it. For code optimisation purposes, the default immediate size to the mov instructions is a 32-bit value. This makes sense because, for the most part, programs can happily live within a 32-bit address space, and people don't do things like keep their data so far away from their code it requires more than a 32-bit address to represent it. Defaulting to using 32-bit immediates therefore cuts the code size considerably, because you don't have to make room for a possible 64-bit immediate for every mov. So, if you want to really move a full 64-bit immediate into a register, you want the movabs instruction. Try it out with the code above - with movabs you should get a R_X86_64_64 relocation and 64-bits worth of room to patch up the address, too. If you're seeing this and you're not hand-coding, you probably want to check out the -mmodel argument to gcc.

I've just uploaded Policy 3.8.1.0 with the first batch of post-lenny changes. There isn't anything that earth-shattering in this set, but I'm hoping that we'll be able to get a lot more done during the squeeze release cycle. Policy has well over a hundred open bugs, quite a few of which we should be able to resolve. I also uploaded Lintian 2.2.8, which is synchronized against Policy 3.8.1 and also reverts some changes around install-menu due to a reverted change in debhelper 7.2.5, and fixes diagnosis of diversions. The last few versions of Lintian have all been a little bit buggy due mostly to the current pace of change. Hopefully this version will have fewer unexpected problems. Here is the summary of normative Policy changes, taken from upgrading-checklist.txt.gz in the debian-policy package.

• Care should be taken when adding functionality to essential and such additions create an obligation to support that functionality in essential forever unless significant work is done.
• Changelog files must be encoded in UTF-8.
• Tighten some format requirements for changelog files from a should to a must.
• Remove alternative changelog formats. Debian only supports one changelog format for the Debian Archive.
• New nocheck option for DEB_BUILD_OPTIONS indicating any build-time test suite provided by the package should not be run.
• All control files must be encoded in UTF-8.
• debian/control allows comment lines starting with # with no preceding whitespace.
• Init scripts ending in .sh are not handled specially. They are not sourced and are not guaranteed to be run by /bin/sh regardless of the #! line. This brings Policy in line with the long-standing behavior of the init system in Debian.
• The start action of an init script must exit successfully and not start the daemon again if it's already running.
• /var/run and /var/lock may be mounted as temporary filesystems, and init scripts must therefore create any necessary subdirectories dynamically.
• /bin/sh scripts may assume that local can take multiple variable arguments and supports assignment.
• User mailboxes may be mode 600 and owned by the user rather than mode 660, owned by user, and group mail.

Nearly a month has elapsed since Debian released Lenny. The interesting thing for me about this release was that I failed to get to the point where running etch was causing me pain. This became even more evident when Katherine commented on my Lenny T-Shirt - I'd forgotten to mention our release to her! Previously there have been a bunch of features coming that I really wanted, sometimes to the point of using backports. It's probably something to do with the way I run Debian; I use stable for remote machines and while in the past this would have meant quite a few work boxes these days it's just a box at my parents' and my colo. Where I felt the pain during this release cycle was on the boxes where I run unstable (my home desktop) or testing (my work box and my laptops). Development on new shininess slows down significantly while we're preparing for release (not unsurprisingly). And there are things coming up I quite like the look of, largely aimed at the desktop arena. I've been running Network Manager 0.7 from experimental on my EEE, for example - I need the HSO 3G support to avoid unnecessary faff on a daily basis. I've got ATI cards in my work and home desktops, so I'm watching the Mesa developments with interest - roll on OpenGL 2.0 support hopefully. The Intel KMS support that doesn't seem to be quite there yet but RSN, no, really, will be nice for my laptops. Experimental VDRs are looking interesting and perhaps worth upgrading my PVR to.

For a long time I haven't really understood the point of view that Debian shouldn't release. Or rather, I haven't understood why people would think that was a serious option. The first Debian box I installed went into a colo facility shortly afterwards. This is the first release I've not been running Debian on a commercial colo box and while I treat my personal colo in a similar fashion it's not quite the same in terms of priorities. However I still think regular releases are hugely important to the project. Some pain for those of us more interested in new shiny is a small price to pay for producing a flexible, reliable distribution, be they server administrators (though I see a worrying number of people decide to install testing on their production servers and then rarely/never update it), derivers or appliance vendors.

Mind you, now we've released I'm going to have to do all those things I've been putting off. I should probably start with moving my Perl bits into pkg-perl. And maybe I should switch over some sources.list files to point at squeeze instead of lenny...

Things are feeling unstable around here. That's mostly a good thing, but ... Switching to ext4 seemed like a good idea since I was resizing my file server's filesystem anyway. And I'd used ext4 on my laptop for a while, with no problems. But, I ran into a bug that causes ext4 to hang. So I'm left with my main server at home piling up hundreds of processes in uninterruptible sleep when this breaks. And, eventually running out of memory and dying. There's a fix, in newer -pre kernels, but so far I have not had luck getting those kernels to work with my Thecus. There's a regression in those kernels that breaks the NIC, and my attempt at backing that changeset out still left it unable to get on the net. Did I mention that I don't even have a serial console on this machine? That natively building a kernel on armel takes about 12 hours? Makes this sort of thing much fun. Need to find the right header and spend some time with a soldering iron one day. Converting back to ext3 would mean resizing a 2 TB array, which ran resize2fs out of memory when I tried it. It looks like I will have to leave /home unmounted while I'm on my trip, to keep the rest of the server up. When I get back, I've ordered a third terabyte drive, and will use it to switch back to ext3. Having these silly amounts of storage is fun, until you have to admin it. The other fun I've run into is that a new version of mpd has horrible performance on arm. Max Kellermann was very helpful in tracking down the issues involved, and helping me get a version that has at least reasonable performance again. Oh, and if someone could tell me why gnome terminal 2.24.3 can't open links anymore, I'd appreaciate it.

With Debconf9 being on the european mainland this year, a plane flight isn't necessary. Since flying in cattle class isn't my idea of 'fun', I had been planning to take the train to Caceres this year. So, with the public announcement of debconf registration being open, I thought I'd look at train options. Short version: I think I'll be going to the train station and ask them there, rather than looking something up on the NMBS website. Long version:

Trajectory	Date	Departure	Arrival	Train
Mechelen -> Brussels North	2009-07-13	07:02	07:18	IR 3128
Brussels North ->K ln Hbf	2009-07-13	07:27	09:15	ICE 11
K ln Hbf ->Mannheim Hbf	2009-07-13	09:54	11:24	ICE 515
Mannheim Hbf ->Basel SBB	2009-07-13	11:36	13:47	ICE 375
Basel SBB ->Arth-Goldau	2009-07-13	14:03	15:45	IR 2177
Arth-Goldau -> Milano Centrale	2009-07-13	15:50	18:50	CIS 21
Milano Centrale ->Barcelone Estacion de Franca	2009-07-13/14	19:40	09:43	EN 11274
Barcelona Estacion de Franca -> Barcelona Sants	2009-07-14	09:43	10:28	Transfer
Barcelona Sants -> Madrid-Puerta de Atocha	2009-07-14	12:00	15:23	AVE 3122
Madrid-Puerta de Atocha	2009-07-14	16:40	20:02	TLG 194

And that's after asking to arrive 'around' noon on the 15th. Although I'd love to combine ICE ('Intercity Express', German high-speed trains), CIS ('Cisalpino', Italian high-speed trains), EN ('EuroNight', pretty much what the name says), and AVE ('Alta Velocidad Espa a', Spanish high-speed trains) in a single trip, I think I'll pass for this particular suggestion. And that's ignoring the fact that 5 minutes for a transfer in Arth-Goldau (a place I've never heard of before, let alone been in that train station) is rather tight, and that I don't know whether I'll be able to make it in 45 minutes from one train station to another in Barcelona without actually knowing the city. More soon.

Yesterday evening, I registered for Debconf9 in C ceres. I also registered my first proposed event: the now traditional cheese and wine party. Being in Spain with no regulations on food import for the traditional cheese providers in the Debconf crowd, I promise that this cheese party will be E.V.I.L. And Debconf 9 will be a great success, of course. See you in C ceres!

I was not in a work mood tonight and then decided to lose my time reading about timezones (thanks to a Chinese user who wants to add Beijing to the list of timezones for China...which has only one, including the colonies of Tibet and Xinjiang). The list given on Wikipedia is a completelt hilarious time waster, indeed. My favourite one is probably the mention then some parts of British Columbia (Canada) are in UTC-7, namely Most of Peace River Regional District except Fort Ware, Pink Mountain as well as Regional District of East Kootenay and finally (take a breath!) Regional District of Central Kootenay east of the Kootenay River and parts east of Kootenay Lake that are south of and including Riondel (Creston doesn't observe DST), and Columbia-Shuswap Regional District east of the Selkirk Mountains. Canada timezones are really a gem, where one learns that Northwest Territories (one of the largest states) follows UTC-7 except a town named Tungsten and.....a tungten mine. Some other gems in that page are the places that leave in the future at UTC+13 and UTC+14, in Tonga and Kiribati. A other one? Arizona is also funny with the entire state being UTF-7, but the Navajo nation not observing DST, but the Hopi reservation (that's an enclave of the Navajo nation) observing it. That gives an interesting nightmare when travelling over there in summer. And finally, living in Florida probably requires a very high intelligence to understand what time it can be when "east of the Apalachicola River, plus the portions of Franklin County and Gulf County south of the Intracoastal Waterway, west of the Apalachicola River" uses UTC-5 while "west of the Apalachicola River, except for the portions of Franklin County and Gulf County south of the Intracoastal Waterway" use UTC-6. There are other such hilarious thingies on that page. I hope you'll enjoy it as much as I did.

Translink are breaking the trains for 3 months at the end of March. This is manifesting as the line between Ballymena and Coleraine being entirely shut (it's only single track, so it's not like they can do one side at a time), but will hopefully result in a smoother, faster journey. Not much good for the 3 months when my journey goes from a single train ride to a train, bus, then another train ride. And gets longer by 20-30 minutes (so potentially 5 hours a day commuting. Yay!). I think not.

So I have rented a flat in Belfast, conveniently only 3 minutes walk from work. 3 month leases are hard to find so I have it for 6 months. I'm a bit freaked. I shouldn't be really; I lived alone when Katherine first moved back to NI. But that was in a house I'd been in for ages rather than somewhere entirely new to me. It proved its worth last night when we were able to walk to Do You Remember the First Time? in a couple of minutes though. Being in the city centre is damn handy, though I'm having to get used to the regular noise of traffic again. :)

(This post primarily a test of Drivel; it's not worth the hassle of dealing with Virgin or BT to get a broadband connection, so I'm going to end up continuing to use my Orange 3G dongle and trying to do things in local clients where possible.)

Quick tip: you can access the iPhone-tailored version of Google Reader from any computer, using the URL http://google.com/reader/i/. I quite like using it from a regular desktop machine.

It took us a couple of weeks to organise, but we had a small Lenny release party in Cambridge last weekend. We had the usual crowd of Cambridge folks, plus Noodles and codehelp. Jason Clifford from UKFSN even threw some cash our way to help cover the costs - Thanks Jason! :-) We started at the Regal pub in town, then headed back to my place and drank until late. Quite a number of the revellers also bought some of our shiny new Lenny release T-shirts! If you'd like one, look at the details here and mail me! Update: Fixed the URL to the T-shirt photo. Doh!

So I am going to do something that nobody on the Internet is doing lately: post a review of the Kindle 2 after having only used it for three days. Shocking, yes, I know. I had never even seen a Kindle of either model before getting the Kindle 2. I had, though, thought about getting an eInk device for some time. The $359 Kindle 2 price tag caused me significant pause, though in the end I went for it due to the 30-day return policy. On the surface, I thought that it would be weird to have a Kindle. After all, how often am I away from the computer? And there s a small local library a few blocks from where I work. But I had a hunch it might turn out for me like my iPod did: something that didn t sound all that useful from reading about it, but turned out to be tremendously so after having actually used it. Turtleback Delivery I ordered my Kindle 2 with standard shipping, which meant that it went by FedEx Smart Post. Here is my SmartPost rant. There are two words in Smart Post that are misleading. I once had an item take literally a week to make it from St. Louis to Kansas. That is, I kid you not, slower than the Pony Express ran in 1860. This time, my Kindle made it from Kentucky to Kansas in a mere five days. Oh, and it spent more than 24 hours just sitting in St. Louis. The Device Overall, the device is physically quite nice. It is larger and thinner than I had imagined, and the screen also is a bit smaller. It is usually easier to hold than a paperback, due to not having to prevent it from closing too far at the binding edge. The buttons are easy to press, though I could occasionally wish for them to be easier, but that s a minor nit. The Screen The most important consideration for me was the screen. The eInk display as both stunningly awesome and disappointing. This is not the kind of display you get on your computer. Or, for that matter, any other device. It isn t backlit. It reacts to light as paper does. It can be viewed from any angle. And it consumes no power to sustain an image; only to change it. Consequently, it puts up a beautiful portrait of a famous author on the screen when it is put to sleep, and consumes no power to maintain it. The screen s response time isn t anywhere near as good as you d expect from a regular LCD. It flashes to black when you turn a page, and there is no scrolling. On the other hand, this is not really a problem. I found the page turning speed to be more than adequate, and probably faster than I d turn the page on a real book. The resolution of the display has the feeling of being incredible. The whole thing provides a far different, and in my eyes superior, experience to reading on an LCD or CRT screen. My nit is the level of contrast. The background is not really a pure white, but more of a light gray. This results in a contrast level that is quite clearly poorer than that of the printed page. At first I thought this would be a serious problem, though I am growing somewhat more used to it as I read more. Reading Experience Overall, I ve got to say that it is a great device. You can easily get lost in a book reading it on the Kindle. I m reading David Copperfield for the first time, and have beat a rather rapid path through the first five chapters on the Kindle already. And that, I think, is the best thing that could be said about an ebook reader. It stays out of the way and lets you immerse yourself in your reading. The Kindle s smartly-integrated Oxford-American Dictionary was useful too. One thing about a novel written 150 years ago is that there are some words I just haven t ever heard. Nosegay, for instance. You can move a cursor to a word to see a brief pop-up definition appear, or press Enter to see the entire entry. This is nice and so easy that I m looking up words I wouldn t have bothered to if I were reading the book any other way. A nosegay, by the way, is a bouquet of showy flowers. Buying Experience The Kindle has a wireless modem tied to the Sprint network on it. The data charges for this, whatever they may be, are absorbed by Amazon in the cost of the device and/or the books you buy for it. This turned out to be a very smart piece of engineering. I discovered on Amazon s Kindle Daily Post that Random House is offering five mostly highly-rated sci-fi books for free on the Kindle for a limited time. So I went over to the page for each, and made my purchase . It was only a click or two, and I saw a note saying it was being delivered. A few minutes later, I picked up the Kindle off the kitchen counter. Sure enough, my purchases were there ready to read. Impressive. This level of ease of use smells an awful lot like Apple. Actually, I think it s surpassed them. You can delete books from the Kindle and re-download them at any time. You can initiate that operation from either the PC or the Kindle. And you can also browse Amazon s Kindle store directly from the device itself. I haven t subscribed to any magazines or newspapers, but I gather that they deliver each new issue automatically the moment it s released by the publisher, in the middle of the night. I pre-ordered the (free to Kindle) Cook s Illustrated How-to-Cook Library. It makes me way happier than it should to see This item will be auto-delivered to your Kindle on March 26 in the order status. Free Books Amazon s Kindle library has a number of completely free Kindle books as well. These are mostly out-of-copyright books, probably sourced from public etext places like Project Gutenberg, and converted to the Mobipocket format that is the Kindle s native format with a minimum of human intervention. As they are free, you can see them in Amazon s library if you sort by price. And, of course, Amazon will transfer them to the Kindle wirelessly, and maintain a copy of them in your amazon.com account. Unfortunately, as with free etexts in general on the Internet, the quality of these varies. I was very annoyed to find that many free etexts look like they were done on a typewriter, rather than professionally printed. They don t use smart quotes; only the straight ones. When reading on a device that normally shows you a faithful print experience, this is jarring. And I spent an inordinate amount of time trying to find a copy of Return of Sherlock Holmes that actually had the graphic figures in Dancing Men. Ah well. Your Own Content Amazon operates a mail server, username@kindle.com. You can email stuff to it, and it will convert it to the Kindle format and wirelessly upload it to your kindle for a fee of $0.10. Alternatively, you can use username@free.kindle.com, which does the same thing at no charge, but emails you back a link to download the converted work to install via USB yourself. I tried it with a number of PDFs. It rejected about a dozen times from only my single mail message a PDF containing graphic images only. However, it does quite well with most text-heavy PDFs notably doing an excellent job with Return of Sherlock Holmes from bookstacks.org the only source I found that was both beautifully typeset and preserved the original figures. Unfortunately, the PDF converter occasionally has troubles identifying what should be a paragraph, particularly in sections of novels dealing with brief dialog. I have also sent it some HTML files to convert, which it also does a great job with. You can also create Mobipocket files yourself and upload them directly. There is a Mobipocket creator, or you can use mobiperl if you are Windows-impaired or prefer something scriptable on Linux. The device presents itself as a USB mass storage device, so you can see it under any OS. There s a documents folder to put your files in. You can back it up with your regular backup tools, too. And it charges over USB. Web Browser I haven t tried it much. It usually works, but seems to be completely down on occasion. It would get by in a pinch, but is not suitable for any serious work. The guys over at XKCD seem to love it; in fact, their blog post was what finally convinced me to try the Kindle in the first place. Final Thoughts I ve ordered a clip-on light and a leather case for the Kindle. The light, I believe, will completely resolve my contrast complaint. The leather case to protect it, of course. I can t really see myself returning the Kindle anymore. It s way too much fun, and it s making it easier to read more again. And really, if Amazon manages to reach out to a whole generation of people and make it easy and fun for them to read again and make a profit doing it, of course they may move up a notch or two from being an evil patent troll company to a positive social force company. Wow, never thought I d say that one.

It's crazy to think that this is just past the one-year anniversary of my surgery. A year out and I can lift about 12kg (25lbs for the Americans and Iranians), I'm not on routine or regular pain killers of any sort. I still have to take it easy and need to make sure I get to the gym more often - when I don't I start to get pain and have trouble keeping up with Leif. I have discovered that I can't sleep on airplanes anymore, even in Air Canada business class. So, no more red-eyes for me. It's hard to shed a tear over that one. =) I suppose I have a lot of little limitations like that, but a year in I don't notice them so much anymore.

I shudder a bit to think that at this time last year, I was just home from the hospital. A long walk for me was a trip to the mailbox: I was only up for it every couple of days, and it took me about 30 minutes, including stops for sitting.

Thanks for your love and support, everyone.

yesterday i went to a punk concert, drank some beers, danced some pogo, fell on the slippery floor and crashed my knee once again. still i dont regret it (yet?), it was great to feel alive and young again, though its the perfect proof i'm not young anymore. but it was definitly worth it, i saw them the first time in 1997 or 98 and yesterday was their goodbye concert. wunderbar.
not sure if i should really blog this..

I ve been working as of late with psycopg2, the Python bindings for PostgreSQL. Documentation is a bit, erm, lacking. You can get the basic stuff working by reading Python s DBAPI-2.0 specification, since psycopg2 follows it, but (as I ve only recently realized) discovering the extra features that exist needs you to poke the source. The other day I was wanting to pass to the execute method of a cursor a sequence to fill the value list in a SQL IN statement. The DBAPI spec does not mention this usage at all, and a cursory question in #python told me it wasn t supported (required?) by the spec. It is, in any case, supported in psycopg2. You just need to

  import psycopg2.extensions

and it just works: you can pass tuples, and they ll be converted to a value list. If you want to be able to pass lists as well:

  from psycopg2.extensions import SQL_IN, register_adapter
  register_adapter(list, SQL_IN)

There is also a row factory in psycopg2.extras that offers a dictionary interface, instead of tuple-based. Finally, I have no idea why, but when I started with psycopg2, I came to get the idea that only named parameter substitution was supported (INSERT ... VALUES (%(var1)s, %(var2)s)), but simple %s substitution is also supported: just pass (of course) a tuple as the second parameter to execute, instead of a dictionary.

Hi, I m looking for a tool to create simple websites (5-10 pages), only static content (no PHP). Think of the typical small business or sports club website: text + pictures about what they did/do, nothing really complex. Requirements:

usable by a non-programmer who doesn t want to know about gory technical details (HTML, CSS, etc)

doesn t make it too hard to create a website with a few pages (i.e it should do a bit more than just allowing to create one page)

built-in support for (S)FTP uploading of the website, usable by a non-programmer after someone computer-litterate will have set it up. (that s not an absolute requirement, but would still be nice to have. also, what are the nice graphical tools to do that, currently?) I m asking for two different persons with different needs. So please don t limit your replies to free software (although it would be nice): cheap proprietary applications might be adequate as well. Regarding platform, Linux, Mac OS X or Windows would all be suitable. I don t think that a CMS is the right tool here: the focus should be put on the content (which won t be often modified anyway) and its presentation, not on the ease of modification. For example, I d like to be able to position pictures in a precise way. However, I might be interested in a CMS that is:

easy to setup (for a reasonably computer-literate person)

extremely easy to learn (a couple hours at most, for a non-programmer)

easy to transform into something that looks good In your comments, it would be great if you could include examples of what you did with those tools (yes, links are welcomed!) Thank you.

It s been quite the week. Stomach Flu Last Friday, my stomach was just starting to feel a little odd. I didn t think much off it a little food that didn t go over well or stress, I thought. Saturday I got out of bed and almost immediately felt like throwing up. Ugh. I probably caught some sort of stomach flu. I was nauseous all day and had some terrible diarrhea to boot. I spent parts of Saturday, Saturday night, Sunday, and Sunday night supervising some emergency downloads as the BOFH would say. By Sunday afternoon, I thought I was doing good enough to attend a practice of the Kansas Mennonite Men s Choir. I made it through but it wasn t quite as up to it as I thought. Monday morning I woke up and thought the worst was behind me, so I went to work. By evening, the worst clearly was not behind me. I was extremely cold, and then got very hot a few hours later. Tuesday I left work a little early because of not feeling well. Servers Wednesday a colleague called me at home before I left to say that the ERP database had a major hiccup. That s never good. The database is this creaky old dinosaur thing that has a habit of inventing novel ways to fail (favorite pastime: exceeding some arbitrary limit to the size of files that no OS has cared about for 5 years, then hanging without telling anybody why). My coworkers had been working on it since 5. I went into the office and did what I could to help out, though they had mostly taken care of it. Then we went to reboot the server. It didn t come back. I/O error on sda just after init started, and it hung. Puzzled, as it just used that disk to boot from. Try rebooting again. This time, I/O error as the fibre channel controller driver loads. Again, puzzled as it just used that controller to load grub. Power cycle this time. And now the server doesn t see the fibre channel link at all. Eep. Check our fiber optic cables, and power cycle again. And THIS time, the server doesn t power back up. Fans whir for about a second, then an ominous red light I never knew was there shows up. Eeep! So I call HP. They want me to remove one CPU. Yes, remove one CPU. I tried, and long story short, they dispatch a local guy with a replacement motherboard. Can you send along a FC controller, in case it s dead too? Nope, not until we diagnose a problem with it. Local guy comes out. He s a sharp guy and I really like him. But the motherboard wasn t in stock at the local HP warehouse, so he had to have it driven in from Oklahoma City. He gets here with it by about 4:30. At this point the single most important server to the company s business has been down almost 12 hours. He replaces the motherboard. The server now powers up yay! And it POSTs, and it . doesn t see the disks. !#$!#$ He orders the FC controller, which is so very much not in stock that they can t get it to us until 8:30AM the next morning (keep in mind this thing is on a 4-hour 24/7 contract). Next morning rolls around. Outage now more than 24 hours. He pops the FC controller in, we tweak the SAN settings appropriately, we power up the machine, and . still doesn t see any disks, and the SAN switch still doesn t see any link. EEP! Even the BIOS firmware tool built into the controller doesn t see a link, so we KNOW it s not a software issue. We try plugging and unplugging cables, trying different ports, everything. Nothing makes a difference. At this point, while he ponders what else he can replace while we start migrating the server to a different blade. We get ERP back up on its temporary home an hour later, and he basically orders us every part he can think of while we ve bought him some room. Several additional trips later, he s replaced just about everything at least once, some things 2 or 3 times, and still no FC link. Meanwhile, I ve asked my colleague to submit a new ticket to HP s SAN team so we can try checking of the switch has an issue. They take their sweet time answering until he informs them this morning that it s been *48 HOURS* since we first reported the outage. All of a sudden half a dozen people at HP take a keen interest in our case. As if they could smell this blog post coming So they advise us to upgrade the firmware in the SAN switch, but they also say we really should send this to the blade group; the problem can t be with the SAN and of course the blade people are saying the problem s GOT to be with the SAN . We try to plan the firmware upgrade. In theory, we can lose a switch and nobody ever notices due to multipathing redundancy. In practice, we haven t tested that in 2 years. None of this equipment had even been rebooted in 390 days. While investigating this, we discovered that one of the blade servers could only see one path to its disks, not two. Strange. Fortunately, THAT blade wasn t mission-critical on a Friday, so I power cycled it. And it powered back up. And it promptly lost connection to its disks entirely, causing the SAN switches to display the same mysterious error they did with the first blade the one that nobody at HP had heard of, could find in their documentation, or even on Google. Yes, that s right. Apparently power cycling a server means it loses access to its disks. Faced with the prospect of our network coming to a halt if anything else rebooted (or worse, if the problem started happening without a reboot), we decided we d power cycle one switch now and see what would happen. If it worked out, our problems would be fixed. If not, at least things would go down in our and HP s presence. And that worked? What? Yes. Power cycling the switch fixed every problem over the course of about 2 minutes, without us having to do anything. Meanwhile, HP calls back to say, Uhm, that firmware upgrade we told you to do? DON T DO IT! We power cycle the other switch, and have a normal SAN life again. I let out a WOOHOO! My colleague, however, had the opposite reaction. Now we ll never be able to reproduce this problem to get it fixed! Fair point, I suppose. Then began the fairly quick job of migrating ERP back to its rightful home it s all on Xen already, designed to be nimble for just these circumstances. Full speed restored 4:55PM today. So, to cap it all off, within the space of four hours, we had fail:

• One ERP database
• ERP server s motherboard
• Two fiber optic switches but only regarding their ability to talk to machines recently rebooted
• And possibly one FC controller

Murphy, I hate you. The one fun moment out of this was this conversation: Me to HP guy: So yeah, that machine you ve got open wasn t rebooted in 392 days until today. HP guy: WOW! That s INCRED oh wait, are you running Linux on it? Me: Yep. HP: Figures. No WAY you d get that kind of uptime from Windows. And here he was going to be all impressed.

If I open a link from within one of my terminals, it causes Firefox to load that link in a new tab, which it activates (puts on top). Unfortunately, the browser window also steals the focus. This is driving me nuts at times. I want the new tab in the foreground, but I don t want the browser to steal the window focus. Firefox would not be Firefox if it didn t have an obscure, under-documented feature that almost achieves what I want, but comes with negative side-effects. As I learnt over at UbuntuForums, I can enable browser.tabs.loadDivertedInBackground, and the browser stays unfocused. The negative side-effect is that new tabs are loaded in the background. I cannot decide whether that s more or less annoying than the focus thievery. Rock, hard place, rock, hard place could someone please offer a way out? How can I make firefox load links in foreground tabs, but to keep its greedy little fingers off my focus? The rest of this post is personal. In cooperation with my therapist, I ve stopped wondering about Firefox design choices, and my life quality took a boost from that. Every other day, however, I find myself plotting ways to get rid of the Firefox crap, only to conclude that none of the alternatives provide the functionality I want/need, which are provided by some of the Firefox plugins: AdBlock+, CustomizeGoogle, Firebug, NoScript, TreeStyleTabs, and, of course, Vimperator. And there are others too. That s like being forced to drive a GM truck because the stereo you like doesn t work in any other car. I better go and swallow some more pills. Update: I ve filed a bug with the Mozilla bug tracker: Mozilla bug #481844

I have a bit of money invested in stocks and funds, passively managed by my bank. I ve made a bit with risky speculations, and I ve lost about the same amount with the markets dive of the last 18 months. Net benefit: zero cash, a bit of experience. Time to move on I am not an interesting customer, so I was all the more surprised to get a call from an employee of the bank the other day. He identified himself as Crisis Manager or something similarly ridiculous, mentioned at least 20 times that he was a regular employee and wouldn t receive provisions, and he also kept assuring me that he s doing himself what he s suggesting to me: leveraging the cost-average effect. I turned him down, probably a bit too briskily. Let s investigate why. Banks are good at devising ingenious tricks to fool people; it s their business after all. One of those tricks is the so-called cost-average effect. It s awesome, they say, because it is the first product that adapts to the ever-changing market: when prices are up, it buys smaller numbers of shares, and in low times, it stocks up on them by buying larger quantities, automatically. Fancy, no? It isn t, because the entire idea is to get your signature on a contract that obliges you to pay a fixed sum each month, with which shares are then bought. Obviously, when prices are up, numbers you get for your money are low, and vice versa. Observe how well the banks manage to turn that basic rule into a selling point. They are good at that. In fact, that s probably all they re good at. Obviously , inflation is also factored in, so the monthly fixed sum actually increases by a projected 5% each year, which they call inflation protection . While this strategy investing fixed-sum installments rather than lump sums certainly has its benefits, and can level out the risk of throwing a large sum of money at the market, it is commonly considered a suboptimal strategy, especially when employed over longer spans of time. For more information, I refer you to the frequently cited article Nobody gains from dollar cost averaging analytical, numerical and empirical results , published in the 1992/1993 issue of the Financial Services Review journal. So why would the bank approach me with this offer ? Let s go hypothetical and check out Ashley s investment practices: Ashley bought a fund five years ago at 100 per share, and it s now down to 80 . The loss is thus 20%, a -4% factor (20% / 5 years) that affects the overall performance of the portfolio. Ashley doesn t really care much about the percentages. After all, the only thing that counts is the worth of the shares when sold right now. Everything else is history and only influences the size of the tears one weeps in times like this, which is a psychological malfunction, not a function of the financial markets crisis. The bank, however, does not like negative performance values. Positive percentages yield happy (blinded?) customers, who get much more enthusiastic (greedy?) to put more money into the giant machine that makes the banks tick which is still in the process of blowing up into their faces. But negative performance values cause investors to become conservative and leery. They d rather hold on to their hard-earned cash for fear of losing more. That s the flip-side of aforementioned psychological malfunction, and this is the core of the reason why some of the big investors get bigger even in times like this they do not succomb to the feeling . The banks, however, need cash, and loads of it, and while the state is busy printing notes, they take every chance to get more. Private clients are wonderful prey, as they are less informed about the system than investors. So, to get cash from the clients, the banks have to circumvent the psychological malfunction. They cannot boost the market value share, but there are two values that factor into a percentage, and the cost-average effect, sold as the best thing since sliced bread, is nothing but a way to affect the other number: the sale price, the divisor. If the fund stays below the original sale price (100 ), each time Ashley pumps more cash into the machinery each month, this divisor decreases, effectively decreasing the percentage value. In the long run, this yields happier customers, who are willing to put more money in, especially if the performance turns positive. The baseline is now lower, so a return to previous heights would correspond to larger performance values than ever before. Obviously, a positive market causes the baseline to grow, but the cost-average deals continues to pay off for the banks. Since Ashley put that signature on the contract, the bank has a guaranteed cash flow for the contract s duration, while Ashley wonders why the percentage remains dampened. Cost-average means decreasing the percentage, in good as well as in bad times. Just now, the bank doesn t have to work so hard to get even more money from you. It gets worse: usually, those contracts are embedded in some sort of life insurance deals with grand promises in 40 years to come, and over all that time, it ll be quite clear who wins and who loses. Scary, huh? I was going to end with the advice to stay away from the cost-average effect, but I think I can just as well make it more general: if you don t understand a deal, don t sign it. If you abide by that rule, you would stay away from cost-average effect deals all by yourself. Maybe my little writeup has helped, nonetheless. NP: DJ Ti sto: Live at Innercity

At some point in the past, when I was still thinking that Glest can be packaged in a sane way, I had a conversation with upstream developer martinho on the Glest forum (they call it board) about them integrating a patch in order to support big endian machines.

After some messages on their board, it was clear that they don't want to integrate any patches and started to throw out random fake arguments against the integration of my patch and other patches.

When he figured out that I replied and debunked all of his fake arguments, he went on and actually removed my posts from the forum. It was so weird since I have never seen that kind of behaviour before and some of his posts and other people's posts were replies to my posts and things didn't made any sense after the deletion.

At that moment I have managed to save one small bit of the conversation via Google's cache, but now I have finally decided to make it public.


Note: Back then, I kept this post private in the hope that I was maybe too involved in the conversation at that moment to make a rational decision about it, and hoped things would get better in time. They didn't. Meanwhile they migrated to another forum software and they said they lost most/all(?) of the previous content from the old forum.



Here is the conversation bit I managed to save.
Original text is in italic font. Regular font is for my current comments for clearing up the context of the discussion. The text retains the BB syntax and [quote]-d text is martinho's own words.



[quote="martinho"]thebohemian, eddyd,

I think you guys are missing the point, this is not the place for any [/quote]

Congrats, you are avoiding to answer the points I have raised.

[quote]
- The Glest project is made by a bunch of friends for fun, we ask for nothing, we answer to no one. So I beg you not co complain, because it is out of order.
[/quote]

In that case I don't see the point of the "development" forum. care to explain?
And, from my PoV, these are your users you're arrogantly dismissing.


The following was a reply to my request to add big endian support through a patch I provided at that time. The patch wasn't complete to actually make glest work on BE machines, but it didn't broke little endian support at all. He was trying to argue that there were no valid big endian machines on which the game would run. Integrating the patch would have made my life a lot easier in continuing the work on the patch.

[quote]
- Our target plarforms are PCs running Windows (I'm responsible for this) or Linux (matze is the boss here). We don't care if Glest doesn't run in a big endian dishwasher with Solaris.
[/quote]

Linux on PowerPC is a viable platform. So is Mac OS X on PowerPC.



I made the mistake of saying that some parts of the code where not that clean and could be improved. More on that below.

[quote]
- One of the keys of Glest success is how extremely clean our code is. Bringing random people in would break this, and we are not going to do it. [/quote]

So you are the only keepers of true knowledge of clean code? Hahahaha!

There were at least 3 patches that provided this through word/double-word swapping. I don't see how embedding the patch can affect the unity of the code.

BTW, you are aware you can "carve" people into coding in your style before accepting in, aren't you?



I suggested using int8, uint8, int16, uint16 in order to make sure that the parameters are of the expected fixed size since plain char, int, long, long long have different sizes depending on the architecture. The usage of int and int16 was inconsistent and I also suggested using always the numbered variants.

I said the approach that was then taken in code was not portable and that was the reason for the proposed change.

[quote]
- Glest has been coded with portability in mind, some of your comments about portability are really naive (compilers don't care if you declare function parameters with fixed byte length, they just use whatever is the fastest int in the platform). Fixed length data types are used when needed,
[/quote]

What? When did I say anything about that? I said the code uses inconsistently "int" and "int16" and should use the later always because the code assumes the sizes on i386, which are not always applicable - see long on amd64.


If you mean that I can't enforce the width of the parameter, then you should know better, those types are defined in a platform dependent way to be platform independent. Yes, there are cases where [b]on the stack[/b] the size of the passed data is bigger (they might be padded), but that is transparent to the functions and, in spite of that, the functions will see the proper values for the parameters.

Also, the packing behaviour is compiler and architecture dependent, and some compilers produce the weirdest packing (if you are allowed access to such docs, look for Green Hills compilers' for V850 and you'll be stunned).


Please, don't make a fool of yourself by trying to discredit me, just not to loose face.

Later he just removed my posts not to loose face.




As a sane programmer, I made my patch to do the swapping only on big endian platforms, and the code was enabled in through the precompiler, so there was no performance hit on the little endian arches.

Of course, any more-than-newbie programmer would agree that having two data sets for a single game is stupid and is better to do the swapping in the application itself to avoid space wastage and maintaing an alternate big endian data set.

[quote]
no more, no less. About endianness, we don't care. If we wanted to fix the problem anyway, the right way is to do the byte swap in the data, this is how it is done in commercial games, since it is more efficient.
[/quote]

You just proved how much bullshit you can deliver in one phrase just for the sake of winning an argument. All portable games and all applications that have data files prefer to do swapping on the spot just because swapping the data on the disk (or providing BE and LE data) is plain stupid and inefficient from a disk and bandwidth PoV.

For Glest, swapping the data after reading from disk is not even affecting performance in a manner that could be significant or noticed, not even for a processor running at 500MHz, since that would be done only on load, once, and just for the bits of data which are relevant; these are mostly headers, since the actual data, in most cases, is just a long byte vector. This translates in not more than 3 or 4 instructions in assembler (an init, an exchange, an increment and/or a loop instruction), and that would happen before starting a game, which is dead time anyway and wouldn't be noticed unless the data would be huge (order of hundreds of MB or GB) and the whole data set would need swapping.

Even if the data itself (not headers) represents data which needs to be swapped, that is still irrelevant in terms of speed.


BTW, the reason that in commercial games it might be done like that is because there the games are distributed in a package per platform anyway and having two data sets is not an issue since they are using the same amount of CDs/DVDs, but doing the same for a game like Glest would mean doubling the storage used and it would increase the complexity for a release, not to count the confusion of the users.

I still doubt commercial games do this, but I might be wrong.