As mentioned in my last entry, Livejournal have changed their url scheme to get around a problem in mozilla. The trouble is down to the -moz-binding CSS property. I'll repost a copy of the short lived posting to the lj_dev community earlier.

Discussing the URL Change As we recently announced in [info]news, we have changed the canonical URL of most journal, community, and syndicated content. While we did offer subdomains to Paid users in the past, this is now the canonical URL that we will link and redirect to for all journals. However, all communities are located at http://community.livejournal.com/username and syndicated accounts at http://syndicated.livejournal.com/username. Due to the way certain proxy servers are configured, and the fact that the HTTP RFC prohibits it, the canonical URL for journals with a username starting with an underscore will be http://users.livejournal.com/username. We are however offering a free rename if you fall into this group. So now the technical side of all of this and why it was a required change; bear with me as its 3am my time and I'm sitting in John F. Kennedy Airport after a five hour flight. Late last week we become aware that it was possible to use the "-moz-binding" CSS attribute within Mozilla and Mozilla Firefox to execute arbitrary offsite JavaScript. As this attribute is designed to allow attaching an XBL transform and JavaScript to any node within the DOM, it is quite easy to use in a malicious fashion. We immediately altered our cleaner to strip this attribute from entries and comments, though also realized that wasn't even half the battle. As we allow custom CSS in many of our styles, as well as the ability to link to an external stylesheet in a variety of fashions, it was quite possible to take advantage of this exploit and hijack the session cookie of any user who views your journal. As we, along with many other sites, used one cookie to authenticate a user, this cookie was quite powerful if stolen. If the user had not chosen to bind their cookie to their IP address, a malicious user could steal it, login as that user, deface the account and SPAM with it, as well as modify that user's style to include the exploit thus causing this problem to spread much like a virus. Borrowing the idea from another development team within Six Apart, we decided we needed to break our cookies into three categories. One cookie would be our application management cookie, this cookie would only be accessible on www.livejournal.com where we will not display untrusted content. A second cookie will be accessible on all subdomains of livejournal.com, though it only will say if you are logged in or not; it is solely for optimization. We then will issue one cookie for each journal you visit. This cookie will be only accessible on username.livejournal.com or community.livejournal.com/username as it is limited to a single journal. This cookie will only grant you the permission to read protected entries and post in the particular journal. This means that if the journal owner steals your cookie, they will be able to do nothing more than view their journal and comment upon it as if they are you. In the end you will have n+2 cookies, with n being the number of journals you visit. Due to the fact that we cannot clean every external CSS stylesheet linked to every time we generate a journal page, this change is required. While it does not fully protect us from some new cross site scripting vulnerability that can be exploited via entries or comments, they are much easier to block, patch, and recover from quickly. With Mozilla deciding to allow the execution of arbitrary JavaScript via CSS, there is no other viable solution than the one we have undertaken. We developed a plan to phase all of this in over the next week, URL change first and then followed by the cookie change, though this morning we were made aware that this was being actively exploited. As such we took our week time line and shortened it to about twelve hours. While URLs have been changed at this time, our cookie handling change has not yet happened. This should however be expected to take place within the next day or two as well as various other cleanups and fixing bugs we've already encountered. Please feel free to let us know if you have any questions and open a support request if you find a bug or encounter a problem. Sorry all of this came with just about zero warning, but in the end we could not wait longer to fix this problem.

Thanks to Daniel Silverstone for providing me with a cache copy of the text.

This over-the-counter medicament may help us in the future. When someone’s behavior suggests they may be in need of it, feel free to suggest it to them (this includes me). This is the European box. By clicking on the image, you will see the Engrish version…

Edd Dumbill writes from time to time about hinting or using the right fonts so they look nicer - At the first glance, I found three such articles in ~six weeks.
I don't know if I'm too atypical in this area as well... But for most of my work, I prefer fonts not to be beautified by antialiasing. Strangely, it seems to depend on which program (or, maybe, activity) I am working - I have no grudge against antialiasing in a web browser, but I swear to God that if you antialias my emacs or my xterms, I will hate you. It just gets more tiring.
I have no rational explanation for this - I just noticed the fact after using it for a while. I get more tired if the borders of the letters are not crisp - and I don't really care about the little details in smaller sized fonts.
Maybe that's part of the reason antialiasing is nicer in the browser - After all, it is common to find text too small to read there. At 1280x1024 (or 1400x1050 on the laptop) with the usual font size, I get a nice 170x66 rxvt. That's quite OK for most of the work that requires attention - and for following large amounts of text, decreasing the font size to a somewhat tiring size gives me a nice 200x92 window - almost always enough room. Crisp fonts. Easy to read.
Does anybody else feel the same, or am I just too looney?

We caught the Caltrain to San Francisco yesterday, and it was our first train ride so far. I have a bit of a childhood interest in trains, so it is always fun for me to compare and contrast the differences in rail systems that I see around the place.

Ticketing

There are these huge ticket machines, where you purchase a ticket before boarding the train. Fares a calculated on how many zones you travel in. So you have to consult a map of the line (it's just a single line) and work that out first. The machines take cash or a credit/debit card, with the usual disturbing lack of authentication for the latter. The tickets themselves are just a piece of card. There's no magnetic stripe or anything. The 10 ride tickets are also just a piece of card, from what I could tell. You "validate" them before riding by putting them in a gadget the essentially chops a portion of the side of the ticket off.

The trains themselves

I can't remember precisely, but I don't believe there were power-lines over the track. I'm pretty sure we had a diesel-hauled train. The engine on one that went in the opposite direction was certainly big and noisy. The cars are double-decker. Upstairs is pretty unusual. They are single-seats on either side of of an open void through to downstairs. There's a stainless steel shelf that runs the length of upstairs at about chest-height for putting luggage on, and below that you can see right through to downstairs. (I later discovered that this allowed a ticket inspector to walk down the aisle downstairs and view everyones tickets on both levels). The seats were all high-backed. The whole thing had a "long-distance" feel about it. There was a baggage car. You could take your bike on board (so, like the light rail, I think we'll do some cycling tours of places further away than what we'd directly cycle to).

Ticket inspection

Speaking of ticket inspectors, on the ride in, there were none (though there did seem to be hoards of roaming Caltrain employees wandering up and down the train), and on alighting at the station, no one checked our tickets on the way out, and there were no barriers or anything. You just wander in and out of the station as you pleased. Frankly, I was quite amazed. We did have a very diligent ticket inspector on trip back.

Speed

Definitely not the fasted thing in the world. We had all-stops trains both ways, and it took about an hour and a quarter. But it was fairly cheap, compared to driving and parking: $10.50 for a "day-pass" (what I'd called a "return").

After setting up the Backup Manager’s website, I took the time to install mailman in order to handle mailing lists nicely (congrats to the mailman maintainer, it installs itself like a charme). Everything is now ready for welcoming subscribers to the backup-manager mailing lists. I’m about to release a new version of Backup Manager, featuring a new method for making customized backup: you are now able to enter a command and all the output sent to stdout by this command will be used as the archive content. It can then be compressed with the tool of your choice if needed. Here is an example of the concept:

command=ssh myhost -c 'mysqldump -uuser -ppass database'
filetype=sql
compressor=bzip2

This will be provided to backup-manager through arrays, so we can implement as many archive methods as we like.
I called this method the “pipe” method, I gues you know why. This is already available in the svn repository.

So following Florian’s chastisement of my “threatening” fellow Debian developers, Charles Plessy becomes annoyed by a bug in apt-file (its default configuration expects curl, but wget is what’s installed on most systems), at which point Luk Claes then starts threatening to NMU whether the maintainer likes it or not. Naturally that’s not the correct thing to do for a report the maintainer’s addressed and said is not a bug, 0-day NMU policy or not. Naturally, pointing this out brings Charles back into the fray, to complain further. A followup to that produces this off-list reply from Charles:

On Fri, Jan 13, 2006 at 05:13:15PM +1000, Anthony Towns wrote :
> I'm not disputing whether it's a bug or not, the maintainer is. If
> you are *helping* the maintainer, then fine: do an NMU.
Dear Anthony,
I would love to, but I am not a developer. And I am amazed to see that
more energy is spent in arguing rather than solving the problem. I hope
Luke will NMU this package and close those shameful bugs.
> In my experience you almost always get a better response from people if
> you assume they've got a good reason for doing what they have been doing,
> rather than just trying to add extra punctuation to your sentences.
> Admittedly, punctuation is pretty cool...
That kind of sentence reflects your inclination for ad-hominem attacks.
They poison the -devel list.
Best,

So yay for people who aren’t developers, in the n-m queue or maintaining a package, who don’t understand Debian’s processes, yet still think it’s great to pontificate about what Debian’s processes are, go on about how developers are “arrogant experts” and who think “punctuation is pretty cool” is a “poisonous ad hominem attack”. But what I hate most is people who think they’re contributing to Debian by mailing people privately to tell them how horrible they are. Gag. For those playing along at home, the proper process to follow in a dispute with a maintainer is to bring it up to the technical committee, not to try forcing the situation, whether that be by reopening bugs or playing bug ping-pong. It’s really not complicated. It’s even documented (5.8.3 of the Developers Reference).

So following Florian’s chastisement of my “threatening” fellow Debian developers, Charles Plessy becomes annoyed by a bug in apt-file (its default configuration expects curl, but wget is what’s installed on most systems), at which point Luk Claes then starts threatening to NMU whether the maintainer likes it or not. Naturally that’s not the correct thing to do for a report the maintainer’s addressed and said is not a bug, 0-day NMU policy or not. Naturally, pointing this out brings Charles back into the fray, to complain further. A followup to that produces this off-list reply from Charles:

On Fri, Jan 13, 2006 at 05:13:15PM +1000, Anthony Towns wrote :
> I'm not disputing whether it's a bug or not, the maintainer is. If
> you are *helping* the maintainer, then fine: do an NMU.
Dear Anthony,
I would love to, but I am not a developer. And I am amazed to see that
more energy is spent in arguing rather than solving the problem. I hope
Luke will NMU this package and close those shameful bugs.
> In my experience you almost always get a better response from people if
> you assume they've got a good reason for doing what they have been doing,
> rather than just trying to add extra punctuation to your sentences.
> Admittedly, punctuation is pretty cool...
That kind of sentence reflects your inclination for ad-hominem attacks.
They poison the -devel list.
Best,

So yay for people who aren’t developers, in the n-m queue or maintaining a package, who don’t understand Debian’s processes, yet still think it’s great to pontificate about what Debian’s processes are, go on about how developers are “arrogant experts” and who think “punctuation is pretty cool” is a “poisonous ad hominem attack”. But what I hate most is people who think they’re contributing to Debian by mailing people privately to tell them how horrible they are. Gag. For those playing along at home, the proper process to follow in a dispute with a maintainer is to bring it up to the technical committee, not to try forcing the situation, whether that be by reopening bugs or playing bug ping-pong. It’s really not complicated. It’s even documented (5.8.3 of the Developers Reference).

In an effort to have a signed repository I have reorganized it and thus has different sources.list syntax:

deb http://people.debian.org/~rdonald/nvidia unstable nvidia-graphics-drivers
deb http://people.debian.org/~rdonald/nvidia unstable nvidia-graphics-drivers-legacy
deb http://people.debian.org/~rdonald/nvidia unstable nvidia-xconfig
deb http://people.debian.org/~rdonald/nvidia unstable modules
deb http://people.debian.org/~rdonald/nvidia unstable nvidia-settings
deb http://people.debian.org/~rdonald/nvidia unstable nvidia-kernel-common

and you can group them like:
deb http://people.debian.org/~rdonald/nvidia unstable nvidia-graphics-drivers nvidia-kernel-common

oh and there's :
deb http://people.debian.org/~rdonald/nvidia sarge modules

The old syntax will still work for a while.

oh and the key info for apt:
gpg --keyserver hkp://wwwkeys.pgp.net --recv-keys F641D1A6
gpg --armor --export F641D1A6 sudo apt-key add -

B2BB 3BA0 90DD 8DBD 0298 3B89 2122 53A4 F641 D1A6

Don t you love it when it looks simple and then it is actually simple, but you are not up to the task?

I am loving the xlibs-dev transition, really, I mean it! Two of my packages, fkiss and xdigger got bit by this issue, and I immediately fixed the bugs with a little help from a script provided by Moritz Muehlenhoff. really cool.

So I follow instructions in The Debian Wiki, run the script, check with pbuilder, upload and profit! Fun! End of world hunger! Err, almost. So I get all excited and decide to look at the bugs. And I find a nice one: #346347. Not bad, gpsim FTBFS, Luk also took a look at it and didn t really like what he saw, it neede no xlibs-dev or substitute for them and sitll FTBFS.

But wait! Just by looking at gpsim s bugs page, I find out that there s a New Upstream Version(tm) (#248931) available since 1 year and 241 day ago. Also, NMUs that have not been acknowledged for as long as almost two years... Not nice. And there s even a patch (#345215) for building it on, GNU/kFreeBSD... how sexy!

So I get all obsessive about it. But newer upstream needs a newer libgtkextra-dev version (#347397), which of course was not packaged (don't we all love Murphy?) and that s where I am stuck at. I loathe libraries. But still I gave it a try. And failed. Several times.

Fun? Yes! Profit? Nope! Do you want to take a look at it? Easy. It s all here.

Amaya also FTBFS. And I want to NMU amaya. Badly. Since I first heard about amaya, and realized that I am named after a web browser (sucky as it might be) and after getting many IRC highlights (every time that amaya FTBFS is mentioned on IRC), I have been wanting to use/adopt/NMU it. Amaya NMUs amaya, how recursive can it get?

I will go to bed now, and try not to keep thinking about it, not because I want to become an early riser, Gunnar, no need to worry! But reading about it I thought it would be healthier for me to wake up everyday at the same hour, so that s all I am aiming for. If you know me, you might be rolling on the floor laughing as you read this. So am I! But, if you know me, keep in mind that I have been drinking coffee with no sugar since August... Hey! You are not laughing anymore?

Vi and Emacs are doing fine. Emacs was not pregnant after all, or so the vet told me. It was hard to see her slowly become herself after surgery. Vi was perfectly OK all along, but she looked terrible. Anyway, she felt better by 6am the next day and came to bed, perfectly groomed, and purring like nothing had happened. A week has passed and she has started doing the gory part today... she s biting away the scar tissue, and it bleeds a bit. Oh my Spaghetti Monster!

The good:

• Organized a hotel for my sister to stay in next week when she comes to visit me.
• Managed to have one of my security bugs result in the first DSA of the year.
• “Used my own initiative” to get a CVE ID for Ulf - who got the second DSA of the year.
• Am once again capable of doing pull-ups with one arm.
• Saw doctors, and should be back at work soon.
• Almost solved the mystery of why the Elves had to leave Middle-Earth.
• Started working on an open photo-publishing system, primarily for my own use, but may well end up being open to the world. (Depends on my motivation and bandwidth costs.)

The bad:

• Forgot that smstools is in both Sarge + Woody.
• Forgot to go shopping for washing powder; and have few clean clothes left to wear.
  • Actually I have lots of clothes, many clean, but nothing both sexy and clean.
• Got the whole half-beard thing happening - due to lack of razors.
• Haven’t got a 2006 calandar yet. (physical/paper)

Somehow we've managed to accumulate a variety of Vegemite bottles. So many, that we should have enough to last us until next time we get a chance to restock.

Thanks to Grant, I discovered a couple of interesting blog posts about how to become an early riser, and about polyphasic sleep. I'm certainly interested in becoming an early riser. I'm not naturally a morning person, but for a few years I managed to force myself into it, but then I lapsed back into being a night-owl sleep-in type person. At the moment, when the weather permits, I'm trying to cycle to work with enough time to eat breakfast there and be operational at my desk by about 8:30am. It's just as fine to start at 10am, but I'd rather not get into that habit, and just treat it as a bit of a splurge from time to time. This of course means I can't stuff around in the morning, and ideally I need to leave the house by about 7:45am, with appropriate flow-on affects on when I wake up and get out of bed. It certainly takes a lot of self-discipline to get out of bed when the alarm goes off. At the moment, I tend to opt for two alarms. One to wake me up and get snoozed a lot, and one to actually get out of bed. If I was more disciplined about getting out of bed, I could just have the one alarm go off, and get extra sleep. The extra waking hours gained by polyphasic sleep are tempting also, but I think it's altogether too disruptive to try at the moment. An interesting concept nonetheless. So once we're a little bit more settled, I'll hopefully become more of a morning person again. I think the key is consistency. Sleeping in on weekends just buggers everything up, which is a shame. I quite like my weekend sleep-ins. Downside is you spend half your weekend asleep, when you could be out exploring a fabulous new foreign country, for example...

Pepsi Max

Pepsi One

jumper

sweater

petticoat

jumper (many sniggers when you refer to "putting on your jumper")

entree

appetizer

main course

entree (ordering entree-sized dishes will not have the desired result)

I've already fielded enquiries from two people who have found my blog via Google, and given the 10,500 visas a year being issued, there's only going to be more, so in the interests of not spending too much time answering questions individually, I'm going to write a detailed account of things. Disclaimer: This article is not a replacement for doing your own homework. I expressly disclaim any responsibility for you getting your visa application denied, being cavity searched when you attempt to enter the US, or being sent to Guantanamo Bay. In particular, this information is subject to change, and as I'm not applying for a new visa every day, I'm not going to know about the changes. If you find something here that is blatantly wrong, or out of date, please let me know, but at the moment, I'm writing about my experiences. I'm undecided about maintaining a website devoted to capturing the state of the art in E-3 visa applications. If in doubt, spend the money (and I do mean spend, it's not a cheap phone call) and call the information line - 1902 941 641 - to speak to a human being, and ask all the questions. Resources

• The Australian Government's Department of Foreign Affairs and Trade's page on E-3 visas
• The US Consulate's E-3 visa page
• The US Consulate's page on applying for a non-immigrant visa

What's the process? Your potential employer (sponsor) needs to file a Labor Condition Application (see the form here). At present, this form hasn't been brought up to date with respect to E-3 visas, it's the same form used for H1-B visas. Your employer needs to have written at the top of each page "E-3 - Australia - to be processed". This bit of handwriting is apparently key. You need to have the signed LCA in your hot little hand before you roll up at the US Consulate to apply for your visa. As for how long these take to be processed by the Department of Labor, the FAQ says a week for postal applications. From my experience, and the experience of a couple of other people I've spoken to, it takes anything up to about four weeks. Generally speaking, you do not have to concern yourself with the LCA process, unless for some reason, you're sweating on the LCA so you can apply for your visa. My personal opinion is that the whole process is so long, bureaucratic and unpredictable, that you really don't want to be making any firm plans based on any assumptions on when you think anything is going to happen. Start your planning from when you have been issued your visa, not when you expect you will be issued your visa. Once the LCA has been issued, your sponsor will send it to you along with a wad of paperwork, most of which you just take with you to the consulate. You can make an appointment online. Paperwork-wise, you need to take a completed DS-156 and a DS-157 if you are male between 16 and 45. These forms are available here. You can make an appointment online, the links to make an appointment at each of the US Consulates are also here. I found that the lead time to an appointment was around a month, so if you know the LCA application has been made, you can probably make the booking at the Consulate, so as to try and parallelise things a little bit. What do you need to take to the interview? You need to take your completed DS-156 and DS-157 forms. You need one for each person. So if you are the principal (i.e. the person being hired), you need a DS-156 for yourself, and your spouse, and whichever of you is male needs a DS-157. Bring the sponsor's offer letter too. You need a receipt for a non-refundable visa application fee. You need one receipt per person applying for a visa. You get these receipts by paying $130 per person at any Australia Post outlet. As the visa is non-immigrant, you need to be able show some sort of ongoing connection with Australia. If you own property here, and are planning to continue owning it while in the US, bring a rates notice. I don't know what else is useful to demonstrate an ongoing connection. Also bring a bank statement, as you need to demonstrate an ability to support yourself while in the US. Bring birth certificates and marriage certificates. Also bring a copy of your University transcript, and any assessments that your employer may have had done that state that you hold the equivalent of a US degree. I'm not sure how hard and fast the whole degree thing is. I have an incomplete degree and about 10 years of work experience. This was sufficient. I don't know how you'd go with absolutely no degree and lots of work experience. Finally, you need to bring two US-sized passport photos. These are 5cm x 5cm, which are not what your average passport photo is like if you go somewhere and ask for a passport photo. Processing takes about 3 days, although you get an indication on the day as to whether you are successful or not. If you don't want to come back to collect your passport with the visa in it, you need to leave sufficient stamped, self-addressed envelopes so that the Consulate can mail the passports back to you. It's a bit hazy as to what constitutes 3 days, but assume 3 full working days, plus whatever time it takes for the postage, so up to 5 working days. Good luck with your visa application. Let me know how it goes, and if this has been of assistance.

This blog has been semi-offline for at least a few hours. Sorry about that. Through an interesting (at least to me) series of events, I discovered that my custom “popularity” hack/plugin was out of control. It started earlier today when one of the the RAID discs on this server gave a SMART error. I’ve never been able to really judge whether SMART errors are “for real” or not—sometimes I’ve had a disc with a serious SMART error that, after another test, will run fine for another year or two without incident. In any case, I wasn’t so worried, since this system is running RAID-5, so a single disc failing, even catastrophically, doesn’t result in any data loss or downtime. I decided to take the disc out of the RAID and run some SMART tests on it. I marked it with mdadm as FAILed (mdadm —fail) and then removed it. Strangely, the disc first passed a short test with no problems, but then ceased to be recognized as having SMART capabilities at all. I’ve had that problem before and never figured it out—the SMART capabilities seem to come back eventually. Not knowing what best to do, I just put the drive back into the RAID to see what would happen. Since the disc had already been marked as FAILed, the reconstruction started from scratch. Although I’ve got a pretty good (2Ghz) CPU and plenty (2G) of RAM, kblockd and md0_raid5 are using up most CPU cycles and everything has slowed down significantly. Reconstruction seems to be proceeding fine, just slowly—it will take 24-36 hours at the current rate. Presumably it would be much faster in single user mode, but I can’t take this server offline. In the meantime, I noticed that my blog had stopped responding entirely, while other blosxom blogs served from this same server were fine (maybe a little more latency than usual). So I started parsing through the differences in my blosxom installation and everyone else’s to see what could possibly be slowing things down so much as to time out the blog. Eventually I identified the culprit: my custom-made “popularity” plugin, which reports the most popular entries on this blog and the number of hits they have received. I hacked it together several years ago. I think at the time I just wanted to see if it would work, with the plan to come back later and fix it. I guess I forgot the “fix it” part. My popularity plugin creates a log file that it reads in each time the blog is accessed, and then appends to that file. Over the years, that file has grown to 17 megabytes. Although this is a huge waste of system resources, I didn’t notice it when the system was running at full speed. With the reduced performance from the RAID reconstruction, however, this meant that my blog never finished loading at all. If you made it this far, you certainly deserve some sort of geek admin award. Congratulations. I deserve some sort of stupid admin award, myself. P.S. Apparently I must have forgotten to check my horoscope today. The laundry machine flooded the basement when I didn’t check the sink into which the laundry machines empties. I should probably avoid sharp objects for the rest of the day.

Somehow we've managed to accumulate a variety of Vegemite bottles. So many, that we should have enough to last us until next time we get a chance to restock.

Hm. What might be causing the following symptoms: I plug in an USB memory card reader, it is correctly recognized, however when inserting an MMC card and accessing it, the USB stack hangs. This happens under both Linux and Windows, and unloading the EHCI driver to force the bus to 1.1 does not change anything. My current theory is based on CONFIG transfers working, but BULK, INT and ISO not, so I suspect the differences in signalling that I dimly remember from the last time I read the spec.

I had a bunch of old packages in my ~/debian/nbd/ directory, that didn't serve much purpose anymore, so I wanted to clean them out.

wouter@grep.be:~/debian/nbd$ rm -Rf *

After that finished, I realized that this directory also contained my CVS checkouts of the NBD upstream CVS. And one of those checkouts contained quite a lot of uncommitted code. Whoops. So, what do you do then? Stop writing to disk immediately. Pull the plug. Except this is a laptop, so, push the power button until you don't see anything anymore; and next, try to read the lost file from a different installation. That "different installation" was an easy one. I've always had a second, minor, 'play' partition on my laptop, that I use to try out all sorts of operating systems. I've used it to run Debian stable, Ubuntu, MacOS X. It currently contains Gentoo; I intend to try out FreeBSD for PowerPC some time. But the fact that it already had Gentoo meant I could use that one to run debugfs on the (not mounted) file system, and try to find my files back. Except that I'd never done that before, and I didn't know where to start. And since my gentoo installation isn't set up entirely right, I couldn't get networking to work okay, thus, no help through IRC. My first guess was to check the ext3 journal. There's a command to dump any random inode to disk in hex format (dump <inode number>, with the square brackets), so that should help. I thought. Hah I spent two hours looking at kernel code, trying to find out what the on-disk format of the ext3 journal is (which, as it turns out, isn't part of the ext3 driver; it is a totally different driver, called jbd), only to find out that it wouldn't help me; a journal obviously only contains information on the new data it's going to write, not on the old data it's overwriting. And seen how removing a file means you overwrite a directory inode block with newer information that does not contain the reference to the removed file anymore, that wasn't helpful. At least I learned what the ext3 journal's on-disk format is. Might be useful in the future. My next move was looking at debugfs again. Found that it has some command called dump_unused, which will dump the raw data of any blocks that are not allocated, provided that raw data isn't just zeroes. That's a lot of blocks; bzip2'ed, I had already 1.5G before I killed it. Fortunately, an easy 'bzgrep' told me that the file was, indeed, in there somewhere. I did a commit now. Just to be sure :-)