I want to use the Ext4 filesystem on Xen DomUs. The reason for this is that the problem of fsck times on ext4 (as described in my previous post about Ext4 [1]) is compounded if you have multiple DomUs running fsck at the same time. One issue that makes this difficult is the fact that it is very important to be able to mount a DomU filesystem in the Dom0 and it is extremely useful to be able to fsck a DomU filesystem from a Dom0 (for example when you want to resize the root filesystem of the DomU). I have Dom0 systems running CentOS5, RHEL5, and Debian/Lenny, and I have DomU systems running CentOS5, RHEL4, Debian/Lenny, and Debian/Unstable. So to get Ext4 support on all my Xen servers I need it for Debian/Lenny and RHEL4 (Debian/Unstable has full support for Ext4 and RHEL5 and CentOS5 have been updated to support it [2]). The Debian kernel team apparently don t plan to add kernel support for Ext4 in Lenny (they generally don t do such things) and even backports.debian.org doesn t have a version of e2fsprogs that supports ext4. So getting Lenny going with Ext4 requires a non-default kernel and a back-port of the utilities. In the past I ve used CentOS and RHEL kernels to run Debian systems and that has worked reasonably well. I wouldn t recommend doing so for a Dom0 or a non-virtual install, but for a DomU it works reasonably well and it s not too difficult to recover from problems. So I have decided to upgrade most of my Lenny virtual machines to a CentOS 5 kernel. When installing a CentOS 5 kernel to replace a Debian/Lenny kernel you have to use console=tty0 as a kernel parameter instead of xencons=tty , you have to use /dev/xvc0 as the name of the terminal for running a getty (IE xvc0 is a parameter to getty) and you have to edit /etc/rc.local (or some other init script) to run killall -9 nash-hotplug as a nash process from the Red Hat initrd goes into an infinite loop. Of course upgrading a CentOS kernel on a Debian system is a little more inconvenient (I upgrade a CentOS DomU and then copy the kernel modules to the Debian DomUs and the vmlinuz and initrd to the Dom0). The inconvenience of this can be an issue in an environment where multiple people are involved in running the systems, if a sysadmin who lacks skills or confidence takes over they may be afraid to upgrade the kernel to solve security issues. Also apt-get dist-upgrade won t show that a CentOS kernel can be updated, so a little more management effort is required in tracking which machines need to be upgraded. deb http://www.coker.com.au lenny misc To backport the e2fsprogs package I first needed to backport util-linux, debhelper, libtool, xz-utils, base-files, and dpkg. This is the most significant and invasive back-port I ve done. The above apt repository has all the packages for AMD64 and i386 architectures. For a Debian system after the right kernel is installed and e2fsprogs (and it s dependencies) are upgraded the command tune2fs -O flex_bg,uninit_bg /dev/xvda can be used to enable the ext4 filesystem. At the next reboot the system will prompt for the root password and allow you to manually run e2fsck -y /dev/xvda to do the real work of transitioning the filesystem (unlike Red Hat based distributions which do this automatically). So the state of my Debian systems running this is that the DomUs run the CentOS kernel and my backported utilities while the Dom0 just runs the backported utilities with the Lenny kernel. Thus the Debian Dom0 can t mount filesystems from the DomUs which makes things very difficult when there is a problem that needs to be fixed in a DomU, I have to either mount the filesystem from another DomU or boot with init=/bin/bash .

• [1] http://etbe.coker.com.au/2009/12/01/transition-ext4/
• [2] http://etbe.coker.com.au/2010/01/05/ext4-and-rhel5-centos5/

The Tool Factory closed its doors on 31st December 2009. As you may remember, our co-op is one of few LLP co-ops and we mentioned 2amase LLP in our case study as one of the others. 2amase were closely linked to The Tool Factory LLP. Their goodbye message lists multiple reasons for the closure, but a key part seems to be a dispute between members about the copyright of their proprietary Business Plan Writer tool. I think there s a strong resonance between cooperative principles and free and open source software, so I m not suprised that proprietary software development has contributed to failure of a co-op, but it s still a shame. Goodbye and best wishes for the future to The Tool Factory workers. I hope to promote links between cooperatives and free software more in 2010. New Co-operatives UK head Ed Mayo mentioned it in his first speech, so the time seems right. If you d like to help put on some co-op FOSS events, leave a comment here or get involved in this open co-op discussion, please.

Mark this one down as another in the long list of "duh" once you realise what is going on! Bug report comes in about a long running daemon that has stopped logging. lsof reports the log file is now named logfile~ and further more is deleted! This happens after a system upgrade scenario, so of course I go off digging through a multitude of scripts and what-not to find the culprit... Have you got it yet? Try this... vi is very careful and renames your existing file, so that if anything goes wrong when writing the new version you can get something back. It's a shame the daemon doesn't know about this! The kernel is happy to deal with the rename, but when the backup file is unlinked you're out of luck. Confusingly to a casual inspection your log file looks like it's there ... just that nothing is going into it. (oh, and if you tried that, you might like to restart syslogd now :) Moral of the story -- overcome that finger-memory and never use vi on a live file; you're asking for trouble!

So, while Richard is busy doing research and arranging or facilitating several Debian workshops, here I am, gulping down Idlis and Dosas by the dozen. It's been a fun visit, and I got to visit several places in my native state, but since my visit to Mumbai is rather short, I won't be able to say hello to several friends this time (Sorry Kartik). Maybe this would happen only when I am back in India on a permanent basis; let's see...Here are some general guidelines for travelers which I'd like to share:

• They say it doesn't snow in Dallas. Don't believe them. I had to stay overnight at Dallas, and then embark on a trip in the other direction (trans-Pacific) to get home. It wasn't a really fun journey.
• If you're ever rerouted due to flight delays, ensure you travel by a similar route, or else, ensure that you're not a vegetarian/vegan. If you fly to Tokyo and Hong Kong, for instance, be sure that you've specified your meal choices well in advance. Of course, you can live on fruits and bread, but you'll really be doing yourself a favour by not missing your flights in the first place; I don't know how I have wronged the weather gods this time.
• Ensure that you don't keep too many valuables (which Customs might be suspicious of) in your checked-in baggage. This is important, especially to get through situations where you find that your baggage doesn't arrive with you (my baggage came a day late, and, thankfully, cleared customs quickly since my clothes were duty free, thanks to the many heavy duty cycles they've gone through in washing machines).

It's been a really fun trip, and I am sure I'll enjoy the rest of the trip, till I get back to my old routine and resume work.Finally, in keeping with the spirit of a blog I try to understand (albeit, in vain, I must accept), I duplicate the concept of a post from that blog, here is an up-to-date list of airports in which I have sat, but never left except by way of airplane:

• LGW
• IDR
• EWR
• CLT
• TKO
• HKG

More updates later. Bye for now.

A new littler release (now at 0.1.3) just went out of the door this evening. littler provides r (pronounced littler), a shebang / scripting / quick eval / pipelining front-end to the the R language and system. This version adds a few minor behind-the-scenes improvements:

• interactive() now evaluates to false as you'd expect in a non-interactive scripting front-end. To restore the previous behaviour, new switches -i or --interactive have been added.
• Some of the 'cleanup' functionality described in Section 8.1.2 on 'Setting R callbacks' from the R Extension manual have been adopted.
• Example scripts install.r and update.r received an update based on lessons learned from the R 2.10.0 roll-out and package rebuilding.
• A few build issues were improved, a minor manual page formatting bug was fixed.

As usual, our code is available via our svn archive or from tarballs off my littler page and the local directory here. A fresh package is in Debian's incoming queue and will hit mirrors shortly.

Had you been on our yard tonight, you would have seen a strange sight. Me, dressed warmly (but not quite warmly enough), jumping on the icy and snowy hood of my pickup. Some maniacal fit? An obscure part of my 3-day-long taking out the trash routine? Not exactly. Before I tell you why I was doing such a strange thing and almost in the dark I feel compelled to take a detour through ancient Greek poetry. You heard right. I ve been reading ancient Greek poetry lately. Those Greeks were forever running around receiving signs and portents from the gods. If ever a bird flew by carrying a live snake, and you saw the snake bite the bird while looking over your left shoulder, you knew Zeus was telling you not to attack the enemy that day. Or something. And if you, fool that you are, attacked the enemy that day anyway, you should be not in the least surprised to find a spear through your shoulder in just a few pages time. If ever someone received a sign from Zeus in modern times, surely it was I, tonight. I say that because, as I was trying to close the hood on my pickup after charging its battery, the hood nearly snapped in two. The hood is supported by some spring-loaded brackets in the back, so to close it, you have to push down on the front. I did so, and was surprised that it was going down so easily. Then I realized, apparently in the nick of time, that only the front was going down. The back was staying put, aiming up in the air as ever. So I tried to straighten the hood back out. I then tried to get it down by pulling at it halfway back, but just couldn t get enough leverage to make it budge. So what next? Why, jump on it of course. I got it most of the way closed that way, then jumped/slid down, latched it in front, then crawled back up there to push down the sort of rusty mountaintop that had formed halfway down the hood where it was buckling. OK, sorted. So to recap By this point, I ve had the battery charging for 24 hours, inflated the tire, removed the bungee cord around the brake pedal, removed the battery charger, coerced the hood shut. Time to start it. I hop in, pump the accelerator a few times, and turned the key to start. I heard the sickening aaaaaaaaRRRRRRRRRRRRuuuuumphhh . of a weak battery. So it wasn t charged enough to crank fast enough to have any hope of starting in winter. So I go get the car, pull it up close, being careful to leave myself enough room to maneuver between it and the electric fence. I get out the jumper cables. Then it occurs to me: I have to open the hood again. DOH! I open it just a bit, and hook up the jumper cables. It turns over a bit faster. For just a bit. I get a small sputtering of life from it, but it fades faster than an enemy of Hector at the battle for Troy. So I have Terah come out and put her foot on the accelerator of the car while I try to start the pickup. This too proves futile. So I disconnect the jumper cables, close the pickup s hood with significantly less care than a 31-year-old rusty and nearly-broken hood deserves, roll the window back up, and go for plan B: 4 trips to the road with stuff in my car trunk, and leave the giant cardboard boxes that are covered with snow in the pickup bed for next month. Obviously I have displeased the gods. Forgotten to offer up a sacrifice to Penzoil, feared goddess of internal combustion, perhaps? Or perhaps it was just a sign that I need a new pickup? I m so convinced of the latter that I didn t even bother to replace the bungee cord. Postscript Terah walked in, and said, Ah, I thought I heard you writing a blog post. Yep. It s about pickups and the Greek gods. <very slight pause> Oh, of course it is.

For a long time it has been obvious that in all cases anti-piracy technologies discourage purchases and in many cases encourage piracy. I first discovered the significance of this in about 1991 when I attended a public lecture by a senior employee of Borland and a member of the audience claimed that the Borland product he bought didn t function correctly due to anti-piracy measures. The Borland employee firmly stated that Borland did not use anti-copying technology on any of it s products, didn t have any plans to do so, and the problem in question must have been caused by something else. Of all the hostile questions that were asked, this was the only one that caused the speaker to appear agitated so it was obviously an issue that was considered to be important within Borland. In the late 80 s anti-piracy measures were mostly based around creating floppy disks that couldn t be easily copied (violations of various aspects of the disk formatting standards). This meant that you couldn t make a backup copy of the data, so it wasn t uncommon for people to seek pirate copies of their commercial software for daily use to avoid wearing out their valuable original floppy disks. Then the dongle was invented and people who bought software sometimes sought pirate copies so that they could use their printer and their commercial software without having to change plugs on their PC. But in those cases the benefits to uncrippled software to the users were small. Now a large part of the battle on copy protection concerns DVDs. If you had a DVD of a recent movie and an MP4 which would you rather watch? Would you prefer to be forced to watch some anti-piracy rubbish for a couple of minutes at the start of the movie (with fast-forward disabled) or would you prefer to just start watching it? Would you prefer to be able to pre-program the sections of the movie that you watch (as some parents desire to skip the sex and/or violence in movies for their teenagers) or would you prefer to be forced to watch the movie straight-through with only a manual fast-forward to skip sections? Would you prefer to have a DVD that can t be played properly on many (most?) computers because of the CSS encoding or an MP4 that plays on everything from PCs to mobile phones without an issue? Would you rather have 100 movies in the spare space on your laptop hard drive when you travel and 1000 movies on your desktop system or the much smaller number of boxed DVDs that you can store? I think that in most cases a pirate MP4 will give a better experience than a DVD. So the question is, why pay for a DVD when in most cases you get a lesser experience than you will get from a MP4 file downloaded by bittorrent? One reason for buying the DVD is to support the film industry. But I doubt that such a profitable industry will get much sympathy in today s economy. Another reason is the morality, some people consider piracy to be theft (it isn t by definition theft requires that for at least a moment the property be completely in the possession of the thief) and therefore avoid it. One technical reason for buying a DVD is the fact that it may have multiple languages supported, it will have subtitles, it may have an audio track with the creators giving a commentary, and it may have extra scenes that were cut from the main release. I believe that work on adding subtitles to the video file formats is a work in progress, so it s only a matter of time before the DVD rips include all this extra data. Really the content creators should focus on making a product that meets the needs of users and that they want to pay for. Pirating books is technically possible, but almost no-one does it. Some successful authors such as Charles Stross freely publish significant parts of their work and Cory Doctorow freely publishes all his work in electronic form. Books just work well, they meet the needs of users and people want to buy them. Sure they can sell them second hand, lend them to other people, and it s technically possible to pirate them, but they remain profitable. On my documents blog I have a page of links to free short stories that I liked [1] and a page of links to free books [2]. It seems to me that creators of other copyright content should consider how they can be of service to their customers. We are all familiar with corporations and misguided individuals who get whiny about the supposed losses due to piracy. Bruce Everiss has unfortunately joined this trend and demanded the disconnection of Internet users based on unproven accusations of game piracy [3]. I don t know whether the game buying experience sucks as badly as the DVD buying experience, but based on the reports of locked-down consoles that have to be cracked before they run Linux I expect that the modern game industry is doing at least as badly as the movie industry. They need to provide things that users want! One thing to note is that a Windows or console game player who uses pirate games will probably buy some games at some future time, while someone like me who uses free software both by principle and because it gives a better user experience will probably never pay for a game (I haven t got time to play all the free games so I probably wouldn t even buy a Linux game).

• [1] http://doc.coker.com.au/fiction/free-short-stories/
• [2] http://doc.coker.com.au/fiction/free-books/
• [3] http://www.bruceongames.com/2010/01/05/304149300-stolen-from-activision/

Well that is the end of my time off, tomorrow it is back to work. However my break started with snow and it seems its finishing that way.


Unfortunately despite my previous enthusiasm this weather is becoming a little tedious. Schools have been closed today so I had to child mind instead of getting anything done.

Speaking of getting things done...I have completely failed to do anything productive whatsoever for the last couple of weeks. I have played some computer games, watched some TV read some books and listened to music and you know what? Aside from a tiny guilty feeling, I have thoroughly enjoyed myself.

I suppose I should apologize for not contributing to all those open source projects that could have used my help, but that would imply remorse which I seem to be lacking. Oh well

Mentioning music, I have acquired a selection of new music over the holiday which I ought to talk about. In no particular order:

• Iron and Wine - "Our Endless, Numbered Days"
• Paramore - "All We Know Is Falling", "RIOT" and "Brand new eyes"
  
• Phoenix - "Wolfgang Amadeus Phoenix"
• Lady Gaga - "The Fame Monster"
• The xx - "XX"
• Muse - "The Resistance"
• Cymbals Eat Guitars - "Why There Are Mountains"
• St. Vincent - "Actor"
• Carter Burwell - "Twilight-The Score"
• The Darkness - "Permission to Land"
• Oasis - "Stop the Clocks"
• Chameleon Circuit - "Chameleon Circuit"
• Hank_Green - "I'm So Bad at This" and "So Jokes"

Unfortunately none of it was outstanding which sometimes happens when I take a punt on a large pile of new music. On the upside none of it was particularly bad either, so a kind of "no win" deal. Though on re-listening the "XX" album is starting to grow on me and the albums from the DFTBA label are so inexpensive they are more than worth it.

Once a month, I take our recycling to the road. This is something far more annoying than most city-dwellers may suspect. For one thing, the road is about 1/4 mile away. Over a month, we fill up at least three large plastic trash cans full of recycling. If I walked each of those there to be picked up, then walked them back to the house after, I d have walked 3 miles. So, I use our $75 pickup to get the trash to the road. The process is a long one, and it starts two days before the recycling is due to be picked up. Monday, 5:30PM: Find an outdoor extension cord in the basement. Put on snow-suitable shoes. Go outside, plug the extension cord in at the elevator. Grab the battery charger, hook it up to the pickup. Unfurl and untangle the extension cord, plug it in. Blog about it if desired. Tuesday, 5:30PM: Go outside. Disconnect the battery charger, coil up the extension cord. Plug in the air compressor and get it going. Run the hose to the tire that keeps going flat and hope it reaches. Fill up the tire. Return the air compressor hose to the elevator, turn off the air compressor. Go to pickup and actually get in. Remove the bungee cord from the brake pedal that is holding it up, preventing the brake lights from being on 100% of the time and draining the battery even faster. Start pumping the accelerator. It s not a fuel-injected engine, after all. Crank the engine. Stop cranking, pump some more, trying to avoid the hole in the floor under the accelerator. Eventually get it started, hopefully. Roll down the windows to prevent the exhaust fumes from getting too strong. Drive the stuff to the road. Unload the three cans, plus all the extra cardboard from Christmas that didn t fit in them and is now soggy and snow-covered. Drive the pickup back to the yard. Park it. Replace the bungee cord on the brake pedal so it ll start again tomorrow. Wednesday, 5:30PM: Dress warmly yet again. Go to the pickup, hoping the tire is still inflated and the battery is still charged. I m going to be optimistic and assume they are. Hope that the pickup starts after only a few minutes of pumping. Go back to the road, only to find that the Kansas wind has scattered the trash cans and their lid all over the county. With luck, I ll be able to find them on foot. Sometimes, I have had to get in the pickup and drive down the road to find all my lids. Stash all of this in the pickup, drive it back to the yard, park, replace the bungee cord, and close it up for another month. I m promising myself that when I upgrade to a better pickup, it ll be one worth at least $200.

The MirBSD Midi-ISO (bi-arch manifold boot) and NetInstall for both i386 and sparc have been upgraded to the 20091226 snapshot (sorry for the delay). A separate news announcement will be done when a full ISO (MirBSD + MirGRML) is done. Other than that, I have fixed a couple of things all over the place, jupp for example. The planned release of mksh R39b is still not done though, as I m only human as well, and too much hacking isn t something one can do without relaxing some in between. On the Debian front, my RCBD #1 was continued, here s #1 results:

• Carry over from day #1
• bug #552791 acorn-fdisk Copyright file does not contain verbatim copy of the license or a pointer to one
• bug #562647 gidentd Does not work with ipv4 after recent change in netbase
• bug #558812 dietlibc incorrect license in debian/copyright
• bug #531937 autossh FTBFS on mipsel due to missing -fPIC
• New ones
• bug #563522 acorn-fdisk cleanup patches
• bug #563525 gidentd cleanup patches

Explanations: I did go overboard during the first patching session, but I suppose this is what the NM learning period is for too. The autossh maintainer said thanks and will probably integrate my patches, so I don t need to NMU. I could close the dietlibc bug. The other two didn t look as good, I had to separate the fix for the RC bug (and other required fixes, such as ftp-master rejects there were none though) and my other fixes; I submit the former as NMU diffs again and pointed Zack to the .dsc files, and opened the aforementioned two new bugs with the rest of the diffs, so the proper maintainers can take and apply them. There s questioning if gidentd should be removed (see the PR for more); the acorn-fdisk upstream (arm-fdisk it s called there) is not actively developing but will receive patches; the autossh maintainer said thanks but I didn t yet hear back from upstream. The binutils as intel_mode bug was fixed upstream and in experimental for my case, but I had to reopen things because the variant documented in binutils-current still doesn t work, so others (who use the new, more intel-like, syntax) don t run into it. Luk sent me a request to do more mipsel-FTBFS-due-to-toolchain-bugs workarounds. Will do (but can t promise to do so before the upcoming BSP. Robert Millan incorporated something like manifold-boot into GRUB 2, after I described it to him (the debian-bsd@ people are currently sorting out some heisenbugs with it, though). Now there s three variants (but then, this helps spotting bugs that don t appear in all implementations). sendmail 8.13.4 is out, I wonder when OgreBSD will upgrade I could do it myself again, but this time it s not that urgent. Still waiting for the TLS extension, though

RCBW - #17 Given that my "most blogged" Debian activity since last summer has been RCBW, what else could be my first post of 2010 if not a RCBW update? With further ado, here are this week squashes by yours truly:

• bug #559898 - flowcanvas - fixed in unstable, tag as needed
• bug #562503 - man-db - downgrading severity (RC-ness was in zlib, which is now fixed)
• bug #541610 - docbook2odf - switch dep away from (now gone) libxml-sablot-perl
• bug #528088 - ttf-aenigma - fonts are actually free, as detailed in debian/copyright
• bug #544113 - tiger - remove dep on essential package (bonus: misc deps/debhelper fixes)
• bug #561466 - ttf-beteckna - rebuild against fixed defoma (thanks Julien Cristau who spotted a wrong fix of mine)
• bug #561470, bug #549987 - halevt - add missing dep on policykit (bonus: QA upload with misc fixes)

No particular remarks, but a long overdue tip of the week: the turmzimmer BTS interface is a great tool for bug squashing. Still, it has a issue with comments which can slow down some activities, but which you can help getting better! First of all you should know that comments are per-package and not per-bug (which might not be obvious, especially since often packages listed on the page have just one bug per-package). That means that you should generally mention the bug you are referring to when you post comments, unless the comments are really package-specific. Additionally, old comments are persistent, so you might get baffled by old comments that "invalidate" current issues. So, if you notice a comment that predates the actual bug numbers, please remove the comment by editing it and stating "no comment" in the form. Ah, happy new year (in case you care about sharp, calendar-based time partitions).

Two weeks after the Rcpp 0.7.0 release, Romain and I are happy to announce release 0.7.1 of Rcpp. It is currently in the incoming section of CRAN and has been accepted into Debian. Mirrors will catch up over the next few days, in the meantime the local page is available for download too. A lot has changed under the hood since 0.7.0, and this is the first release that really reflects many of Romain's additions. Some of the changes are

• A new base class Rcpp::RObject that replaces RcppSexp (which is still provided for compatibility); it provides basic R object handling and other new classes derive from it.
• Rcpp::RObject has real simple wrappers for object creation and a SEXP operator for transfer back to R that make simple interfaces even easier.
• New classes Rcpp::Evaluator and Rcpp::Environment for expression evaluation and R environment access, respectively.
• A new class Rcpp::XPtr for external pointer access and management.
• Enhanced exception handling: exception can be trapped at the R even outside of try/catch blocks, see Romain's blog post for more.
• Namespace support with the addition of a Rcpp namespace; we will be incremental in phasing this in keeping compatibility with the old interface
• Unit test for most all of the above via use of the RUnit package, and several new examples.
• Inline support has been removed and replaced with a Depends: on inline (>= 0.3.4) as our patch is now part of the current inline package as mentioned <here.

As before, fuller details are in the ChangeLog on the Rcpp page.

Most readers probably have installed Debian version 5.0 (Lenny) or an older version and are using shadow passwords with the md5 hash algorithm. This is not bad but not good enough. You can find out the details by looking at /etc/passwd and /etc/shadow (as root). The second column in these files should be a simple character x in /etc/passwd and a string like

$1$biMft/Pr$Lo3zPpiItdLZrzx8t/mTy0

in /etc/shadow. The number 1 between the 1st and 2nd $ sign means md5, the following string biMft/Pr is the salt and the last string Lo3zPpiItdLZrzx8t/mTy0 is the actual hash for the password ('testmd5' in this case). The salt is used to avoid attacks based on precomputed hash tables.

The package pam has switched to the stronger sha512 algorithm in version 1.1.0-2 on 31st august 2009. Look for a line like

password        [success=1 default=ignore]      pam_unix.so obscure sha512

in file /etc/pam.d/common-password if you have installed at least version 1.1.0-2. After changing the password I have the new password string

$6$qjc5gFgK$vaz/gLKMyDuhsVOU2oVIkDZrD0.reJM.2Ft3CMEoAsjN/lenvHC2ls6g/MY1ZaYaYBP3HHDOxel1dvTerl17q1

in /etc/shadow. The number 6 means sha512 and the hash

vaz/gLKMyDuhsVOU2oVIkDZrD0.reJM.2Ft3CMEoAsjNlenvHC2ls6g/MY1ZaYaYBP3HHDOxel1dvTerl17q1

is much longer than before.

The pam_unix module in combination with the sha algorithms allows specifying the number of rounds for hashing the password with the argument rounds=... which defaults to 5000. My current machine needs about 20 milliseconds to hash my password. That can be tested with the command

/usr/bin/time -f %U su testmd5 -c true

I have changed the number to 1 million

password        [success=1 default=ignore]      pam_unix.so obscure sha512 rounds=1000000

to make brute force attacks more difficult. After changing the password again the string is

$6$rounds=1000000$Va4plzLi$EtixueZQ1ZQlzQa7eHHsG6UcNvu.EnuCqM79kIyUe82eAZ.JNegn4SBY1RduYlACs0RWLFHD4d//PzQXMsCqk0

with the number of rounds embedded. The su command needs 1.79 seconds now which is an acceptable delay for the login process considering the improved security.

Don't forget to change the root password too if you have set one.

I run a personal server that hosts web space for a few friends. Probably the most popular thing to do with the space is to install WordPress and run a personal blog. A few days ago, I discovered some attackers were abusing one of the sites. Once we upgraded the site to the latest version of WordPress, the attack went away. So I wrote a tool that, every night, emails me a report of the locations of old versions of WordPress. A sample email:

Current version of WordPress: 2.9

oldroommate has WordPress 2.5 in /home/oldroommate/web/oldroommate.com/

Eek! WordPress 2.5 is old!

How it works Each time it runs, it looks at wordpress.org to see what the current version is. The code to do that is written in Python and uses lxml.html. It prints the current version in the report, and it uses it when analyzing WordPress installs. To analyze WordPress installs, it executes locate readme.html, looking for WordPress's tell-tale documentation file. For every such readme.html, if it matches a simple regular expression suggesting it's a WordPress readme file, it performs the following analysis:

• It does some simple seding and greping looking for the version of WordPress described by that file.
• If the version detected is older (as calculated by dpkg --compare-versions), it creates a one-line report like the one above about oldroommate.

If it found any installs of old WordPress, it prints a report like the one blockquoted above to stdout.

How to use it To get emails, I run it with cron. You can add a stanza like this to your crontab (edit it with crontab -e):

@daily cd find-out-of-date-wordpress; ./look.sh

Ta-da, nightly reports.

To get a copy Do a git clone:

git clone git://git.asheesh.org/find-out-of-date-wordpress.git

or browse its gitweb.

Feedback I'm quite interested to hear what others do to avoid old web apps being attacked. If there's another bit of software that monitors web apps for needing upgrades, I'd love to hear about it! Obviously if you have feedback on this tidbit I wrote, let me know. (To me, apt-get doesn't seem to be the answer. Web apps (especially PHP ones) don't usually seem to support keeping the code in one place with multiple different configuration files. And users get excited about the latest and greatest and don't want to wait for me to upgrade, and I can't blame them.) If some of you don't like the Python dependency or anything else, I do welcome patches!

Yes, aptitude can list packages by priority, but what I wanted was a simple list of just package names. I've got two ways of getting a bare list:

Updated with many thanks to Phil Hands.

$ aptitude search '~prequired' sed -ne 's/^i [ A] \([^ ]*\).*$/\1/p' tr '\n' ' ' sort -u fold -s ; echo

#!/bin/sh
# Licence: GPL-3+
FILE= find /var/lib/apt/lists/ -name '*unstable*Packages'
LIST= grep-dctrl -n -s Priority '' $FILE sort -u

for P in $LIST; do
[ "optional" != "$P" -a "extra" != "$P" ] continue
echo "Priority: $P"
grep-dctrl -n -s Package,Priority "$P" $FILE sed -ne '/^'$P'$/!h;/^'$P'$/ g;p ' tr '\n' ' ' fold -s
echo
done

The advantage with the script is that is lists all packages in all priorities except optional and extra and can be restricted to individual Packages files and therefore to individual repositories and components. (Change the find expression if you have more than one repository specifying unstable or adapt the script to accept a path to the relevant Packages file on the command line.)

I really ought to have used perl because pattern matching in perl is a
lot more natural to me than using sed or awk. I do tend to chain a
couple of sed commands, just so that it's easier to work out which bit
is broken when it breaks.

The final version of this script will end up in emdebian-grip-server,
converted to perl with POD content, translated and reading the Packages
files using Debian::Packages::Compare.

$ ~/scripts/list-priority
Priority: important
adduser apt-utils apt aptitude bsdmainutils libbz2-1.0 cpio cron libcwidget3
debian-archive-keyring dhcp3-client dhcp3-common dmidecode ed libgdbm3 gnupg
gpgv groff-base ifupdown iproute iptables iputils-ping logrotate libept0
libsigc++-2.0-0c2a libtasn1-3 libusb-0.1-4 man-db manpages module-init-tools
nano libncursesw5 net-tools netbase netcat-traditional libnewt0.52 whiptail
libssl0.9.8 libpopt0 libreadline5 libreadline6 readline-common rsyslog
tasksel-data tasksel libwrap0 info install-info traceroute udev vim-common
vim-tiny wget libxapian15
Priority: required
libacl1 libattr1 base-files base-passwd bash coreutils debconf-i18n debconf
debianutils diffutils dpkg e2fslibs e2fsprogs libcomerr2 libss2 libc-bin libc6
findutils gcc-4.3-base gcc-4.4-base libgcc1 libstdc++6 grep gzip hostname
lsb-base libdevmapper1.02.1 lzma liblocale-gettext-perl libselinux1 libsepol1
libtext-charwidth-perl libtext-iconv-perl libtext-wrapi18n-perl libunwind7 mawk
libncurses5 ncurses-base ncurses-bin libpam-modules libpam-runtime libpam0g
perl-base procps sed sensible-utils login passwd libslang2 initscripts sysv-rc
sysvinit-utils sysvinit tar tzdata bsdutils libblkid1 libuuid1 mount util-linux
zlib1g
Priority: standard
apt-listchanges at bash-completion bc dc bind9-host dnsutils libbind9-50
libdns53 libisc50 liblwres50 bsd-mailx libsasl2-2 libdb4.8 libdb4.5 libdb4.6
libdb4.7 debian-faq doc-debian doc-linux-text libc6-i386 locales exim4-base
exim4-config exim4-daemon-light exim4 file libmagic1 gettext-base libgnutls26
libgpm2 libkeyutils1 libgssapi-krb5-2 libgssrpc4 libk5crypto3 libkadm5clnt6
libkadm5srv6 libkdb5-4 libkrb5-3 libkrb5support0 less lsof libcap2 libedit2
libevent-1.4-2 libgc1c2 libgcrypt11 libgpg-error0 libgssglue1 libidn11
liblockfile1 libnfsidmap2 librpcsecgss3 libunwind7-dev libxml2 m4 mime-support
mlocate mutt ncurses-term ftp telnet nfs-common libldap-2.4-2 openssh-client
patch libpci3 pciutils libpcre3 perl-modules perl portmap procmail
python-central python-minimal python python2.5-minimal python2.5
python-reportbug reportbug wamerican libsqlite3-0 tcpd texinfo time ucf w3m
whois

Hmm, next step is to check the "Priority: standard" set for Emdebian - it looks far too large for what would be standard on an embedded device. LDAP will be dropped for one - and what is a -dev package doing in there? libunwind7-dev isn't even installed on my amd64 Debian desktop!

Probably check the important and required listings too, then tweak what we need to change via the archive overrides for Emdebian Grip.

Come to think of it, I might need a PHP version of this to read live data from the repository, documenting the changes.

This has been a big decade for me. Ten years ago, I was living in Sunnyvale, in the heart of Silicon Valley; VA had just had its crazy IPO; my unvested options were worth over a million dollars on paper. I remember sitting in first class[1] on the way out of Atlanta after the holidays and wondering what one did with a million dollars. Avoid working too much? Have a lot of interesting computers? Build a distributed house[2]? Mostly I decided to not count my chickens, which turned out to be the right choice. Six months later, I started debian-installer, in many ways my biggest project to date. In spring of '01, VA was crashing; I was sick of the Valley; gloom was everywhere. I took the hint and quit, cashed in my options for five figures, not seven, bade forewell to the many people I'd come to know, and moved back east. By reading my old journal, I can tell that subconciously, I had long before decided what to do when it was time to leave the Bay Area. So without really thinking too much about why I was able to do this, about how not just my own family, but others were indirectly making it possible, I moved to a 150 acre wooded "farm", and mostly went off and did mostly my own thing for three years. This was such a huge life change. I lived in a building affectionatly called "the shack". It was renovated from a chicken coop in the 70's and leaked during heavy rain, but I preferred it to the main house (warmer). I went back to dialup for internet. I hauled groceries and most everything half a mile in by hand. There was an outhouse and washwater hauled from a creek. I cut barely enough firewood and readapted my body from California to cold. My main tech project during this time was mooix. In some ways my largest failed project, but it was neat to be able to go off and do such a thing. This was a blissful, reflective, peaceful, amazing time. Parts were a second childhood. Somewhere in there I properly confronted my mortality. And learned to use a chainsaw. And over the course of three years, I dug myself into a rut, a hole. I wasn't going anywhere. (Except for a month in Honduras, and my first DebConf in Oslo.) In '04, I found myself talking on the phone to a cosmonaut in a house that had been without power for days in a hard winter. I realized I was ready to leave the farm if I could figure out how to do it, and I almost took him up on the job working on his new Linux distro -- but at the last minute Oslo connections suggested I work for SkoleLinux instead, and so I did. Soon after moving into a rented house in town, I was jaunting off to Brazil for a month, a great time. Followed by a productive year where I got heavily back into debian-installer, started Debian's testing security tream, and generally worked a lot. The next summer I fooled around with an RV, and sort of fell into a job with ADS. But my time on the farm had changed me, I think. I ended up going back to the farm for a year and working part time from there. During this time I also started ikiwiki and other smaller projects. One day I came back from a camping trip and found the kitchen had been torn down in my absence. Reminded unsubtly that I was not the only one with a claim to the place, I got out, though I somehow suspect not for good. For the last four years, I've been more outwardly settled, buying a house in town, continuing to work at the same job, starting little projects like pristine-tar, mr, and etckeeper. About two years in I began to feel antsy sometimes (and had a minor medical scare), and set up the yurt as a relief valve so I can have crazy adventures in the cold woods again. Still, I sometimes feel that a big change is building up. From here, this decade feels like it contained endless changes, important decisions, interesting projects, strong experiences. The 90's, containing only little things like college and moving to California, pale in comparison. Maybe the naught's will be the decade I always remember.. or maybe not. Anything could be around the corner.

---

1. Bumped up due to an airline scheduling snafu, not because I was throwing money around.
2. Concept of mine perhaps not worth explaining here although it was oddly similar to how I ended up living at the farm. Good ol' subconcious..

The following developers got their Debian accounts in the last month:

• Ludovico Cavedon (cavedon)
• Gilles Filippini (pini)
• Christopher Taylor (ctaylor)
• Oliver Korff (okoli)
• Franck Joncourt (franck)
• Antonio Radici (antonio)
• Emmanuel Bouthenot (kolter)
• Tim Retout (diocles)
• Dario Minnucci (midget)
• Hector Oron (zumbi)
• Albin Tonnerre (lutin)
• Modestas Vainius (modax)
• Christoph Egger (christoph)
• Andrea Veri (and)

Congratulations!

Christian writes about the nightmare of using the proper name for people in an international context. I ll add that even in the same country/culture, you can have a hard time, because of another factor: the generational context. People don t necessarily address other people from their generation the same way as they would address people from different generations. And these rules may also vary when people age. Even people from the same generation can have different ways to address people. All in all, naming someone is complicated everywhere, though it is more complicated in international context. I ll also add that there are at least two persons involved in a conversation, and that while it is nice that the sender tries to make an effort to address a person the proper way, the recipient should also not take it bad when someone from far away uses the wrong name, and don t make a fuss about it. That s called tolerance. To give an example that is not about names, but about such tolerance, President Obama recently made a visit to the Japanese Emperor, and bowed before him. If you are living in the US, you probably heard about that, because the media there made a fuss about the US president bowing. It so appears that by japanese standards, he indeed did bow too deeply. But the japanese didn t care much. He s not japanese, he doesn t have the cultural background, and therefore can be wrong. At least he tried. What actually made news in Japan is the US news making a fuss about it.

On the train to my parents' house for Christmas I finished up a wild run through the book Coders at Work. The book isn't even mine, but I'd been borrowing it very often, sometimes to the chagrin of its owner, because I could barely put it down. The book is a collection of interviews with 15 great programmers of our time, starting with Jamie Zawinsky and ending with Donald Knuth. It's written in an interview style each interview starts with a brief introduction to the person being interviewed, summarizing what the person is known for and what he or she has accomplished and a few of the highlights of the interview, and then a transcript of the interview follows, with the author/editor, Peter Siebel, will saying something or asking a question, and the interviewee responding. I was skeptical about this format at first because I feel like it can be an easy way out of good editing and make the reader have to do the work of the editor, but on finishing I think that Siebel uses the format to his advantage in this case. For one, the speech format allows the reader to really form a picture of how the person being interviewed speaks and would act in a conversation. Jamie is somewhat bitter and pretty informal. Brad Fitzpatrick is flippant and energetic, his speech littered with profanity and colloquialisms. Others seem more stately and verbose Joe Armstrong's responses can go on for a page or more. In this way, not only do readers learn something about what these greats have learned about programming, but we also feel a bit more like we've met or know them, and can connect to them more as people. I always have this problem where I want to read computer books, but often computer books seem inextricably tied to the computer, so there's this dynamic of reading a bit and then wanting to get on a machine and try something out, write some code, play around especially with books focused on a specific language. Coders at Work retains some of this computer-book dynamic in that I constantly encountered things that I want to investigate or play around with more: Erlang, OCaml, various papers and essays, Knuth's literate programs, and books such as Higher Order Perl and others. Siebel makes a point to ask each person what her short-list of books and papers programmers should read are, so this book is a great source of pointers to other reading material. Unlike a more specific book, however, keeping a list in a notebook was enough to settle the mind to read away-from-a-computer for chapters at a time. It's obvious that despite the interview format, Siebel has done some serious editing. None of the prose is boring to read, and I can't imagine that the text is a straight transcript of how the interviews went. He also has arranged the interviews in an order such that different interviews play off each other. In Branden Eich's interview, for example, he disparages the book Design Patterns:

I never bought the Gamma book. Some people at Netscape did, some of Jamie Zawinski's and my nemeses from another acquisition, they waved it around like the Bible and they were kind of insufferable because they weren't the best programmers.

In the next chapter, Joshua Bloch names it as a book he thinks programmers should read:

Another one, which I have slightly mixed feelings about but I still think everyone should read, is Design Patterns. It gives us a common vocabulary. There are a lot of good ideas in there.

Similar plays, such as Ken Thompson and Fran Allen disagreeing on the badness of C, happen in later chapters, tieing together the different chapters and illustrating how even really good programmers disagree on the Right Thing all the time. Clearly the craft of programming is no settled thing. Besides the general structure of the book being well thought-out, the material is generally thought-provoking and interesting. One thing that stood out to me was Joshua Bloch describing what he called the "empathy gene", which is what a programmer has to have if he's going to be able to design good APIs and programming languages he has to be able to put himself in the shoes of the person who will be using the language or API. This is one thing that differentiates how different programmers can be good at different things. Another thing that stood out to me is that many of those interviewed stated that they don't use much in the way of modern tools and IDEs Joshua Bloch and Simon Peyton-Jones both touch on this, just to name a couple examples, even though some say that they think using these tools would make them more productive, especially when it comes to refactoring. This is a testament to the power of inertia sometimes there is just no chance to be unproductive now in order to be more productive later. Or perhaps just a sign that a programmable text-editor can stand on the same level as a heavier tool in terms of productivity in the right hands. I could go on with examples, but the conclusion here is that I thoroughly enjoyed Coders at Work, and I think it is a book that is well-worth the time spent reading the entire thing.

Christian writes about differences on the international level; how even something as simple as people's names isn't always easy, and that one has to be quite careful to avoid calling people 'Dear Verhelst', or some such, because you assumed that their first name was indeed their given name, while in their culture the last name is the given name. Funnily, Christian advocates using capital letters to clarify which part of one's name should be considered the family name. Sorry to disappoint you, Christian. Writing one's last name with all capitals is a French typographical convention. While it is understood in other places, it is not often used outside of France (except in organisations with a strong link to France, obviously); and it is, in fact, shouting. It is also not actually helping any. I should, in fact, somewhere in my mailbox archive have an email reply from a South Korean guy that starts off like so: Dear TIA, to which I then had to explain that no, TIA is not my first name, it is an abbreviation for "Thanks In Advance". Note the use of capitals.