While gzip is the standard Unix tool when it comes to compression, there are other tools available and some of them are performing better than gzip in terms of compression ratio. This article will explain where you can make use of them in your Debian packaging work. In the source package A source package is composed of multiple files. The .dsc file is always uncompressed and it s fine since it s a small textual file. The upstream tarballs can be compressed with gzip (orig.tar.gz), bzip2 (orig.tar.bz2), lzma (orig.tar.lzma) or xz (orig.tar.xz), so choose the one that you want if upstream provides the tarball compressed with multiple tools. Put it at the right place and dpkg-source will automatically use it. Note however that packages using source format 1.0 are restricted to gzip, and the main Debian archive currently only allows gzip and bzip2 (xz might be allowed later) even if the source format 3.0 (quilt) supports all of them. The debian packaging files are provided either in a .diff.gz file for source format 1.0 (again only gzip is supported) or in a .debian.tar file for source format 3.0 (quilt) . The latter tarball can be compressed with the tool of your choice, you just have to tell dpkg-source which one to use (see below, note that gzip is the default). In a native package, dpkg-source must generate the main tarball and you can instruct it to use another tool than gzip with the --compression option. That option is usually put in debian/source/options:

# Use bzip2 instead of gzip
compression = "bzip2"
compression-level = 9

For 3.0 (quilt) source packages, this option is not very useful as the debian tarball that gets compressed is usually not very large. But some maintainers like to use the same compression tool for the upstream tarball and the debian tarball, so you can use this option to harmonize both. In native packages, it s much more interesting: for instance the size of dpkg s source package has been reduced of 30% by switching to bzip2, saving 2Mb of disk. In the binary packages .deb files also contain compressed tar archives and by default they use gzip as well:

$ ar t dpkg_1.15.9_i386.deb
debian-binary
control.tar.gz
data.tar.gz

data.tar.gz is the archive that contains all the files to be installed and it s the one that you can compress with another tool if you want. Again this is mostly interesting for (very) large packages where the size difference clearly justifies deviating from the default compression tool. Try it out and see how many megabytes you can shove. Another aspect that you must keep in mind is that those alternative tools might use important amount of memory to do their job, both for compression and decompression. So if your package is meant to be installed on embedded platforms, or if you want to build your package on low-end hardware with few memory, you might want to stick with gzip. Now how do you change the compression tool? Easy, dpkg-deb supports a -Z option, so you just have to pass -Zbzip2 for example. You can also pass -z6 for example to change the compression level to 6 (it s interesting because a lower compression level might require less memory depending on the tool used). The dpkg-deb invocation is typically hidden behind the call to dh_builddeb in your debian/rules so you have to replace that invocation with dh_builddeb -- -Zbzip2 . If you are using a debhelper 7 tiny rules files, you have to add an override like in this example:

%:
	dh $@
override_dh_builddeb:
	dh_builddeb -- -Zbzip2

If you are using CDBS, you have to set the variable DEB_DH_BUILDDEB_ARGS:

include /usr/share/cdbs/1/rules/debhelper.mk
[...]
DEB_DH_BUILDDEB_ARGS = -- -Zbzip2

I hope you found this article helpful. Follow me on identi.ca or on twitter. One comment Support my work

Is Low Quality in Italian Academia related to the choice of Low Quality Software? Diego Gambetta and Gloria Origgi wrote an interesting paper titled L-worlds: The curious preference for low quality and its norms [1]. The paper describes how in Italian universities (and large portions of Italian life) there are covert agreements that both parties in a transaction (the employee and the employer or the buyer and the seller) will deliver less than agreed while pretending that they are offering the agreed exchange. People who offer high quality in exchanges are discriminated against because they make people who offer low quality in exchange feel guilty. Nathan suggests that this is the explanation for people choosing to pay for inferior software from Microsoft instead of getting superior software for free [2]. Now it does seem quite plausible that someone who is offering low quality goods in the manner of Italian academia would refuse to consider software from any company other than Microsoft, after all the easiest way of selecting software is to phone a MS representative and be told exactly what to buy. But I don t think that this explains even a significant fraction of the people who refuse free software. There is no direct analogy between bilateral agreements to produce low quality and the choice of MS Software because MS Software is quite expensive (they demand what would be considered a high quality trade in the jargon of the paper). If someone was to buy one of the cheaper laptops in Australia (around $650 new) and upgrade to the full version of Windows 7 along with purchasing the home version of MS Office then the cost of software would be almost as much as the cost of hardware. If they wanted one other MS product then the cost of MS software would probably be greater than the cost of hardware. Hardware costs are steadily falling and MS prices are only increasing, for people who use MS software we should expect that soon the MS tax will be the majority of the costs of running a typical PC. Good Reasons for Choosing MS One thing we have to consider is that there are people who have good reasons for using MS software. One example is the companies that depend on proprietary applications which are central to their business. When the entire company s data is stored in an undocumented proprietary database it s really not easy to change to a different application even when everyone in the company knows the software to be of amazingly low quality. If the vendor of the proprietary application in question decides to only support MS Windows then it s customers (victims?) have no choice about which OS to use. One interesting thing to note about such companies that are locked in to proprietary software is that the amount that they spend per year on license and support fees is usually greater than the cost of hiring one good programmer. If a few such companies formed a consortium to develop free software to manage their business where each company paid the salary of one programmer then after a couple of years of development they could move to the free software and reduce their operating expenses. Another category of users who have a good reason to choose MS is the people who play games seriously. If you want to play games then MS Windows does offer some real advantages. The price of games will usually be a fraction of the hardware cost (the serious gamers spend a lot more on hardware than most people) and MS Windows is apparently the best PC OS for commercial games. Personally I ve found that there are more than enough free games on Linux to waste my time, Warzone 2100 [3] is one that I currently play, and I ve tried Battle for Wesnoth [4] in the past and found it too time consuming and addictive. How a Preference for Low Quality could lead to Microsoft I think that everyone who has any significant experience in the computer industry has encountered companies that have large areas of low quality. This generally tends to be in large corporations as small companies can t afford the waste. In some large corporations Linux on the desktop is never considered, even when people are hired as Linux sysadmins and there are obvious productivity benefits to having the same OS on the desktop as on the servers (even if two desktop PCs are required so that proprietary software can be run on MS-Windows). Major wrote a good satire of the corporate IT non-working culture with a comparison to medical work [5], it illustrates the principle of a coalition to ensure low quality. He later documented how he was sacked by the low quality coalition at a company that uses a lot of Microsoft software [6]. So it does seem that when customers don t care at all about the quality of the result it does help drive some sales for Microsoft. But that doesn t explain the market share that they have. It takes a lot of work to get Market Share without Quality Microsoft has spared no effort in gaining market share. Every possible effort including buying out small competitors, aggressively FUDing competition, using all manner of legal attacks (including the threat of patent suits), and deliberately breaking standards has been used. I think it s reasonable to assume that the MS senior management are not entirely stupid, they do so many things that are unethical and possibly illegal because they know that they need to do so to maintain their market share. The result is that the market capitalisation of MS is almost as high as that of Apple and Apple makes vastly superior products. Given the amount of effort that MS uses to keep market share it seems apparent that they aren t just relying on customers not caring about quality. Should users have to Understand Computers? My observation is that most users don t want to know much about how their computers work. The desire to understand computers seems to be about as common as the desire to understand cars, people just want to buy one that looks good and have it work. The difference is that cars are very compatible while computers aren t. Cars have the same controls in the same places and large parts of the design are specified by law so that they can t differ between models. Proprietary software is usually intentionally incompatible with other software (both open and proprietary) to try and gain a competitive advantage. Hardware is often incompatible due to the rapid developments in technology and the requirements for new interfaces to take advantage of new features. In concept it seems reasonable for someone who is about to spend $30,000 on a car and $1000 on a computer (for hardware and software) to spend 30 times longer considering which brand of car to buy. One could argue that more than 30 times as much consideration should be given to the car as most people can t afford to discard a car that they don t like. As people spend a few minutes considering which brand of car to buy they can be excused for spending a few seconds considering which type of computer to buy. But once a choice has been made about which software to use it s very difficult to change to something else, while in comparison it s easy to drive a car that was manufactured by a different company. So a poorly informed choice made at an early stage can have costly long-term affects when buying software. If we had mandated open standards for file formats and data interchange then users would be able to make choices that don t result in their data being locked in to some proprietary format. Such standards could be set through government tender processes, if every government agency was to only buy software that complies with open standards then the proprietary software vendors would scramble to make their products less incompatible. The result would be that bad choices in purchasing software could become learning experiences that result in better purchases in future instead of being a lock on users that forces them to keep using the same software that doesn t satisfy their needs. Conclusion I think that the best thing about the paper by Diego Gambetta and Gloria Origgi is that it highlights the issue of low quality. No-one wants to be considered a loser, so maybe this can encourage people to strive for high quality (or at least try to make their work suck a little less). Regardless of the conclusion they eventually reach, it s probably good for people to occasionally wonder do I suck? .

• [1] http://www.sociology.ox.ac.uk/documents/working-papers/2009/2009-08.pdf
• [2] http://www.advogato.org/person/ncm/diary.html?start=405
• [3] http://wz2100.net/
• [4] http://www.wesnoth.org/
• [5] http://blog.ninbtma.org/2007/10/will-i-miss-it.html
• [6] http://blog.ninbtma.org/2007/11/just-forward-fucking-ticket_6090.html

The Apple universe is happy to live with whatever Steve Jobs allows to get in. Numerous examples going back at least 20 years are available; most recent Apple's law action against a manufacturer offering compatible batteries and now an external harddisk for iPods. Microsoft is obviously the traditional enemy for many (personally, I think there's not much difference between MS, Apple and Google anymore.) So no surprise that MS Exchange plays a role in the story about a remote wipe function of many smartphones. Just being connected to a MS Exchange server allows the administrator of that server to remotely factory-reset the smartphone. I'm not sure if there's a complete list of vulnerable smartphones, but at least iOS, webOS and Android based phones are affected. Personally, I'd say this is a serious security issue. But of course, according to the manufacturers, it is a function for the convenience of network adminstrators so they can wipe stolen phones remotely. Update: Nico: If I'm somebody who is really after your data, I've got your smartphone and I'm clever enough not to connect it to the net before I got the data out. So you're only catching the Oh look what I accidentally found case, which I think is really just giving you a false sense of security. Anonymous: Google is the company with the Do Not Be Evil company motto and the very good press and marketing team touting everywhere how much they do for the good of everybody. Except when it cuts into their earnings. Like really cooperating with the Linux kernel folks. Like living up to what they announced they'd do in China. It's also the company covertly collecting WLAN data, and only admitting to exactly as little as they can get away with when it comes into the open. And, in the end, simple greed will do the rest: it's a powerful big company with plenty of very well paid career positions, and in my experience this company tends to attract people who will use this power in any way they can to increase their income. At this level of business, a company motto like the one of Google is just a few meaningless words. The real company motto is Earn Money. In Whatever Way. Other examples are told by anybody who needed to interact with their so called customer support, either because they got their free account locked or because they had a specific request as a paying customer placing ads on Google (and I'm talking about a company with at least several CHF 10k per year in ads, not just a small nobody.)

Last month I gave a presentation on the interaction between Android and kernel upstream at Linuxcon. The video for that is now available here (requires registration). Contrary to stories you may have heard, I do not dropkick anyone through a window.

There's some parallels between the Android/upstream scenario and Canonical's approach to upstream. Mark wrote a lengthy defence of Canonical's focus on components that they feel need development, while not putting development effort into things they feel are good enough already. That's pretty consistent with the discussions I had with him at the Ubuntu development meeting in Oxford over six years ago. Back then the focus was on taking all the excellent software that already existed and concentrating on providing it as a single polished and integrated product. It was successful - what's easy to forget now is that the first release of Ubuntu was massively more usable out of the box than any other Linux distribution available at the time, and it's absolutely undeniable that its release spurred increased efforts on the part of competitors. But I don't think the same focus is being applied any more.

The most obvious (and most controversial) example is the Ayatana project. Ayatana's a pretty explicit statement that Canonical don't view the existing Gnome UI as being suitable for their vision of the Linux desktop. That's absolutely fine. However, unlike many of the papercut projects, Ayatana is a set of complete reimplementations of functionality that behave differently to their upstream equivalents. There's no meaningful sense in which that's not a fork of Gnome. And, let me emphasise, I don't think that's a bad thing.

Let's go back to Android. While we talk about how much we'd have preferred it if Google had interacted with upstream before implementing wakelocks, the realistic outcome is that there's no way that we'd have accepted them even if they'd been posted years before hardware shipped. If there'd been a productive outcome to that aspect of the conversation it would have involved Android having a very different power management policy. Sometimes you're not going to convince people that something is better without implementing it and seeing how well it works. The problem with wakelocks is not that they exist or that Google feels they work better than the alternative, but that they end up integrated into drivers in a way that can cause divergence between mainline and Android.

Ayatana potentially has the same outcome. At some level, well-integrated applications have to be aware of the environment in which they're running. The application indicator patches are an example of that. Carrying conditional code increases maintenance burden, reduces test coverage and is a net loss in the long run. If upstream Gnome never gains support for this functionality, and if Ubuntu continues to depend on it, then Canonical have a delta that they'll have to carry forever. That benefits nobody.

Forking because you believe that your approach is better is a completely valid development model, but in the long run can cause problems if you don't have a long-term strategy for how to resolve that fork. For all we criticise Google's ability to get Android code into the mainline kernel, they've put orders of magnitude more effort into doing so than Canonical have in terms of getting Ayatana's code into mainline Gnome. This isn't a function of Google's relative size - the Android kernel team is on the order of 10 people. It's not enough of a difference to explain the disparity.

Canonical would be perceived as much better team players if there was an indication of their long-term plan in terms of Unity and the Ayatana projects and getting that code into mainline Gnome and integrating with the Gnome shell. It's completely unsurprising that they're viewed with distrust until that happens.

My tip of day. When you lose control of a process (for example with a process launched with disown or, another example, when you close your SSH terminal with unsynced mutt), thanks gdb.

$ tty
/dev/pts/13
$ gdb --pid= pidof mutt 
(gdb) call close(0)
(gdb) call close(1)
(gdb) call close(2)
(gdb) call open("/dev/pts/13", 2, 0)
(gdb) call dup(0)
(gdb) call dup(0)
(gdb) detach
Detaching from program: /usr/bin/mutt-org, process 2932

Thanks to this comment on superuser.com for the original idea. Edit: correction on dup() calls. Thanks to Peter Pentchev for his explanation in comments.

dpkg-source is the program that generates the Debian source package when a new package version is built. It offers many interesting command-line options but they are often not used because people don t know how to ensure that they are used every time the package is built. Let s fill that gap! It is possible to forward some options to dpkg-source by typing them on the dpkg-buildpackage command line but you d have to remember to type them every time. You could create a shell alias to avoid typing them but then you can t have different options for different packages. Not very practical. The proper solution has been implemented last year (in dpkg 1.15.5). It is now possible to put options in debian/source/options. Any long option (those starting with -- ) can be put in that file, one option per line with the leading -- stripped. Here s an example:

# Bzip2 compression for debian.tar
compression = "bzip2"
compression-level = 7
# Do not generate diff for changes in config.(sub guess)
extend-diff-ignore = "(^ /)config.(sub guess)$"

Notice that spaces around the equal sign are possible contrary on the command line. You can use quotes around the value but it s not required. The debian/source/options file is part of the source package so if someone else grabs the resulting source package and rebuilds everything, they will use the options that you defined in that file. You can also use debian/source/local-options but this time the file will not be included in the resulting source package. This is interesting for options that you want to use when you build from the VCS (Version Control Repository, aka git/svn/bzr/etc.) but that people downloading the resulting source package should not have. Some options (like --unapply-patches) are only allowed in that file to ensure a consistent experience for users of source packages. You can learn more about the existing options in the dpkg-source manual page. Read it, I m sure you ll learn something. Did you know that you can tell dpkg-source to abort if you have upstream changes not managed by an existing patch in debian/patches? It s --abort-on-uptream-changes and it s only allowed in debian/source/local-options. Be sure to subscribe to the RSS feed or to the email newsletter to not miss useful documentation for Debian contributors! 3 comments Support my work

After a few hours of hacking, I came up with Debian Bugs Search. This Ultimate Debian Database tool enables to query different kinds of Debian bugs (especially RC bugs using a multi-criteria search. It is aimed at being a replacement for the unofficial RC Bugs tracker, which has some bugs and fails to display the correct status for some bugs (Debian Bugs Search sees 450 RC bugs affecting squeeze, vs 278 for the unofficial RC bugs tracker). While Debian Bugs Search still lacks some features that the unofficial RC bugs tracker has, it also brings some new features. The main advantage is that it makes it easy to find some classes of interesting bugs, for example: RC bugs affecting squeeze and sid, tagged patch, sorted by time of last change. Why weren t they fixed using the patch? RC bugs affecting squeeze and sid, tagged pending, sorted by time of last change. Why weren t they fixed by an upload if they were tagged pending a long time ago? Bugs affecting squeeze but not sid, and marked as done. Why didn t the fixed packages migrate to testing? I m open to suggestions of improvements, bug reports, or better, patches. Since it is UDD-based, it is quite easy to add more information. My own TODO list is:
- add popcon information (also enable sorting by popcon)
- add a way to display/hide additional columns (popcon, package s versions)

How pivot_root won against a stupid RAID controller This post is brought to you by a very stupid 3ware 9650SE RAID controller, which doesn't understand the concept of growing a RAID array after all disks have been replaced by bigger ones. Or, well, it can, but the procedure, involving sending array dumps to the technical support, then receiving a program to run under DOS (!), doesn't inspire me with confidence. Furthermore, this stupid RAID card doesn't allow easy reordering (if it's possible, and I just missed it, please let me know) of the RAID arrays at runtime, only from the BIOS screen, and I didn't have access to that (only poor man's serial console under Linux). So, after buying new harddrives, and replacing the original ones in the RAID1 array with double the capacity, I realised I have to do a manual conversion. Basically, go into single-user mode, remount all read-only, split the RAID1 array into its (identical) components, change root directory to the second disk, re-partition the first disk, copy the data from the 2nd to the 1st disk, then re-add the 2nd disk to the 1st one (repartitioned) to re-establish the RAID1. All this, on a headless (no keyboard, no VGA) machine, just with a serial console. Of course, if this is just your local machine, you can easily reboot from an external media, and do all this easily from a full user-space. In any case, I'm writing this down, as there seem to be no easily findable guides on how to do the resize on a 3ware with its restrictions. Procedure Disclaimer: you might lose data via this, there are no guarantees. Have up-to-date backups. First, make sure you have power control (you can power-cycle the box if needed) and the serial console works. Also, make absolutely sure that the RAID array is fully consistent, and that you don't have any bad sectors on the drives, as you will lose redundancy during the operation. If /usr is not on your root file-system, then you need to copy /usr/sbin/chroot to somewhere on it. Also, if you have tools for the array that are not on your root file-system, you should copy them there now. In my case, that was tw_cli. First reboot Reboot into init=/bin/sh. Note: I've tried to simply go into single-user mode, or to reboot with emergency, but this doesn't work, because in both modes, beside the init process with PID 1 and your shell, there's another init process (the parent of the shell), and I didn't know how to tell it to re-execute itself. To the main init process it's easy, you just run telinit u, but that child keeps the old root directory mounted. If it's possible to re-exec that child too, then this step can be skipped please let me know how! Changing the root mount Make sure everything is mounted read-only (should be), then split the raid array into its component drives, using whatever tool the card needs. For me, it was:

# tw_cli /c0/u0 migrate type=single

The second drive now should be visible e.g. as /dev/sdc, so mount it somewhere, and pivot_root to it:

# mount -o ro /dev/sdcX /mnt
# cd /mnt/
# pivot_root . mnt/
# exec chroot .

At this point, the new root (on some partition on /dev/sdc) is mount at the root, and the exec call replaced the current shell with another one that has the right root directory. So nothing should keep the old root directory (at /dev/sdaX) in use, and umount /mnt (or umount /dev/sdaX) should succeed now. Recreate the first raid array Now, since it's no longer in use, you can delete and recreate the first raid array. Once it's available, you have two choices:

• either do a bit-for-bit copy, using dd, from /dev/sdc to /dev/sda, then run sfdisk -R /dev/sda to re-read the changed partition table on the target drive
• or repartition /dev/sda if you also want to change the partition layout

I did the latter, since I wanted to use the new alignment standards (the initial installation was still using sector 63 for the boot partition, etc.) as a future preparation for either SSD or 4K-sector drives. So I used:

• fdisk -cu /dev/sda, and created partitions accordingly
• then re-did my physical volume for LVM, but with --dataalignment 1024k, to have the same megabyte alignment (yes, I went a little overboard)

In any case, either copy the entire disk, or each partition in turn, or possibly pvmove from the old to the new disk, etc. Switch back the root mount At this point, /dev/sdc is no longer used, and you can again change the root directory. This is similar to the first pivot_root, but now using /dev/sda as a target. All the instructions are the same. Convert back to RAID1 Now, since (in our example) /dev/sdc is no longer in-use, you can delete that drive and rejoin it to the first drive, recreating the RAID1 array. In my case:

# tw_cli
/> /c0/u2 delete
blah blah [y/n] y
/> /c0/u0 migrate type=raid1 disk=X

At this point, the machine is again redundant (or will be once the resync finishes), but it still needs one more step or it won't boot correctly. Reinstall the boot-loader Now, for reinstall the boot loader, you'll probably need to mount /boot (and mount it read-write), and do whatever you need. In my case, I re-ran lilo, but grub-install /dev/sda should also work. Final reboot And now, you only need to reboot into the normal, resized system. Since you can't easily use shutdown when booted with /bin/sh as init, just use /proc/sysrq-trigger to sync/unmount/reboot. Thoughts While I use a hardware RAID controller for the battery backup it offers, I have never thought it cannot do such a simple operation (compare with mdadm --grow /dev/mdX --size=max). Second, I knew about pivot_root but have not used it before. It's a very very nice thing, and the only downside is that you can't really use it with more than a few processes, since it's hard/impossible to change a random process' root directory. But anyway, it works, and it's damn handy for doing things to the root file-system or in general to the root drive (e.g. just repartitioning it) when you can't boot from another media (e.g. remote machine).

The Exoclimes conference ended on an interesting perennial problem: what should we call these things we work on? The climate folks pointed out what the astronomers have grown used to: the telephone numbers we use as planet names. The two most studied Hot Jupiters, for example, are HD209458b and HD189733b. Come on, we could do better than that. In fact the naming worries started earlier when Frank Selsis asked people to avoid calling the planets he works on "super-Earths" (or should that be Super-Earths?). Dimitar Sasselov had accidentally kicked up a kerfuffle in the media a few weeks back by calling them "Earth-like" planets, when he meant "Earth-sized" planets. Selsis pointed out non-scientists will interpret "super-Earth" to mean "like Earth, only Better!", when we mean its just a bigger ball of rock and ice than Earth. But what to call them? Well, at least one planet already has a name, sort of: HD 209458b is frequently called 'Osiris'. And there is a published list of possible planet names. But Osiris the Egyptian god is more famous than Osiris the planet, confusing google. And people are going to assume things based on the planet names: are we really going to name a planet Vulcan? One suggestion was that if we found a system with seven planets we could call them after the Seven Dwarfs, but this was ruled out: don't mention the war (but dwarfs could also mean brown dwarfs!) At the end of the conference, there was show of hands: who agreed with naming planets? of about 100 present, all in favour, none against. The delegate to the IAU commision on the matter, which had voted against, duly noted. Perhaps we will name them then. Tags names, exoplanets, astronomy

[Nothing related to Debian here, move on if you don't care about my recurring obsession with the end of the world, Cassandra pains, or Big Phat Living Room Revolution yearnings].

Although I am still looking for a community of anarcho-primitivists that want to accept me joining their frugal living, and simply slip off the social construct, the system, the fucking face of earth, call it what you will, I could find some comfort in my favourite feminist blogger's take on the subject of my midlife crisis regarding ecology, capitalism, patriarchy, child bearing, overpopulation, meat consumption and all the rumination of consuming thoughts that (still) make my life miserable and have kept me running away from human contact for a year.

[Emphasis mine]

At what point does human culture depart from the Natural? With the invention of computers? TV? Cars? The cotton gin? Electricity? Taco stands? Gunpowder? The printing press? Written language? Shoes? Crop cultivation? Yurts? The wheel? Did humans become unnatural when the good old days of picturesque, endless agrarian toil, feudal oppression, unchecked disease, ignorance, and death from dysentery at 35 turned into the bad new days of urban post-industrial capitalism where a pound of fair-trade organic coffee costs $12.99 and your email inbox is full of spam?

ah. Everything humans do, or have ever done, is natural. We can t do anything else. The idea that modern culture is un-natural is nostalgic and inaccurate. Living off the grid in a yurt is good in some absolute sense, whereas driving an SUV from a suburban bungalow to the stripmall is bad ? Come on. This a romantic, but misguided view. The cosmic reaction to a 20 Ford Expedition is the same as to a sanctimonious Prius: bupkis. The universe doesn t give a fuck about you or your lifestyle choices. It doesn t give a fuck about the economy, oil spills, or civil unrest in Blargistan. It doesn t give a fuck about katydids. Eventually our whole planet will be erased from space, and the galaxy won t bat an eye. The inevitable extinction of our species (imminent, according to research here at Spinster Laboratories) via the exhaustion of available resources is as natural as a fresh-picked peach. As Andre 3000 and other dude philosophers have observed, nothing is forever.

Yes, yes, when people use the word natural what they really mean is free of chemical additives and maybe some of the assorted hippie concepts that go with that narrative. Barter economies, home furnishings made from bamboo, vegan cookbooks, living in the country. While I would argue that it is just as natural for people to put chemical additives in things as it is to not put chemical additives in things, I admit that it is appealing to fantasize that the source of human misery is an unnatural isolation from Nature, and that doing yoga on an organic rubber mat and drinking organic spinach smoothies will put me back in sync with the cosmos.

But alas, I m already in sync with the cosmos, and so are you. In other words, this is it. This is what we ve become, and this is what we get. Which is not to say that a person can t fantasize about a verdant paradise full of songbirds and polar bears and Bengal tigers, untouched by human influence. Only, that world isn t a world we could actually live in. The minute you add contented children, lazy from a carefree day at the swimming hole, eating ripe plums on the porch at sunset to that scenario, natural history changes, and it s right back to our scorched-earth dystopia. Our giant brains use up resources, it s as simple as that.

As long as we re still here, though, we might as well try to make the best of it.

http://blog.iblamethepatriarchy.com/2010/06/19/heads-up-art-week-starts-on-monday/

Though the comfort is somewhat ephemeral, because deep inside, I know she's wrong, I know another way is feasible, but I have given up already, I have very well come to terms with extiction, I get an enormous pleasure from the thought of the very well deserved human mass suicide-by-cop^Wnature, and in fact, I trust and hope it will save me personally from facing the hard times ahead, as I am a damn coward, paralized by fear, hopeless about the flock ever getting it (is it so hard, really?), while doing as much as I can but completely overwhelmed by the need of changing a world everyone seems to like just as it is. I can certainly empathize with much more disturbed ways of airing similar anguish, among many other examples, such as The Unabomber, or The Weather Underground. I don't even want to go there as I know violence all too well and it is not fun to be on the receiving end. But I see the screaming desperation behind the motives of acts I am not at the liberty to discuss here. I wonder if consumption of resources in the form of books will numb the disorder or make it worse and what sort of medical arrangement the stablishment has for it. As you click away the previous links will mean nothing to you because a)it is Wikipedia after all, and b)you probably don't even understand what the problem of eating animal corpses is, humans are omnivores after all, and what is all this nonsense about dropping language altogether.

WTF, there was no comfort after all, and I am still a nutjob.

Now that Rcpp got updated to 0.8.6, we have an updated RcppArmadillo release containing mostly updates to Conrad's Armadillo version 0.9.70 as well as some more templated sugar magic. RcppArmadillo makes it easy to write highly efficient and highly readable C++ code for linear algebra (based on Armadillo) in R extensions (using Rcpp for the interface). The short NEWS file extract follows:

0.2.6   2010-09-12
    o   Upgraded to Armadillo 0.9.70 "Subtropical Winter Safari"
    
    o   arma::Mat, arma::Row and arma::Col get constructor that take vector
        or matrix sugar expressions. See the unit test "test.armadillo.sugar.ctor" 
        and "test.armadillo.sugar.matrix.ctor" for examples.

More information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

A lot is being said on how Ruby is packaged in Debian and Ubuntu. In this post, I m trying to go through the most common myths and give my position as a Debian Ruby maintainer. Note that those are my views, not necessarily those of the other Debian Ruby maintainers. Myth: Ruby is completely outdated in Debian and Ubuntu There s a culture in the Ruby community of always using the latest bleeding-edge version. There s also a culture (though this one has recessed a bit, I think) that it s perfectly fine to change APIs in incompatible ways if it makes the software slightly better. In Debian (and Ubuntu) stable releases, we try to provide a rock-solid version of Ruby that won t be allowed to change API during the support lifetime of a release. Bugfixes for important bugs can be applied, but that s basically all. So it s not surprising that Ruby in released versions of Debian and Ubuntu is behind the latest upstream version. In development branches, we are following the latest upstream versions quite closely. Debian 6.0 squeeze , to be released by the end of the year, includes Ruby 1.8.7-p302, and will include 1.9.2-p0 (the currently of Ruby 1.9.2 in squeeze is currenty a SVN snapshot from beginning of august). There are people interested in installing a packaged Ruby that just works, and other people interested in installing their own version of Ruby (through RVM and such). People in the Ruby community tend to forget about the first category, and consider that, because they install Ruby from source or with RVM, that should be the solution for everybody. We need the Ruby community to respect that different people have different ways of using Ruby, and to stop recommending that everyone using Debian and Ubuntu should install Ruby from source. We also need to leverage the backports services for both Debian and Ubuntu to provide a way to install the latest upstream version on stable releases. The only reason why we are not doing it yet is lack of time. Myth: Ruby is split into a myriad of packages in Debian In the past (pre-2005), a decision was made to split some libraries from the standard library into separate packages. This was motivated by:

• the size of some libraries, like REXML
• the fact that some libraries, like OpenSSL, readline, gdbm and tk require other library packages to be installed
• licensing concerns about shipping some code (a .so) linked with OpenSSL, and some other code (another .so) linked with readline, in the same package.

Most of the split was undone a very long time ago, and the readline and openssl splits were undone this year. The current package in Debian squeeze has:

• ruby1.8: binaries (ruby itself, irb, rdoc)
• libruby1.8: the standard library, except ruby-tk
• libtcltk-ruby1.8: ruby-tk (merging it with libruby1.8 would add several megabytes)
• ri1.8: contains both the ri binary and documentation
• ruby1.8-dev: development headers (to build applications or gems linked against the Ruby C API), and statically built version of the library
• ruby1.8-elisp: Ruby emacs mode
• ruby1.8-examples: Examples
• libruby1.8-dbg: debug symbols

It is always possible to merge more packages, but merging any of the other packages would significantly increase the size of what you get by default, and might discourage people to use it in some contexts (disk space is not always cheap. Think embedded systems and smartphones). Myth: Rubygems is crippled in Debian There are two differences in the rubygems Debian packages. First, gems are installed in /var/lib/gems, not /usr/local/lib/gems. That is because rubygems are seen as state for the Rubygems package, not independant software installed separately. A change for this is currently being discussed, and should be done after the squeeze release. (I don t think that it s reasonable to push that change now). A consequence of installing to /var/lib/gems is that gems binary are outside of $PATH. After the change, gems binary would also be installed in /usr/local/bin. Second, gem update --sytem is disabled in the Debian package, because that would replace code from the Rubygems Debian package with code downloaded from the Internet, and can break your system in subtle ways. We have changed this (in SVN, not in any package in the archive yet) to allow gem update --system if a specific environment variable is set, which gives us the opportunity to display a warning message. Myth: Ruby is slow on Debian Ruby is built with the pthread threading code, because not using it breaks many native Ruby libraries (Ruby-gnome2, for example). And other Linux distributions are doing the same. There are some very old benchmark results floating around in some blogs that show that the pthread code has a performance impact of about 20%. There are been patches for that for a long time (applied in Ruby Enterprise Edition), but they were never properly sent back. Actually, all the discussions about this issue were rumors and blog posts, and there wasn t even a bug report about it! After the bug was reported properly, it was fixed in a timely manner. The fix is not yet included in the Ruby 1.8.7, but is being applied as a patch to the Debian package since 1.8.7.299-2. (So the Debian package is actually newer and faster than upstream Ruby in that regard). What makes Ruby so difficult to maintain in Debian? The main problems we have with Ruby are portability issues. The only officially supported platform for Ruby is IA32 (i386 in Debian), and everything else is best effort. It means that it is not even supported officially on x86_64/amd64! Debian supports a dozen of architectures, and the Ruby package has to work on each of them. Ruby 1.8 is mostly OK nowadays thanks to the help of Debian porters, but we still have serious problems with 1.9.2, since the test suite fails on sparc, hppa, ia64 and Debian GNU/kFreeBSD. Why doesn t Ruby use the alternatives system on Debian? The alternatives system provides a way to switch between different software providing the same functionality. Currently, if you want to use Ruby 1.9.2, you need to use the ruby1.9.1 executable (1.9.1 = ruby compatibility version here). The alternatives system would allow to switch ruby to ruby1.9.1 on a system-wide basis. However, the situation of applications and libraries working only with Ruby 1.8 should be clarified: those should use /usr/bin/ruby1.8 instead of /usr/bin/ruby. Another problem is that the way to specify packages dependencies still need to be determined. There s nothing impossible to do about that, it just needs someone to do the work. Where do we go from here? The Ruby developers (or expert users) community is generally very hostile towards Debian. Many harsh words and insults are used in discussions mentioning Debian in the ruby-core@ mailing list, and there are frequent recommendations to avoid the Debian packages and install from sources, which is quite demotivating (Actually, what prompted that blog post is someone calling Debian-specific changes unforgivable). This atmosphere makes it hard to recruit people, and the Debian Ruby teams are completely understaffed, which is clearly the major blocker to improving the situation further. We are 3 co-maintainers for the interpreter packages, and I m the only Debian Developer that is really active on a regular basis in the pkg-ruby-extras team (that does libraries and applications packaging). We desperately need help, but at the same time I have no time to improve our documentation and make it easier to join the team. While I haven t reached a final decision yet, I am also likely to stop doing Ruby work in Debian after squeeze is released, as I am getting increasingly tired and demotivated.

I'm just back from ExoClimes 2010: Exploring the Diversity of Planetary Atmospheres. An excellent conference: the PDFs of the talks and posters are now online, and they are putting the videos of the talks up soon. But in particular the organizers deserves thanks for bringing exoplanetary scientists and observers together with climate modelers doing Earth (and Mars, Titan, Venus, ...) models. The last talk on Friday was by Peter Cox on Climate change and exoplanet sciences that was far better than expected for the "graveyard shift". One theme of the conference was the need for a 'heirarchy' of models, from simple energy-balance models to full circulation (GCM) models: using progressively more complex models to understand more bits of whats going on. Exoplanet workers mostly use simpler models, progressing now to GCMs, while Earth modellers are moving beyond GCMs to "Earth system" models including biology, etc. Peter pointed out the two styles of work: the exoplanet modelers are short of data, and risk being too speculative. We know little of what the planets are like, and concentrate on implementing physics in the models to see what they might be like. Earth modelers on the other hand are if anything swamped with data: the tendency here is to make the model fit the data, by adjusting parameters until it does so. The danger of this approach is that the model will then not work away from current present-Earth conditions. Tim Lenton pointed out some work that was done with the Met Office model, where they took the radiative transfer part of the model and tested it for other planets, and paleo-Earth conditions. The model blew up : it wasn't capable of x2 or x4 current CO2 levels. (This has since been corrected). Over dinner there were interesting discussions on the different styles within the communities. While the underlying GCMs used come from the Earth sciences, its quite common within the exoplanetary community for a researcher to work on all parts of the model: dynamics one day, radiative transfer the next. In Earth climate work people have become more specialized and someone is a 'radiative transfer' person, and won't touch other parts of the code (even if they can follow them in the huge codes we have today!). On the other hand, there is a greater tradition of model inter-comparison in Earth sciences, where we compare the model outputs to each other for some known test cases ( Held & Suarez, the CMIP5 project, etc.) Apart from some initial work by Emily Rauscher, little has been done on this in exoplanetary models; it was agreed more of this would be a good idea. Radiative transfer (the interaction of 'sunlight' with the atmosphere, where it gets absorbed, scattered and re-radiated) in particular seems to be an area that could benefit from this. In this middle ground Francois Forget showed the work on the LMDZ model and applying GCMs to terrestrial planets. They've successfully applied this model to Mars, Titan, and partially to Venus (a much tougher problem, due to its heavy clouds giving a long radiative timescale). There are problems with correctly explaining super-rotation though. This is where the atmosphere rotates faster than the planet: on Venus for example the planet rotates every 243 days, while the clouds rotate around the planet every 4 days. Sebastian Lebonnois described the possible mechanisms for Venus and Titan; Johnathan Mitchell so did some interesting work on this recently. Different regimes are involved for different rotation rates of the planet. Ralph Lorenz pointed out the lack of "real paleo-Earth" climate work at the moment. While geology has inspired a lot of work on the atmospheric composition, what with the different gas mixtures (meaning earth-model radiative transfer codes don't work) and the faster dynamics meaning super-rotation could apply (Earth's day was about 8 hours long in the Archean era), we don't have a model of the climate yet. It looks like we should treat Earth as an exoplanet. Tags astronomy, climate, exoplanets, conferences

I was quite unhappy with nanoblogger which I was using for quite some time on this blog. Not that it is bad, but it really scales bad. Having more than 600 of blog posts is something which it really has troubles to handle. So I decided to go somewhere else. After looking and testing some ready solutions, I rather decided to implement my own engine using Django. The implementation took exactly one evening including script to migrate old posts, so it was really an easy task. As a bonus I get quite easily reusable components once I decide to change my main website to Django as well. Also while doing this change, I changed the look quite a lot, hopefully this one is easier to navigate and it did not lose any of old functionality.

After a somewhat longer than usual break, we now have a new release of Rcpp on CRAN and in Debian. This release adds quite few things. The main one may be the addition of density, distribution, quantile and random number functions for a rather large number of statistical distribution. Usage is pretty much as it would be in R, yet it is vectorised at the C++ level. A fair number of unit tests were added too, but some work is left to do there too.
Support for complex number was enhanced both in the expressive 'sugar' context and via a few binary operators that had been missing. We also started a new vignette to provide a 'quick reference'; unfortunately this is not quite complete yet. The NEWS entry follows below:

0.8.6   2010-09-09
    o	new macro RCPP_VERSION and Rcpp_Version to allow conditional compiling
        based on the version of Rcpp
    
           #if defined(RCPP_VERSION) && RCPP_VERSION >= Rcpp_Version(0,8,6)
           ...
           #endif
    o   new sugar functions for statistical distributions (d-p-q-r functions)
	with distributions : unif, norm, gamma, chisq, lnorm, weibull, logis,
	f, pois, binom, t, beta.
    o   new ctor for Vector taking size and function pointer so that for example
    	   NumericVector( 10, norm_rand )
	generates a N(0,1) vector of size 10
    o   added binary operators for complex numbers, as well as sugar support 
    o   more sugar math functions: sqrt, log, log10, exp, sin, cos, ...
    o	started new vignette Rcpp-quickref : quick reference guide of Rcpp API
        (still work in progress)
    o	various patches to comply with solaris/suncc stricter standards
    o	minor enhancements to ConvolutionBenchmark example
    o	simplified src/Makefile to no longer require GNU make; packages using
        Rcpp still do for the compile-time test of library locations
    

As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

In the last few years the Israeli government founded some work on openoffice.org by Tk Open Systems Ltd. During the years, the work has resulted with an Israeli version , released after the official release and includes a lot of patches to improve support for Bidi and Hebrew specifically. Although most patches were submitted to the hamburg , a lot of them weren t used and are still waiting for inclusion. The funding the that projects stopped a year ago (9/2009), and the last release of the Israeli version is 3.1.1. Since then we had a quiet period in which 3.2 and 3.2.1 were released. While still enjoying the fruits of the projects in these version, it seems that it s time for the community to take over, as we lost hopes for yet another funds from the government. In may Netanel and I started talking about taking over the Hebrew translation, and in June we got Hebrew added to the openoffice.org pootle server. We started with about 80%, and got to 86% in time to get to the list of languages getting a language pack built for. This was a first achievement as Hebrew didn t have a language pack since early on the 2.x versions. I believe that working as part of the openoffice.org l10n group and with openoffice.org release schedule will only benefit for the Hebrew translation process. We used the language pack to test the translation and improve it. We also were able to get to 89% with the help of more volunteers.I believe it s now much more consistent than before. In the process with manually checked Writer thoroughly, while Calc and Impress got less attention due to the time frame. But we fixed some major translation issues in all of the three components. While testing the translation we also started to report Bidi issues we saw in 3.3 beta, hoping they will be fixed for the final release. For example, the main openoffice.org screen on windows has is getting repainted wrongly when moving the mouse over the options (text, spreadsheet..). This looks very badly as this is the main screen! See issue #97556 for details In addition to report new issues, we also started to keep track of the already opened issues regarding Bidi and Hebrew. I m following the issue system for a few years now, but seeing the list of issues organized in the wiki gave me a better orientation in the problems we have. E.g. finding some related or similar issues and mention each other in the comments, when someone goes to first one, he could also fixed the other as the problem is similar and probably comes from the same code. Organizing also the already closed bugs in 3.x, gave me a some kind of a changelog for the Bidi and Hebrew related issues. In that manner, version 3.1 was a very good for Bidi, thanks for a lot of efforts and work with the Bidi meta issue #79434. The wiki page is in Hebrew, but looks quite OK with Google translate (see here). Hoping to have 3.4 a major release (Bidi wise), I ve opened issue #114236 as another meta issue for Bidi related issues. I ve kept Hebrew specific issues out, as to get it more general and also relevant for Bidi users in other languages. Any help of those issues will be more than welcome (starting with votes and ending with patches).
Filed under: i18n & l10n, Israeli Community, Openoffice.org

After a somewhat longer than usual break, we now have a new release of Rcpp on CRAN and in Debian. This release adds quite few things. The main one may be the addition of density, distribution, quantile and random number function for a rather large number of statistical distribution. Usage is pretty much as it would be in R, yet it is vectorised at the C++ level. A fair number of unit tests were added too, but some work is left to do there too.
Support for complex number was enhanced both in the expressive 'sugar' context and via a few binary operators that had been missing. We also started a new vignette to provide a 'quick reference'; unfortunately this is not quite complete yet. The NEWS entry follows below:

0.8.6   2010-09-09
    o	new macro RCPP_VERSION and Rcpp_Version to allow conditional compiling
        based on the version of Rcpp
    
           #if defined(RCPP_VERSION) && RCPP_VERSION >= Rcpp_Version(0,8,6)
           ...
           #endif
    o   new sugar functions for statistical distributions (d-p-q-r functions)
	with distributions : unif, norm, gamma, chisq, lnorm, weibull, logis,
	f, pois, binom, t, beta.
    o   new ctor for Vector taking size and function pointer so that for example
    	   NumericVector( 10, norm_rand )
	generates a N(0,1) vector of size 10
    o   added binary operators for complex numbers, as well as sugar support 
    o   more sugar math functions: sqrt, log, log10, exp, sin, cos, ...
    o	started new vignette Rcpp-quickref : quick reference guide of Rcpp API
        (still work in progress)
    o	various patches to comply with solaris/suncc stricter standards
    o	minor enhancements to ConvolutionBenchmark example
    o	simplified src/Makefile to no longer require GNU make; packages using
        Rcpp still do for the compile-time test of library locations
    

As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

(Disclaimer: Possibly one of my longest post ever, you may want to scroll to the bottom and only look at the pictures.) So, what s new in SD since last week? Fetching everything From http://bugs.freedesktop.org/, it was possible to download all the bugs I reported there (that means 2 ), but trying to download those reported by Julien led to a crash without any trace. Dichotomy FTW, I came up with a bunch of bugs which would lead to the same result, which confirmed that the amount of bugs processed at once wasn t the issue. Playing around with strace, it appeared that exchanges with the remote server were apparently fine, so an issue on the client-side SOAP machinery got suspected. So let s enable tracing:

use SOAP::Lite +trace => 'all';

Tada! The issue was indeed local: malformed XML got received, leading to a die call issued from the XML-RPC layer. Thankfully, one can set up a fault handler, which I used to display the range of bug IDs which triggered that bug, so that people can look into it and determine what to add to the blacklist until the underlying issue s been investigated (presumably, bugzilla s at fault). Speeding things up With that blacklisting of buggy bugs, one can then try to move to other queries, dealing with more bugs. Some examples follow:

reporter=kibi@d.o
product=xorg&component=Driver/VMWare
product=xorg&component=Driver/nouveau
product=xorg&component=Server/general
product=xorg&component=Driver/intel

Moving from 2 bugs to 10-100 bugs (Julien s or VMWare s) was OK. But then moving to several hundreds of bugs (Driver/nouveau = 500+ bugs) led to noticeable performance issues. Not to mention what happened when one reaches several thousands of bugs (Driver/intel = 2500+ bugs). Indeed, even with network exchanges cached into a local file, processing data was taking up to several dozens of minutes. I knew about Perl s -d:DProf, which helps figuring out where time is spent, but was pointed to -d:NYTProf (and its accompanying tool, nytprofhtml). Some hotspots got noticed:

• There s a huge pile of stuff relying on UUIDs heavily, and a cache is going to be introduced to avoid later calls once a value s been computed once. That s going to benefit all replica types, not just bugzilla.
• I didn t care much about date/time at the beginning, but that turned out to be a very bad idea: since the format returned by bugzilla wasn t matching a well-known format, time was spent in the DateTime::Format::Natural fallback, leading to a big performance penalty. Fixed with a trivial regular expression.

Things got better, but not good enough. There are several Prophet (the engine under the hood) backends, so one can play with:

PROPHET_REPLICA_TYPE=sqlite   # the default for SD
PROPHET_REPLICA_TYPE=prophet

Switching to prophet was a big win, but still not good enough. Indeed, many tiny files are written, and most of the time is spent in I/O. Although I m nothing like a performance guru, I guessed that running on an average laptop, with ext3 and its default commit interval of 5 seconds might not be helping, so I gave a quick try to -o remount,commit=60, and that seemed to help. Even though there are probably other tricks to find in that area (which hopefully won t require root privileges ), there s already a patch which landed in prophet s master branch, replacing File::Spec->catfile with an optimized version: that function alone was eating 10% of runtime In the sqlite case, disabling the auto-commit feature helped reducing the I/O load, but a proper patch is still lacking for now (running into locked database issues, or into missing tables after having created them isn t fun, so I postponed debugging that). Since performance issues looked like they could be solved eventually, I switched back to implementing missing features. Handling more than comments Currently 3 types of stuff are currently fetched from the bugzilla server:

• Bug status: plenty of properties.
• Bug comments: comments that are linked to bugs.
• Bug history: changes that impacted bugs.

(Yes, that means that attachments are totally ignored for now.) Until now, only bug comments were considered. The first comment was used to determine a pseudo-title (using its first line), the reporter, and the creation date. This approach was chosen to try and get a basic sync working quickly, so as to get:

• A list of bugs matching the query.
• All comments for each of these bugs.

Now, the algorithm is the following: from the bug status, determine a set of properties of interest; then walk the history backwards, and update the properties incrementally until the (presumed) initial state is reached. Then create the ticket using this initial state . Adding the incremental property changes to that initial ticket makes it possible to represent the bug s life as a list of Prophet::ChangeSet objects. That s where the fun begins, since properties in the bug status may not match properties in the bug history, so one needs to establish property correspondence. Also, some properties can be multivalued for added fun. I believe that s where most of the time is going to be spent while developing a new replica type in SD: once one knows how to get a hand on needed info on the remote server, the main question is what to do with it. For now, I decided to ignore many fields to make it possible to do a big sync like Server/general, property support will be improved later on. Screenshots Instead of pasting lengthy terminal excerpt, let s use some screenshots instead (sorry, I m not sure how to present such things in an accessible way, suggestions welcome). Cloning Julien s bugs, listing all open tickets, listing all tickets, searching using a regular expression: Displaying bug 42 (that s the local ID): Now, let s start the embedded web server through sd server --port 1234 and point the browser there. List of RESOLVED bugs: Status and comments for bug 42: History for bug 42: Compared to the original bugzilla page: Next time Some items which need work:

• Tweak properties to address the issues raised above.
• Start fetching attachments as well.
• Support further syncs. Currently, a big sync is done once, and there s no way to tell sd to sync new changes since last time, if any. This will probably lead to rewriting how fetching is currently done, which is: discover all bugs, then fetch all comments and all history items, for all of them. Properties like last_change_time will probably be of some help here.
• Have a look at what happens with other bugzilla instances, like Gnome s.

So, conventional wisdom is that once you experience something first hand, it sits close to you, so that you can learn from it, think about it, and possibly invoke it. This rings true for me, except for one particular type of experience finishing a large and complicated project like writing a dissertation, taking PhD qualifying exams, like writing a very complicated review article. Once it is done and over with and a few months have walked away, I no longer can fathom, at all, how I even did it. It is as if I went through the experience and then amnesia set in, erasing and wiping out the fibers of events, emotions, and thoughts that went into making and finishing the project. I find this unnerving, for various reasons. First, this di-juncture is not one I experience with other experiences, even unpleasant ones, like pain (broken collarbone, burnt hand, terrible earache), for I am able, nearly always, to shore up some shadow of those experience. More practically, when I am faced again with needing to conquer what feels unconquerable (which, unsurprisingly, is my current predicament), I have no experiential reserve to guide me. Instead, it is like I am standing at a new shore, for the first time. I tell myself, I have done and I can do it but there is no concomitant emotional register or memory that assures me that this is in fact true. Maybe a few more projects like these and there will be some imprint to steer me in the future but somehow I suspect my experiential amnesia will remain the same.

Sometimes remote accounts are only reachable with a series of ssh/su/sudo commands. Using ProxyCommands in .ssh/config works for simple cases, but not with several hops, or if passwords have to be entered. The belier tool takes as input a series of user@hostname strings and will produce an expect script that does the actual login work.

$ cat input
user@host1 root pw1
user2@host2 pw2
$ bel -e input
$ cat host2.sh
#!/usr/bin/expect -f
set timeout 10
spawn ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user@host1
expect -re  "(% # \\$) $"
send -- "su - root\r"
expect ":"
send -- "pw1\r"
expect -re  "(% # \\$) $"
send -- "ssh -o NoHostAuthenticationForLocalhost=yes -o StrictHostKeyChecking=no  user2@host2\r"
expect -re  @[^\n]*: 
send -- "pw2\r"
expect -re  "(% # \\$) $"
interact +++ return

The generated host2.sh script uses ugly ssh options, but is easily edited. Now I need the same thing for scp... Update: Carl Chenet, belier's author, kindly pointed me to the documentation which has examples how belier can set up ssh tunnels to copy files.