Lately, while using the Twitter web interface with Javascript turned on, I've been redirected to a Privoxy error page whenever I click on an external link:

Request for blocked URL

Your request for http://twitter.com/link_click_count?url=http%3A%2F%2Fexample.com&linkType=web&tweetId=1234567890&userId=12345678&authenticity_token=f5920ebf56bfa111c54bb9f86814cafa76de00fa was blocked.

Block reason: Path matches generic block pattern.

Looking a little closer at the blocked URL, I noticed that for every external link you click, Twitter logs the destination URL, the originating tweet and your user ID.

So what can you do to opt out of this tracking?
Disable JavascriptThis is by far the easiest solution. You can simply install an extension like Noscript and use Twitter with Javascript turned off. That's what I do most of the time since the website is mostly usable without it.

A very fortunate side-effect of using Twitter without Javascript is that you can avoid a number of the worms which propagate through that service from time to time.
Use Privoxy to disable the link counterFor those times when I have to enable Javascript on the site, I use Privoxy, a great privacy-enhancing web proxy, to alter the Javascript code executed when clicking on external links. This effectively disables the extra tracking and takes me directly to the intended URL.

First of all, make sure you have this line in your /etc/privoxy/config:

filterfile user.filter

then, add this filter to /etc/privoxy/user.filter:

FILTER: twitter-link-count Disable the Twitter link tracker
s@twttr\.countClick=function@twttr.dsbldCountClick=function@Uig

and finally turn this filter on for the affected URLs (/etc/privoxy/user.action):

  +force-text-mode +filter twitter-link-count   
.twimg.com/.*/javascripts/twitter.js

Switch to IdenticaIf you want to avoid similar privacy problems in the future, you can also switch to an Open platform which respects your basic freedoms.

The main contestant in this space is Identica. It's a great alternative to Twitter and has a lot of interesting features (like groups and XMPP integration) which are not yet available on Twitter.

The StatusNet developers have kindly released their source code under the AGPL and their goal is to build the most open micro-blogging system out there.

So go over there now and claim your username!

A bit more than two years ago, two young Spaniards on vacation in Latvia maybe went a bit too far during one of their night parties and decided to remove some Latvian flags that hanged from a post in the streets of Riga. They spent 1 month in prison, with charges for outraging the Latvian flag. The Spanish media talked about the disproportionate charges, the ridiculous and medieval laws in Latvia and so on. Today, we learn that Jaume d'Urgell will go to prison due to the outraging crime of substituting the current Spanish flag with the Republican flag of 1931 on the facade of a public building. So much for medieval laws and institutions like the Spanish monarchy.

While git is a completely distributed revision control system, sometimes the lack of a central canonical repository can be annoying. For example, you might want to make your repository published publically, so other people can fork your code, or you might want all your developers to push into (or have code pulled into) a central "golden" tree, that you then use for automated building and continuous integration. This entry should explain how to get this all working on Ubuntu 9.04 (Jaunty). Gitosis is a very useful git repository manager, which adds support like ACLs in pre-commits and gitweb and git-daemon management. While it's possible to set all these things up by hand, gitosis does everything for you. It is nicely configured via git; to make configuration changes, you push the config file changes into gitosis repository on the server. Gitosis is available in Jaunty, but unfortunately there is a bug in the version in Jaunty, which means it doesn't work out of the box. Fortunately there is a fixed version in jaunty-proposed that fixes the main problem. This does mean that you need to add the following to your sources.list:

deb http://gb.archive.ubuntu.com/ubuntu/ jaunty-proposed universe

Run apt-get update && apt-get install gitosis. You should install 0.2+20080825-2ubuntu0.1 or later. There is another small bug in the current version too, as a result of git removing the git-$command scripts out of /usr/bin. Edit /usr/share/python-support/gitosis/gitosis/templates/admin/hooks/post-update and replace

git-update-server-info

with

git update-server-info

With these changes in place, we can now set up our gitosis repository. On the server you are going to use to host your central repositories, run:

sudo -H -u gitosis gitosis-init < id_rsa.pub

The id_rsa.pub file is a public ssh key. As I mentioned, gitosis is managed over git, so you need an initial user to clone and then push changes back into the gitosis repo, so make sure this key belongs to a keypair you have available to the remote user you're going to configure gitosis. Now, on your local computer, you can clone the gitosis-admin repo using:

git clone gitosis@gitserver.example.com:gitosis-admin.git

If you look inside the gitosis-admin directory, you should find a file called gitosis.conf and a directory called keydir. The directory is where you can add ssh public keys for your users. The file is the configuration file for gitosis.

[gitosis]
loglevel = INFO
[group gitosis-admin]
writable = gitosis-admin
members = david@david
[group developers]
members = david@david
writable = publicproject privateproject
[group contributors]
members = george@wilber
writable = publicproject
[repo publicproject]
daemon = yes
gitweb = yes
[repo privateproject]
daemon = no
gitweb = no

This sets up two repositories, called publicproject and privateproject. It enables the public project to be available via the git protocol and in gitweb if you have that installed. We also create two groups, developers and contributors. David has access to both projects, but George only has access to change the publicproject. David can also modify the gitosis configuration. The users are the names of ssh keys (the last part of the line in id_dsa.pub or id_rsa.pub). Once you've changed this file, you can run git add gitosis.conf to add it to the commit, git commit -m "update gitosis configuration to commit it to your local repository, and finally git push to push your commits back up into the central repository. Gitosis should now update the configuration on the server to match the config file. One last thing to do is to enable git-daemon, so people can anonymously clone your projects. Create /etc/event.d/git-daemon with the following contents:

start on startup
stop on shutdown
exec /usr/bin/git daemon \
   --user=gitosis --group=gitosis \
   --user-path=public-git \
   --verbose \
   --syslog \
   --reuseaddr \
   --base-path=/srv/gitosis/repositories/ 
respawn

You can now start this using start git-daemon So now, you need to start using your repository. You can either start with an existing project or an empty directory. Start by running git init and then git add $file to add each of the files you want in your project, and finally git commit to commit them to your local repository. The final task is to add a remote repository and push your code into it.

git remote add origin gitosis@gitserver.example.com:privateproject.git
git push origin master:refs/heads/master

In future, you should be able to do git push to push your changes back into the central repository. You can also clone a project using git or ssh, providing you have access, using the following commands. The first is for read-write access over ssh and the second uses the git protocol for read-only access. The git protocol uses TCP port 9418, so make sure that's available externally, if you want the world to be able to clone your repos.

git clone gitosis@gitserver.example.com:publicproject.git
git clone git://gitserver.example.com/publicproject.git

Setting up GitWeb is left as an exercise for the reader (and myself because I am yet to attempt to set that up).

Whenever I attend Debconf, the first day is always a bit disorienting due to the confluence of jet lag with the sea of male programmers swarming at the venue. I am not sure I should be admitting this but one of the first things that pops to mind is the very cheesy one hit wonder whose video is filled with many bikini clad men, Its raining men. So for a few hours, maybe a day, I sometimes feel out of place but this this first wave of discomfort usually gives way to comfort. This is not only because I connect with friends but because Debconf as well as many other hacker events and this sits in marked contrast to professional academic conferences are remarkably accepting of diversity and difference, at least that has been my personal experience. Most notably, people could care less about how you look, disabilities are consistently accommodated, and some things, like gender changing, which would raise serious eyebrows for instance in most academic conferences, is accepted with next to no gossip. I raise this because this is how I personally have experienced the world of Free Software: overwhelming male but overwhelming accepting of difference as well. So even while I think it is key to confront the problems of discrimination in this world, it is not a simple story, for again, it is an environment that is also accommodates difference and quirkiness to some notable degree. The question of gender in F/OSS is complicated one and here I am not going to ruminate on sources of gender bias except to say that I am sure there are some in operation (as there are in most domains) some of which are internal to Free Software and other s external to Free Software. But what I have been struck by, especially over the course of the summer, is the explosion of sites, blogs, and debates that have confronted gender in Free Software. It has been nothing short of astounding and a really positive turn. The most notable example is this AWEEEEEEEEEEESOME pair: a wiki and a blog that confront gender head on. These are notable because the wiki, for example, catalogs all sorts of controversial events, comments on them, leaving a very visible trace of debate, one that is necessary to change the gender make-up and dynamic within Free Software project. Along with these, the FSF is finally hosting a min-summit, which is great (less great is that participation seems invite only but perhaps they have some good reasons for doing so). And today I just learned of this diversity page coming out the Python project. These are perfect examples of the recursive public in action raised in Chris Kelty s work. And I have long been impressed with the dialogue that has followed from some controversial events in the world of Free Software, including those related to gender and this summer seems to be a watershed of sorts and I look forward to their developments over time. Indeed and this may be controversial as well, but I think my academic field of media, law, and culture has something to learn from these gender politics for there are some very real, though probably unintentional forms of discrimination that are not under that much active discussion. The most glaring problem is the underwhelming presence of female scholars during conferences (and as we know, conferences are exceedingly important for one s professional development and career). What I find most striking about this trend is the number of female scholars is significant. That is, when it comes to scholars and this seems different from the world of tech, there is a sizable community of women scholars and activists so when there is a 5% female participation , as for example, with this event, one has to wonder why is it raining men at these conferences? It also seems that while the debate exists, it is not as vibrant as with what is going on right now in F/OSS. Take for example, this recent mailing list post which unapologetically highlighted the lack of female presence in the up and coming Free Culture event at the Berkman Center. While the post generated a handful of thoughtful responses, including this supremely classic, biting, and quite clever response from Georgetown professor Julie Cohen, the debate did not linger on (however short, the posts and discussion were quite fruitful). What to do? Frankly, blame and finger pointing are pretty counterproductive, mostly because the discrimination is quite unintentional and I believe change can be brought about via more constructive paths. More important is we need to make the issue visible, identify some possible sources, and then create projects that can remedy the problem. Along with Elizabeth Stark, who has also been keen to note and change the gender problems, I am currently whipping together a wiki with a list of female scholars, leaders, and technologists as a resource for folks organizing conferences. We already have a base list (with a remarkably long list of folks) and will hopefully in the course of the next month throwing it up on the wiki. I don t think it will ever be as cool as the Geek Feminism Wiki but it will hopefully do some good!

Listening to Martin Krafft s talk at Debconf (related to his PhD) shed some new light on the idea that I expressed last year I wanted that each maintainer regularly answers a questionnaire so that he has to ask himself whether he does a good enough job with his packages. When thinking of this idea, I only saw the QA side of ensuring good maintenance on all packages, however I believe that the root problem lies further and this project would not be enough: we are interdependent but we are not equipped to deal with this reality. Martin s only merit has been to mention that we are interdependent, but it s worth analyzing a bit. Our organization is centered around individuals acting as package maintainers, and in theory each package maintainer can work on his corner and all goes well. We know that this model doesn t hold any more: transitions to testing require coordination of uploads and timely fixes of RC bugs, keeping up with the work frequently requires several volunteers that have to coordinate, etc. More and more of the work requires a level of availability that a single individual can t offer, yet in our day-to-day work we mainly interact with individuals. Wouldn t it be better if we could immediately know what we can expect from any Debian developer:

• mean time to reply to Debian mail (reading every day or once a week is not the same);
• amount/periods of time spent on Debian (knowing that they spend up to 4 hours mainly on Saturday can be useful);
• current availability for Debian (if they are currently busy with life, we should be able to know it, if they know when it will end, it would also be good to share);
• best way to get in touch for issues (they might have preference between IRC or mail);
• kind of relationship they have with packages where they are listed in Maintainer/Uploaders (an active maintainer that uses the software daily is not the same than a passive maintainer that only packaged the software because it was a build-dependency for some other software that they care about);
• any other information that the maintainer wants to share (about his habits/constraints/values/goals/whatever).

All this information should be shared by all Debian maintainers (some of it is already available but either not publicly or not in any machine-parseable way) and we should actively use it. Here are some examples of use: for each RC bug report, you could look up if at least one maintainer is available and you could ping him explicitly if needed. When you plan an NMU, you could look up if the maintainer is likely to respond in the next day or not, and possibly adjust the number of days spent in the DELAYED queue. When organizing a large-scale transition, you could extract a list of packages whose maintainers are not available and arrange immediate NMUs. Furthermore there are many cases where the project s usual expectation exceed what the maintainer is ready to do. Documenting what part of the job is done (or not) by the maintainer makes it clear for volunteers whether their help is needed and whether they could/would be a better maintainer for a given package. Designing solutions to all these problems is going to be the scope of the DEP2 that I reserved some time ago. It s likely to be some sort of dedicated web interface. I would welcome supplementary drivers for this DEP, so if you re interested, get in touch with me. Partagez cet article / Share This

While git is a completely distributed revision control system, sometimes the lack of a central canonical repository can be annoying. For example, you might want to make your repository published publically, so other people can fork your code, or you might want all your developers to push into (or have code pulled into) a central "golden" tree, that you then use for automated building and continuous integration. This entry should explain how to get this all working on Ubuntu 9.04 (Jaunty). Gitosis is a very useful git repository manager, which adds support like ACLs in pre-commits and gitweb and git-daemon management. While it's possible to set all these things up by hand, gitosis does everything for you. It is nicely configured via git; to make configuration changes, you push the config file changes into gitosis repository on the server. Gitosis is available in Jaunty, but unfortunately there is a bug in the version in Jaunty, which means it doesn't work out of the box. Fortunately there is a fixed version in jaunty-proposed that fixes the main problem. This does mean that you need to add the following to your sources.list:

deb http://gb.archive.ubuntu.com/ubuntu/ jaunty-proposed universe

Run apt-get update && apt-get install gitosis. You should install 0.2+20080825-2ubuntu0.1 or later. There is another small bug in the current version too, as a result of git removing the git-$command scripts out of /usr/bin. Edit /usr/share/python-support/gitosis/gitosis/templates/admin/hooks/post-update and replace

git-update-server-info

with

git update-server-info

With these changes in place, we can now set up our gitosis repository. On the server you are going to use to host your central repositories, run:

sudo -H -u gitosis gitosis-init < id_rsa.pub

The id_rsa.pub file is a public ssh key. As I mentioned, gitosis is managed over git, so you need an initial user to clone and then push changes back into the gitosis repo, so make sure this key belongs to a keypair you have available to the remote user you're going to configure gitosis. Now, on your local computer, you can clone the gitosis-admin repo using:

git clone gitosis@gitserver.example.com:gitosis-admin.git

If you look inside the gitosis-admin directory, you should find a file called gitosis.conf and a directory called keydir. The directory is where you can add ssh public keys for your users. The file is the configuration file for gitosis.

[gitosis]
loglevel = INFO
[group gitosis-admin]
writable = gitosis-admin
members = david@david
[group developers]
members = david@david
writable = publicproject privateproject
[group contributors]
members = george@wilber
writable = publicproject
[repo publicproject]
daemon = yes
gitweb = yes
[repo privateproject]
daemon = no
gitweb = no

This sets up two repositories, called publicproject and privateproject. It enables the public project to be available via the git protocol and in gitweb if you have that installed. We also create two groups, developers and contributors. David has access to both projects, but George only has access to change the publicproject. David can also modify the gitosis configuration. The users are the names of ssh keys (the last part of the line in id_dsa.pub or id_rsa.pub). Once you've changed this file, you can run git add gitosis.conf to add it to the commit, git commit -m "update gitosis configuration to commit it to your local repository, and finally git push to push your commits back up into the central repository. Gitosis should now update the configuration on the server to match the config file. One last thing to do is to enable git-daemon, so people can anonymously clone your projects. Create /etc/event.d/git-daemon with the following contents:

start on startup
stop on shutdown
exec /usr/bin/git daemon \
   --user=gitosis --group=gitosis \
   --user-path=public-git \
   --verbose \
   --syslog \
   --reuseaddr \
   --base-path=/srv/gitosis/repositories/ 
respawn

You can now start this using start git-daemon So now, you need to start using your repository. You can either start with an existing project or an empty directory. Start by running git init and then git add $file to add each of the files you want in your project, and finally git commit to commit them to your local repository. The final task is to add a remote repository and push your code into it.

git remote add origin gitosis@gitserver.example.com:privateproject.git
git push origin master:refs/heads/master

In future, you should be able to do git push to push your changes back into the central repository. You can also clone a project using git or ssh, providing you have access, using the following commands. The first is for read-write access over ssh and the second uses the git protocol for read-only access. The git protocol uses TCP port 9418, so make sure that's available externally, if you want the world to be able to clone your repos.

git clone gitosis@gitserver.example.com:publicproject.git
git clone git://gitserver.example.com/publicproject.git

Setting up GitWeb is left as an exercise for the reader (and myself because I am yet to attempt to set that up).

Oops. Yesterday (Sunday, 31/08/09) I far from any computer-like object for most of the day. When I got back home, of course, I promptly opened my laptop to check my mail who knows what destiny might have for me in a 24 hour period? Maybe I won yet another fortune I have to cash in Nigeria? Maybe there is (GASP!) a new RC bug on one of my packages? But no, my mail server didn't feel like answering to my ssh queries. The connection was established, but shut down before even sending the protocolary SSH-2.0-OpenSSH_5.1p1 string. Fearing an overload (after all, the little bugger is just a Mac Mini running in another room in my house), I tried to check (via Web) its Munin status Apache didn't want to listen either. It answered, but got only access denied. Things started worrying me But (silly me) not enough The machine runs headless1, so I just danced the boring raising elephants song2. Allowed for a couple of minutes for everything to settle, and tried to connect. Horror, now even pings didn't work! So I ran to fetch my old, bulky and trusty monitor. Went back to the machine, plugged it in, switched it off and back on. Everything worked fine this time At least appearingly. I opened up mutt and started happily reading mails, while trying to understand on another console what happened at 07:06 that didn't get logged anywhere and had the machine dead for basically all the day. And then, BRRRT-BRRRT-BRRRT, I started hearing the HDD seeking. I was able to send a couple of mails, but decided to let the machine rest and... Will reduce its disk usage to an absolute minimum. Fortunately, I have already the machine meant to replace it A much nicer, beefier iMac G5, waiting to be vacated from its data, task which has suddenly become prioritary. So, in short: If you need to get in touch with me in the next day or two, don't count on my usual @gwolf.org mail, as it is down. I hope to be able to get the data out of the poor little bugger painlessly after it rests a bit. And I hope not to drown in a sea of mails after I get the replacement back online :-/

1. 1. for those not used to computerspeak: without a connected monitor
2. 2. For those following at home who don't understand how Raising Elephants Is So Utterly Boring, this is a (silly, stupid but useful) mnemotecnic for as-properly-as-possible restarting hung Linux systems: Hold AltGr (right-Alt) + SysRq (in reduced/notebook keyboards, Fn+SysRq), and type R E I S U B (leaving a couple of seconds between key, ideally waiting for all disk activity to settle between each). That means:

   unRaw

   Take control of keyboard back from X,

   tErminate

   Send SIGTERM to all processes, allowing them to terminate gracefully,

   kIll

   send SIGKILL to all processes, forcing them to terminate immediately,

   Sync

   Flush data to disk),

   Unmount

   Remount all filesystems read-only,

   reBoot.

   Well... Reboot the system

Yesterday, we attended the final game of French "Top 14", French rugby league championship, between Perpignan (Perpiny in catalan) and Clermont. Quite good game, though not as good as last year's final (Toulouse-Clermont won by Stade Toulousain). We personnally had no real preferences as our support in rugby generally go to Toulouse, which had been beaten in semi-finals by Clermont. Perpignan won 22-13 against the infortunate Stade Clermontois, who lost its thirdt final in a row. We were fairly sad for Clermont but the catalan game was really much better, particularly in forward lines. As usual, the finals were a great feast for rugby, in the usual spirit of that sport. And the spectacle, in Stade de France, was not only on the field but also around the field, with a stadium turning out yellow (and red on one side, and blue on another). That will conclude that season for rugby....and we're already looking forward to next season where, of course, the Stade Toulousain will revive from its ashes and win both Top 14 and H Cup (there has now been enough of these for the British Isles, now...let's stop this).

Today I went to the La Merced area to finally buy a foldable bike. There were basically four options: Benotto's 20" Utopia, Alubike 16" Foldingbike, Alubike 24" and a custom-made bike at a smaller store, Bici Universo. In Mexico it is also possible to buy a Brompton, although they are not available in larger stores and their pricing is prohibitive. Sergio Mendoza advised me to look for the Dahon bikes at Benotto, but they have discontinued them in favor of their own making. I ended up buying the Bici Universo one. I must say this: I sincerely hope not to regret this option I got this bike because neither of the other ones satisfied me, and this one was at about two thirds the price (MX$1800, plus some extras I requested, MX$2100 That means, US$187 or 113). First of all, after thinking about it for a while, I decided I didn't want a larger, more normal wheel The Alubike 24 (which I'd have to build shopping piece by piece, as they had only the frame for sale Would be a good ocassion to learn more about the whole process!) is basically a great regular bike you can put in a regular car trunk, but is still too large for taking it into public transport or lugging along when travelling, by bus or by plane. Besides, the manubrium1 is not foldable. So it is basically as portable as this venerable Compaq. The Benotto's manubrium does fold, but the bike does not hold itself together when folded, it is not stable. You have to explicitly keep it folded. Not exactly comfortable. I had tested the smaller Alubike at another bike store closer to my home. It feels a bit freer than the Universo one (I'll get to it soon), but still feels a bit kludgy... And at a price about MX$1000 higher, I decided against it. Now, why am I wary with the Universo? Because after all... It is just a bike for kids age 4-7 and 7-12 (?) cut in half, and with a hinge soldered on: It is also a heavy bike (I have not weighed it yet, but comparative guesstimates puts it over the Alubikes), so it might be problematic when travelling... I'll see when in Nicaragua in a couple of weeks ;-) I have yet a couple of adjustments to make to it, but I am very looking forward to travelling with my bike on. I hope it turns out to be comfortable. And in any case, I can of course! donate it to my nephews... who are in the right age group. Some photos:

• The bike, the cats
• The Universo bike, folded next to my regular 26" Benotto
• Nadezhda riding the Universo

1. 1. the word seems strange, but that's Google's suggestion The bar you take with your hands to steer the bike

Voice-controlled waypoints I have it in my TODO list to implement taking waypoints when pressing the headset button of the openmoko, but that is not done yet. In the meantime, I did some experiments with audio mapping, and since I did not manage to enter waypoints while recording them, I was looking for a way to make use of them anyway. Enter findvoice:

$ ./findvoice  --help
Usage: findvoice [options] wavfile
Find the times in the wav file when there is clear voice among the noise
Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         verbose mode
  -p NUM, --percentile=NUM
            percentile to use to discriminate noise from voice
            (default: 90)
  -t, --timestamps      print timestamps instead of human readable information

You give it a wav file, and it will output a list of timestamps corresponding to where it things that you were talking clearly and near the FreeRunner / voice recorder instead of leaving the recorder dangling to pick up background noise. It does not use sound statistics because I have no background whatsoever in audio processing, but it basically finds those parts of the audio file where the variance of the samples is above a given percentile: the higher the percentile, the less timestamps you get; the lower the percentile, the more likely it is that it picks a period of louder noise. For example, you can automatically extract waypoints out of an audio file by using it together with gpxinterpolate:

./findvoice -t today.wav   ./gpxinterpolate today.gpx > today-waypoints.gpx

On the timestamps that it outputs, it will add the modification time of the .wav. This means that if you touch -d "@timestamp" the .wav file with the date you started recording, then you can get decently accurate waypoints provided your system clock is decently synchronized. For example:

START= date +%s 
arecord file.wav
^C
touch -d "@$START" file.wav
./findvoice -t file.wav   ./gpxinterpolate today.gpx > today-waypoints.gpx

In record-audio-on-freerunner I even set the .wav mtime using the GPS time at the start of the recording.

Ever since I read about the ATA TRIM command, I've been thinking that something like that would make a lot of sense for things like NBD. Currently, it's possible to serve a sparse file using NBD. I often do this, for example, testing things like the fix for Bug #513568 (nbd-server choking on a >3 TB RAID device) which would otherwise require me to spend significant amounts of money so that I get enough disks to test. Obviously that's not going to happen. If we're using sparse files, that means we claim to export more than we actually do. This is obviously useful for testing, but there are other cases where this kind of behaviour could be useful; for instance, when exporting swapdevices to several diskless clients, it would be a rare case indeed where all those clients need all of their available swapspace; it might be safe to slightly overcommit the disk space on your hard disk, on the assumption that none of your clients is going to use all that space at the same time. Similarly, when a file is removed on a filesystem on an NBD device, the kernel might use the generic TRIM layer described by Ted in the article above to relay that removal to the nbd-server process, which could then make sure the diskspace is deallocated. However, unfortunately, there currently is no way for an nbd-server to tell the kernel to create a new hole in a sparse file. Creating sparse files is done at file creation time; if you later want to add more holes than you did when creating the file, you need to recreate the whole file from scratch in order to do so. Additionally, when we're talking about a large amount of data that needs to be removed, it's usually much faster to say 'remove this block of data' rather than 'overwrite several megabytes with zeroes here' not just in API, also in kernel-side implementation, provided the filesystem supports extents. I think it might be nice if there were a public (userspace) API to tell the kernel that particular bits of a file or a block device are no longer required, and that they may thus be removed. But then, I'm not very experienced with kernel code, and it might be that I'm just plainly missing something here. Anyone?

I've had a few enquiries from my blog post about trying to control dumb USB-powered lights. I thought I'd just write something up to save myself replying to any more emails. Yes, it's doable. Finding a USB hub that will do it is another story. From my own research, I found someone else who was doing something with USB-powered devices (I can't remember what now), and he had been using a what is now a Linksys ProConnect USB 4-Port Hub USB 2.0 I'd first tried a couple of random cheap hubs from Fry's with no success (fortunately I was able to return them) before I determined that the Linksys one would definitely work. The downside of the Linksys hub is it requires external power. It was also one of the more expensive USB hubs on the market. One of my co-workers, who is an Electrical Engineer by education, said that the USB spec requires the functionality that I wanted, but most chip manufacturers had cut a corner in the interests of cost saving. The Linksys hub uses an NEC chipset. Every other hub that I could get my hands on had a Genesys Logic chipset, and did not work. You can tell if you've got a winner by the output of lsusb -v. If the hub characteristics include "Per-port power switching", you're in business. To do the actual port powering on and off, I'm using a setuid-root hub-ctrl, wrapped with a small shell script, which has the USB ID of the hub and the port number the lights are plugged into hard-coded in it. In my searching, I found also that it may be possible to do with Python, but I did not invest the time trying to find out.

Having (temporarily) flung myself back into a bit of professional Rails app development, I'm doing a JS-heavy site (just to maximise the WTF factor). Since I'm Old Skool, I'm going with Prototype, as it's the javascript library I'm least unfamiliar with. The problem, though, is that I do about a fifth of my work on the train, where I don't have access to the Internet, and the Prototype people haven't seen fit to provide a downloadable version of the docs in HTML. There's a PDF[1], (which lacks any sort of hyperlinking (how very 1990s) and whose page numbering in the TOC doesn't match up to the PDF page numbers), and a CHM which has hyperlinks that don't work half the time. Apart from that, all the docs are online-only, and not very scraping-friendly (wget --mirror doesn't rewrite links correctly when you're also mangling the filenames for local-friendly access -- patch in the making if I can be arsed). So, long story short, I need a non-traditional way to get myself some local docs. Turns out it isn't as hard as it sounds, as evidenced by the below command list:

	git clone git clone git://github.com/sstephenson/prototype.git
	cd prototype
	git submodule init
	git submodule update
	sudo gem install treetop
	rake doc:build

This will peg your CPU for an inordinate amount of time (a smidge over 10 minutes for me; -j 3 would have made things a lot quicker), but you'll end up with a doc/index.html which has some useful stuff in it. It'll look nothing like what's on the website, but it's something, and should be enough to keep me going on those long, lonely train journeys.

---

1. which looks like someone should be expecting a C&D from O'Reilly for look and feel, unless it's a semi-sooper-sekrit O'Reilly production that they don't want to put their name to directly.

As I promised to keep you updated on recent developments on update-manager I am writing this article. Just as a disclaimer: I am not going to write about any recent developments here, but would rather like to point at a piece of code I added to update-manager that could be useful in other applications too. Now, as the title suggests there are sphinx-aware Enums in update-manager. Enums are common constructs in other programming languages like C and allow simple creation of constants with, for example, ascending values (first constant has value 0, second has value 1 and so on). Python unfortunately does not include support for Enums itself, but I found it rather easy to write classes that emulate such a construct. Nothing is new about Enums in Python and there are probably quite a few different implementations out there, but I believe mine is different. The sphinx-aware part means that my implementation automagically updates the docstrings of the created instances and thus allows sphinx autodata method to include sensible information in generated API documentation. I could go on writing about and praising my method, but I believe a short example gives you a better idea how my implementation works and what I wanted to achieve with this. Have a look at this page, which is part of update-manager s new API documentation. You should see rather well-looking documentation of the UpdateManager.Backend.RELOAD_CACHE_STATUS NegativeEnum, the defined constants, their values and some additional information about each value now. Still, nothing too fancy, HTML documentation generated from docstrings. What makes this special is the code from which it was generated:

RELOAD_CACHE_STATUS = NegativeEnum(
  BEGIN = "Started reloading package cache",
  DONE = "Finished reloading package cache")

This not only gives us a RELOAD_CACHE_STATUS enum, along with the RELOAD_CACHE_STATUS.BEGIN and RELOAD_CACHE_STATUS.DONE, but also some documentation, included in RELOAD_CACHE_STATUS docstring, that can be used by sphinx. You can find the Enum code, which is rather short and should be quite easy to understand, here. I hope you find this code as useful as I do.

Due to some things falling into place, I had an opportunity to attend the first two days of last week's Ubuntu Developer Summit in beautiful Barcelona. Somehow, I had never managed to attend a Debian conference either, so it was good to meet a few of the old Debian hands now moving Ubuntu along, as well as a few of the Ubuntu folks. I also gave a short presentation on R in Debian / Ubuntu and the plans for the upcoming Ubuntu release. More on that another time. All told, a well-organised conference in a nice setting -- two stone throws from the legendary Camp Nou. Unfortunately, I had to leave by Wednesday so I missed what was undoubtedly quite a scene in Barcelona following Barca's dismantling of Man U in this year's Champions League final.

I have just got a LG U990 Viewty mobile phone [1]. It s a 3G phone and came free on the $29 monthly cap plan from Three (minimum monthly spend is $29 - but this is free if you use $29 per month). My previous plan was the $29 cap but had a minimum spend of $20 per month, as I never happened to use less than $29 per month I am not paying more. For a modern mobile phone the actual phone functionality is a sideline. If a device was strictly designed to be a phone then I think it would be very similar to the Nokia phones that were available 5+ years ago - the Nokia I had in 1999 performed every phone function that I desired of it. Like most modern phones the LG U990 Viewty suffers in it s phone functionality from the desire to make it do non-phone tasks and from the desire to cripple it to meet the desires of the carriers (not the desires of the users). For example the home screen will always have at least two buttons for paid Three services and I have no configuration option to remove them. Replacing them with speed dial options for a couple of numbers that I regularly call would be handy. As the main screen is a touch-screen there is no excuse for this, they should allow the software to be reconfigured with more useful options. Eventually the Android will kill most of the other phones and this problem of phones being designed to suit the telephone companies instead of the users will be solved. One of the most annoying mis-features of the phone is that it doesn t properly handle address book entries with multiple phone numbers. One of my friends turns his mobile off when he is at home, so I regularly call his mobile and then immediately call his home number if the mobile is unavailable. With my previous two mobile phones I could press the dial button to bring up the list of previous calls and then use the arrow buttons to select from the other numbers that are attached to the same address-book entry, so with three button presses I would be dialing his other number. With the Viewty I have to go back to the address book. The compelling feature of the Viewty is the camera and display. It has a 5MP camera which makes it the second highest resolution camera-phone offered by Three - the LG Renoir (KC910) has 8MP but needs a $99 plan for it to be free. It also has a 240*320 resolution touch-screen display (in a quick search in January when I bought my phone the best resolution display I could find on a phone is 240*400 in the LG Renoir). While the camera is documented as being 5mp there are no specs available about the resolution of the CCD. I want to use the native resolution of the CCD for pictures (I think interpolation is a waste of space). The CCD might actually be 5mp, a picture of the 1400*1050 resolution screen on my Thinkpad allows me to read all the text even when small fonts are in use, so the CCD resolution must be significantly greater than 1400*1050. This is a really important feature as the Viewty will work well for making screen-shots for bug reports about crashed computers (several of my clients have expressed interest in getting one after seeing such a demonstration). One annoying problem is that the camera software takes a while to load, my trusty Sony digital camera starts a lot faster and the LG U890 phone I used for the past two years is also a lot faster and more convenient. This won t work well for photographing unexpected events. When I am traveling by public transport I will photograph the relevant pages of my street directory as I can zoom in to photos of the maps and read the street names, it saves some weight when traveling. The 2G micro-SD card (which incidentally cost $10 from OfficeWorks) will allow me to store a lot of maps. One interesting feature is the video recording capabilities. It can do 640*480 resolution at 30fps (which is pretty good) and 320*240 resolution at 120fps (they claim that you can film a balloon popping). In my quick tests the standard 640*480*30fps mode works well, but the 120fps mode requires much brighter light than most of my test environments, so I have not yet got it working properly. The phone has a reasonable voice recording function, it can record considerably more than 34 hours of audio and the quality is reasonably good if you use an external microphone. It is however quite poor if you use the built-in microphone for a dictaphone function, it seems that quality is poor at any distance. I had wanted to record my LCA mini-conf talks with my phone but unfortunately forgot to bring the adapter for the external microphone. It s a pity that the phone doesn t have a standard microphone socket as I have misplaced my Viewty microphone, when designing the phone they should assume that misplacing attachments is a common occurrence and design it to use common parts. I recently spoke to a journalist who uses his mobile phone to record interviews. He said that his phone supported phone calls, voice recording, and taking pictures - all the essential tasks for his work. It seems that the Viewty would be better than most phones for journalistic work apart from the issue of low quality voice recording when you have misplaced your external microphone. The text editor is unfriendly in the keyboard mode (I have not tried hand-writing recognition), one thing I don t like is the fact that the letters jump when you press them. This does allow changing a letter by moving the stylus before releasing the press (some people consider this a great feature). There are no cursor control keys (which is a serious omission), and the keyboard doesn t resemble a real keyboard. My iPaQ is far better for writing (I once wrote a full-length magazine article on an iPaQ). The stylus is quite strange and interesting. In the picture of the front of the phone the stylus is compacted with it s lid on. In the picture of the back of the phone the stylus is extended with the lid off. The end of the stylus clips in to the lid so that removing the lid drags the central part out of the body. It s an interesting design and with the string on the lid allows the stylus to be attached to the phone when it s not being used. But I have never used it. Even with an iPaQ (which had a proper stylus that attached firmly inside the body of the device) I often used a fingernail on the touch screen. I have not felt the need to ever use a stylus with my Viewty. One final noteworthy thing is the support for Google services. It seems to have client support for YouTube, Google Maps, GMail, and Blogger. This seems to be a major win for Google, the Viewty is one of the most popular phones at the moment and I expect that lots of people who buy them will now have an incentive to use the Google services. Between these sorts of deals and the Android I think that it will be necessary to have some sort of anti-trust action against Google. Google are generally doing good things for the users. I have been quite satisfied to use Google search, Google advertising on my blog, and Gmail. Also I have been moderately happy with Blogger (it was good when I started blogging) and Google Maps is useful on occasion. So generally I am happy with Google, but monopolies are bad for the users so I think that if things continue on their current trend then Google may have to be split into several little Googlets some time in the next few years.

• [1] http://en.wikipedia.org/wiki/Viewty

TurboGears 2 in debian ... soon ! After a long incubation, a few days ago TurboGears 2 has been released. Historically, I've been preferring TurboGears over Django for being closer to the open source philosophy of reusing existing components. Since the long 2.0 release was marking a gap with Django, I was eager to test the 2.0, and I was delighted to find it in Debian. Unfortunately, it doesn't seem to be in good shape yet. In particular it lacks several dependencies before it can even be used to quickstart a project, in spite of its presence in unstable. To give an idea of the needed work, after having installed it manually (via easy_install), my virgin /usr/local/lib/python2.5/site-packages/ has been polluted by 26 egg-thingies. I decided to spend some week-end time to start closing the gap, because we cannot lack such an important web framework in Debian (and ... erm, yes, also because I need it ). Here is the current status of what has been ITP-ed / packaged already (by yours truly):

• ToscaWidgets (ITP): in experimental/NEW, packaging resurrected from a never-uploaded version
• repoze.tm2 (ITP): in experimental/NEW
• transaction (no ITP) : being packaged by the Zope team (temporary unofficial package available)
• repoze.who (ITP): in experimental/NEW
• repoze.who- friendlyforms,plugins.sa,testutil (ITP): in experimental/NEW
• repoze.what (ITP): in experimental/NEW
• repoze.what- pylons,quickstart,plugins.sql : just ITP-ed

Where to Considering, foolishly, the last package as being already done, the way to go is still long:

    zack@usha:~$ ls /usr/local/lib/python2.5/site-packages/ grep -v repoze.what
    AddOns-0.6-py2.5.egg
    BytecodeAssembler-0.3-py2.5.egg
    Catwalk-2.0.2-py2.5.egg
    easy-install.pth
    Extremes-1.1-py2.5.egg
    PEAK_Rules-0.5a1.dev_r2582-py2.5.egg
    prioritized_methods-0.2.1-py2.5.egg
    sprox-0.5.5-py2.5.egg
    sqlalchemy_migrate-0.5.2-py2.5.egg
    SymbolType-1.0-py2.5.egg
    tg.devtools-2.0-py2.5.egg
    tgext.admin-0.2.4-py2.5.egg
    tgext.crud-0.2.4-py2.5.egg
    TurboGears2-2.0-py2.5.egg
    tw.forms-0.9.3-py2.5.egg
    WebFlash-0.1a9-py2.5.egg
    zope.sqlalchemy-0.4-py2.5.egg

Possibly, some of them are already in Debian hidden somewhere, but sure there is still work to be done. If you want to help, you are more then welcome. The rules are simple: all packages will be maintained under the umbrella of the Python Modules Team, but you should be willing to take responsibility as the primary maintainer. Please get in touch with if you are interested, as I'm in turn already in touch with some other very kind volunteers (thanks Enrico and Federico2!) to coordinate who-is-doing-what. Preview packages available What has already been packaged, including a temporary workaround for python-transaction, will be available in experimental after NEW processing. The idea is that nothing will go to unstable until TurboGears 2 will be (proven to be) fully functional. In the meantime, packages are available from my personal APT repository (signed by my key), here are the friendly /etc/apt/sources.list :

    deb http://people.debian.org/~zack/debian zack-unstable/
    deb-src http://people.debian.org/~zack/debian zack-unstable/

Versions are tilde-friendly and shouldn't get in your way when the official packages will hit unstable. Packaging multiple-egg / multiple-upstream packages In all this, I've faced an interesting problem with the python-repoze. who,what -plugins packages. They correspond to a handful of plugins, each of which is about 20/30 Kb. I didn't consider appropriate to prepare 5 different packages, due to archive bloat potential. Hence, you get the usual problems of multiple-upstream packages. To counter some of them, as well as some egg-specific packaging annoyances with multiple upstream, I wrote a couple of very simple helpers:

• get-orig-source.mk (live version): a debian/rules snippet which relies on several debian/*.watch files which are fed to uscan to implement (not yet completely) the get-orig-source target described in policy
• setup.py (live version): a multiplexer for sub-directories shipping distutils-enabled Python modules. That way you can use /usr/share/cdbs/1/class/python-distutils.mk as usual, and setup.py will take care of the "multiple upstream" problem at build time

I'm no Python-packaging-guru so, lazyweb, if you spot in this choice something I utterly overlooked, or if you have improvements to suggest, please let me know. Repacking eggs An interesting problem will be faced in trying to integrate the above approach with Python modules that are shipped only as .egg files (i.e., no tarballs, ... yes there are such horrifying things out there). To preserve uniformity, we would need uscan to support --repack ing of .egg files as they were simple .zip archives. Since eggs are .zip archives ... why not?

On Thursday it was my birthday, and I fulfilled a promise I made to myself many years ago. I've always been a car nut, and when I was about 6 years old and had posters of Lamborghinis on my bedroom walls I promised myself that by the time I reached 35 I would have one. That would be long enough in the future that it might happen, but I'd still be young enough to still be able to appreciate it. Well, as it got to the beginning of this year I realised I was running out of time and my bank balance didn't quite look large enough to buy a Lamborghini. So, I cheated a little and I hired one for the day on Thursday. More pictures here. I had one car booked a few weeks ago, but unfortunately the company involved had to let me down as the car developed problems a couple of days before I was due to have it and needed to go away for repairs. I'm not going to complain too much about them, as it wasn't their fault and they did their level best to make things right: offering a different car, a refund or a later replacement hire. But dammit! I wanted the Lamborghini and I wanted it on my birthday! I quickly rang around on Wednesday to find somebody else, and thankfully the folks at Signature Car Hire were very helpful, sorting out a car with just a few hours' notice. Yay! When they guys turned up on Thursday morning to deliver the car to my office, I couldn't stop myself grinning like the 6-year old boy whose promise I was keeping. Here was a gorgeous car, and it was mine for the day!!! I planned a route for the day to take in some nice driving roads and, importantly, some tunnels. I've watched Top Gear on the TV and I read Autocar every week, and they had both said that taking a fast sports car through a tunnel with the roof down was a good thing to do. I now know that's an understatement! As a whole, the car was awesome: the acceleration, the handling, the brakes were all wonderful. But, as I found out on that day, the noise from the engine was just astonishing. I went through several tunnels that day, heading down past Baldock to Hatfield and back, and at full throttle I could not help but laugh out loud at the music coming from the 5-litre V10 orchestra just inches behind me. It's an experience I'm never going to forget. grin. I also gave short rides to quite a few of the people at work; oddly I had lots of friends that day... :-) I'd also planned other things for the day. I bought myself a new camera in advance to make sure I got some nice photos of the day. Then at the end of the day I went out to the pub with a huge group of friends and I was forced to drink lots of beer. It was horrible, horrible I tell you.

An interesting opinion post in the NY Times describes the research on early education and how it can affect IQ [1]. Among other things children from poor families who are adopted into upper middle-class families tend to end up with higher IQ scores. The article notes that half the population in 1917 would be regarded as mentally retarded by today s standards - finally we have an explanation for WW1! Two Dominos employees tarnish the brand s image with a prank video on Youtube [2]. The next obvious step is for activists to seek jobs at such companies for the purpose of influencing companies. The animal rights protesters outside KFC stores haven t achieved much, but if they worked for KFC and made some nasty videos they would really encourage a change of action. I predict that chain stores will be spending a lot more on security and background checks for their employees in the near future. Cory Doctorow has written an amusing article titled Big Entertainment Wants to Party Like It s 1996 about how the entertainment industry is killing itself by conducting back-room negotiations about new copyright laws [3]. Nate Silver gave a TED talk about racism in elections [4]. The most interesting point was demonstrating statistically that people who don t meet people of other races tend to be more racist. It seems to me that the use of the X-Face: header in email and the use of HackerGotchi in Planets can help reduce the level of racism on the Internet. Cory Doctorow writes about his Geeky writing [5]. His idea for an organised system for donating books to libraries will hopefully be fully implemented soon - it should be easy to do and the incremental costs will decrease as the scope increases. Cory Doctorow writes about the perverse laws that protect criticism of copyright works but stifle praise [6]. In a similar note he has documented a plan for trademark and copyright holders to allow fans to create derivitive works while preserving the original rights AND sharing the profits [7]. So if Cory s idea became popular someone who wanted to create some art work based on a Coke bottle (which is trademarked) could pay the Coca-Cola company a reasonable rate, include an appropriate disclaimer, and things would work out well for everyone. Also this would allow small artists to develop new products that could be used by the large companies (I m sure that anyone who legally released an artwork that turned out to be an effective advert for Coke would receive a lucrative job offer). Bruce Schneier s blog has an interesting article about the poor quality of software used for breath alcohol detectors [8]. It s a great concern that innocent people are being punished due to bad software, but it s only a small part of the problems with the legal system. Mary Roach gave a TED talk 10 Things You Didn t Know About Orgasm [9]. Not as insightful as the usual TED talk, but strange and interesting.

• [1] http://www.nytimes.com/2009/04/16/opinion/16kristof.html


• [2] http://www.nytimes.com/2009/04/16/business/media/16dominos.html


• [3] http://www.internetevolution.com/document.asp?doc_id=175415


• [4] http://www.ted.com/index.php/talks/nate_silver_on_race_and_politics.html


• [5] http://www.locusmag.com/Perspectives/2009/05/cory-doctorow-extreme-geek.html


• [6] http://www.guardian.co.uk/technology/2009/may/13/cory-doctorow-copyright


• [7] http://www.internetevolution.com/document.asp?doc_id=176353


• [8] http://www.schneier.com/blog/archives/2009/05/software_proble.html


• [9] http://www.ted.com/index.php/talks/mary_roach_10_things_you_didn_t_know_about_orgasm.html

Last weekend I finally managed to travel to Barcelona to visit my family and some friends. As my agenda was quite packed with stuff to do, I was unable to find out if any of the Ubunteros had arrived early for UDS, and I left just after lunch on Sunday. Unfortunately, I had totally missed that before UDS, Canonical held their allhands meeting, and it would have been easy to meet them on Friday night after I got in the city. What a pitty, and sorry about this, mdz, I would have loved to meet... :( In other Barcelona news, I'm sure that UDS attendees will be astonished (or fed up!) by the football crazyness going on right now. Last night I went to a culer bar near Woody and enjoyed watching how Bar a claimed brilliantly their 3rd Champions Cup. For someone who normally doesn't care that much about football, the last few weeks have been incredible. Today I visited my 96 year old grandfather, and even if he has lost much of his expressiveness and energy since the last few months, today he was visibly happy and proud of what his Bar a has managed to accomplish this season. Three titles, plus literally going over Real Madrid in Santiago Bernabeu. Bar a is definitely m s que un club, and I'm happy that my grandfather was able to enjoy it.