Ruby has a reputation for being slow. But there s slow, and then there s three seconds to show the help for a fairly command line program . That s not slow, that s ridiculous. The gem cleanup command is the solution. Rubygems has a nice feature whereby you can have multiple versions of a gem installed at once. That s neat, because it allows programs with different gem version requirements to co-exist on the system. Unfortunately, over time, you can end up with many versions of a gem installed, as upgrades pull in ever-newer versions of all your gems. Couple this accumulation of cruft with a need for Rubygems to look at every one of them on every startup, and you can see how you could very quickly end up with a three second startup time. By running gem cleanup, you ll have the opportunity to nuke all the out-of-date versions of your gems. It takes into account dependencies, so it will at least ask you before nuking a gem that another gem absolutely depends on (I d prefer a command-line switch to say, Don t even think about uninstalling something that another gem depends on! , but you can t have everything). If you ve got non-gem things on your system that rely on out-of-date gems (anything using bundler, for instance), those will assplode the next time you run them, but you can always re-run bundle install to get just the specific versions of gems back you need. The end result of my little spring cleaning? Down from three seconds to three quarters of a second. A 75% improvement. Win!

Today (well, technically yesterday by now), the Media were reporting of an open letter by '5 belgian university professors from the IT field' without calling them by name about how Belgian primary education should introduce programming into the curriculum, starting at age 5. Unsurprisingly, this did not go well with a majority of the Belgian populace. Arguments like "what should the school drop to make place for that, then" and "why would people ever need to learn programming" were rampant on some internet fora that I had a short look at. That being said, though, I think these five unnamed professors are correct. Understanding how a tool works is essential to proficient usage of said tool; and while it is possible to teach particular common and popular use cases of a given tool without providing enough background, doing so will only result in confusion when the computer "is acting up" an oxymoron if there ever was one. After all, a computer can't "act up"; it can only follow instructions. Whenever your seems to be "acting up", what's really happening is that someone (most likely, you) gave it the wrong instructions. Or, perhaps, someone evil on the Internet gave it the right instructions (for their goals, anyway) and infected your computer with a virus. But that's unlikely. Computer education currently mostly consists of "teaching software": rather than explaining how a computer does what it does, people are taught that if you click this button, the text will be bold, or if you click that button, the printer will start buzzing. That's all nice and dandy, until the actual computer people encounter when off the school bench and at work has this other piece of software, where all the buttons are in other places and look all "wrong". Because then they get all confused about the fact that things aren't where they're supposed to be. So, yes, it is my opinion that any computer education should help people understand how a computer does the things it does. And what better way to do that than to actually explain things to a computer, the way computer programmers do such things? Note that programming a computer doesn't have to be hard, and it can be something which kids can understand and will find fun to do. For instance, there's this scratch thing from MIT, which was made for teaching programming in such a context. Yes, adding programming to the primary school curriculum will require that some other things are dropped instead, and I could see that being a problem. However, in our modern world computers have become so pervasively important that we would be doing our kids a disservice not to explain them how to understand computers, as opposed to just learn how to use them...

I have written a number of scripts to create Debian foreign architecture (mostly armel and armhf) rootfs images for SD cards or NAND flashing. I started with putting Debian on my Openmoko gta01 and gta02 and continued with devices like the qi nanonote, a marvel kirkwood based device, the Always Innovating Touchbook (close to the Beagleboard), the Notion Ink Adam and most recently the Golden Delicious gta04. Once it has been manufactured, I will surely also get my hands dirty with the Neo900 whose creators are currently looking for potential donors/customers to increase the size of the first batch and get the price per unit further down. Creating a Debian rootfs disk image for all these devices basically follows the same steps:

1. create an disk image file, partition it, format the partitions and mount the / partition into a directory
2. use debootstrap or multistrap to extract a selection of armel or armhf packages into the directory
3. copy over /usr/bin/qemu-arm-static for qemu user mode emulation
4. chroot into the directory to execute package maintainer scripts with dpkg --configure -a
5. copy the disk image onto the sd card

It was not long until I started wondering why I had to run all of the above steps with superuser privileges even though everything except the final step (which I will not cover here) was in principle nothing else than writing some magic bytes to files I had write access to (the disk image file) in some more or less fancy ways. So I tried using fakeroot+fakechroot and after some initial troubles I managed to build a foreign architecture rootfs without needing root priviliges for steps two, three and four. I wrote about my solution which still included some workarounds in another article here. These workarounds were soon not needed anymore as upstream fixed the outstanding issues. As a result I wrote the polystrap tool which combines multistrap, fakeroot, fakechroot and qemu user mode emulation. Recently I managed to integrate proot support in a separate branch of polystrap. Last year I got the LEGO ev3 robot for christmas and since it runs Linux I also wanted to put Debian on it by following the instructions given by the ev3dev project. Even though ev3dev calls itself a "distribution" it only deviates from pure Debian by its kernel, some configuration options and its initial package selection. Otherwise it's vanilla Debian. The project also supplies some multistrap based scripts which create the rootfs and then partition and populate an SD card. All of this is of course done as the superuser. While the creation of the file/directory structure of the foreign Debian armel rootfs can by now easily be done without superuser priviliges by running multistrap under fakeroot/fakechroot/proot, creating the SD card image still seems to be a bit more tricky. While it is no problem to write a partition table to a regular file, it turned out to be tricky to mount these partition because tools like kpartx and losetup require superuser permissions. Tools like mkfs.ext3 and fuse-ext2 which otherwise would be able to work on a regular file without superuser privileges do not seem to allow to specify the required offsets that the partitions have within the disk image. With fuseloop there exists a tool which allows to "loop-mount" parts of a file in userspace to a new file and thus allows tools like mkfs.ext3 and fuse-ext2 to work as they normally do. But fuseloop is not packaged for Debian yet and thus also not in the current Debian stable. An obvious workaround would be to create and fill each partition in a separate file and concatenate them together. But why do I have to write my data twice just because I do not want to become the superuser? Even worse: because parted refuses to write a partition table to a file which is too small to hold the specified partitions, one spends twice the disk space of the final image: the image with the partition table plus the image with the main partition's content. So lets summarize: a bootable foreign architecture SD card disk image is nothing else than a regular file representing the contents of the SD card as a block device. This disk image is created in my home directory and given enough free disk space there is nothing stopping me from writing any possible permutation of bits to that file. Obviously I'm interested in a permutation representing a valid partition table and file systems with sensible content. Why do I need superuser privileges to generate such a sensible permutation of bits? Gladly it seems that the (at least in my opinion) hardest part of faking chroot and executing foreign architecture package maintainer scripts is already possible without superuser privileges by using fakeroot and fakechroot or proot together with qemu user mode emulation. But then there is still the blocker of creating the disk image itself through some user mode loop mounting of a filesystem occupying a virtual "partition" in the disk image. Why has all this only become available so very recently and still requires a number of workarounds to fully work in userspace? There exists a surprising amount of scripts which wrap debootstrap/multistrap. Most of them require superuser privileges. Does everybody just accept that they have to put a sudo in front of every invocation and hope for the best? While this might be okay for well tested code like debootstrap and multistrap the countless wrapper scripts might accidentally (be it a bug in the code or a typo in the given command line arguments) write to your primary hard disk instead of your SD card. Such behavior can easily be mitigated by not executing any such script with superuser privileges in the first place. Operations like loop mounting affect the whole system. Why do I have to touch anything outside of my home directory (/dev/loop in this case) to populate a file in it with some meaningful bits? Virtualization is no option because every virtualization solution again requires root privileges. One might argue that a number of solutions just require some initial setup by root to then later be used by a regular user (for example /etc/fstab configuration or the schroot approach). But then again: why do I have to write anything outside of my home directory (even if it is only once) to be able to write something meaningful to a file in it? The latter approach also does not work if one cannot become root in the first place or is limited by a virtualized environment. Imagine you are trying to build a Debian rootfs on a machine where you just have a regular user account. Or a situation I was recently in: I had a virtual server which denied me operations like loop mounting. Given all these downsides, why is it still so common to just assume that one is able and willing to use sudo and be done with it in most cases? I really wonder why technologies like fakeroot and fakechroot have only been developed this late. Has this problem not been around since the earliest days of Linux/Unix? Am I missing something and rambling around for nothing? Is this idea a lost cause or something that is worth spending time and energy on to extend and fix the required tools?

Hello, World! Docker s in Debian! Isn t that great! Let s all get happy! It s called docker.io, so go ahead and sudo apt-get install docker.io whenever you want :) The first two uploads have a few errors, which are 100% my fault. The first were a set of FTBFS bugs, which were a stupid error on my part. Thanks to olasd for catching the remaining bug, related to stripping the binaries. I m so sorry this happened, it slipped through as a result of me having a local docker binary in /usr/local, which I tested completely before uploading a totally different binary. I won t let it happen again. It should be fixed now. However, this comes with a warning. It appears as though systemd (which, for the record I adore) is allowing lxc-start unmount /dev/pts and friends, which causes a bunch of damage to the host. I ve filed the bug as bug #734813. Currently the outdated mount(1) is holding back a workaround. Hopefully we get util-linux updated so we can hack around this in the short-term. Hopefully the bigger issues get solved. So, if you re a systemd user, please hold off on using docker.io until we resolve this issue in Debian.

Valve Software s Steam is the number one digital game distribution service, with more than 65 million registered accounts. Steam runs on Windows, Mac OS, and Linux x86/amd64 computers, and provides access to several thousand games, at varying price points an enormous growth from less than a dozen games for Windows only about a decade ago. Valve s latest endeavour has been to bring their storefront into the living room, with a three-pronged attack: a new game controller, a programme of licensed x86 consoles to plug into the family TV, and an OS to run on the Steam Machines to tie it all together. December saw the first public release of their SteamOS which, as it turns out, is basically just a preconfigured desktop Linux. Specifically, it s a Debian Wheezy derivative, comprising a subset of 502 source packages from Wheezy; 8 of Valve s own source packages; and 51 source packages which have been either patched compared to Wheezy, backported from post-Wheezy, or both. For example, the compiler used by default is gcc-4.7 (rather than Wheezy s 4.6) and the libc version postdates Wheezy too. Valve s official instructions and installer release concerned quite a few people who had planned to try SteamOS on an older PC, by mandating a large (500GB) hard disk and a PC with UEFI firmware. Very quickly a number of instructions started appearing from people trying to fix what people felt were real issues specifically provision for BIOS-based computers, and installation from optical media. After being assured that redistribution of derivatives of SteamOS were entirely authorized (and, in fact, encouraged) I decided to produce my own variant, calling upon my own experience with debian-installer modification from past and present jobs, as well as calling upon the skills and experience of the UK Debian community as needed. The end result is Ye Olde SteamOSe. This weekend saw the third release of Ye Olde SteamOSe, a derivative designed to greatly widen the pool of computers capable of running Valve s OS. And unlike the first two releases, the public response this time has been crazy. Like, totally crazy. Combined score across several subreddits totals about 1700. 7 pages of Google search results. Hundreds of tweets. Mentions in dozens of blogs around the world. Coverage on the Linux Action Show. Unilaterally added to Softpedia s list of distributions. Almost 2000 views of a video installation walkthrough I posted on YouTube. Crazy. Sadly the idea to try and track visitors to the page didn t occur to me until long after the initial rush subsided, but 1000 visitors on Tuesday for a news story which landed on Sunday is still pretty hot in my book. It s also interesting to observe the demographics of site visitors the #1 referrer on Tuesday was Dutch PC site Tweakers.net, and StumbleUpon outranks reddit for referrals. About 66% of visitors interested in installing my Debian Wheezy derivative derivative came using Windows (20% using Linux), which suggests there s a lot of potential Linux users out amongst the Windows gamer masses. So what s the purpose of this self-congratulatory blog post? Just dick-waving? Well, there s an element of that (I m only human), but I think it might be nice to alert the audience on Planet Debian/Ubuntu, many of whom are not big gamers, to the next big thing in embedded Linux except this time a real GNU/X.org/sysVinit distribution, not some NIH thing like Android. So now you know!

It's almost a month since I last blogged something, and one of my new year's resolution is to change that, a bit. Let's see how it goes. I've been listening a lot to this great band from Sweden recently again, put all their songs onto my mobile phone. It might sound weird because they have a rather aggressive style, both soundwise but also lyricswise, but it helps me to get things off my chest and stay relaxed in the rest of my life. The band I want to present to you was already twice mentioned in some other articles of my blog I noticed, but this is the proper post about them: I'm talking about Clawfinger. They came up in the nineties during the crossover phase and did blend in pretty well, but it's mostly their direct and political statements they carry in their lyrics that did let them stand out. One warning though: the direct language they use might be considered blunt and maybe even offensive by some. The message behind it though should rather get you thinking of your own doings if you consider it to be offensive. Here are the songs:

• What Are You Afraid Of: Most people would have probably gone with their first hit for the topic, but I think this song transports the point extremely well too: It's just a color and I'm color blind, the only color I know is the color of my mind. There's only one race and that's the human race, and every human being's got the right to feel save.
• The Faggot In You: Well, it's like Coyote Too twittered or Andy Singer drew it. Get over it and think about what actually stirs these emotions in you. If you're so sure and you feel secure about yourself and your reality, then why do you need reject and refuse where other people stand sexually?
• Life Will Kill You: Let's face the fact, it will. ... and given the choice between your own life and death I suggest that you cherish the time you have left 'cause time waits for noone and we're all growing older. Life for today, not for a future that might never come.

Like always, enjoy! And maybe also think about it a bit. :)

/music permanent link Comments: 4 Flattr this

Review: Honeyed Words, by J.A. Pitts

Series:	Sarah Beauhall #2
Publisher:	Tor
Copyright:	July 2011
ISBN:	0-7653-2907-7
Format:	Trade paperback
Pages:	412

Honeyed Words is a direct sequel to Black Blade Blues and probably doesn't make sense to read out of order. Sarah, Katie, their friends, and Katie's family are still recovering from the climactic events of Black Blade Blues and licking their wounds, but things are slowly looking up. Sarah is getting more comfortable openly expressing her sexuality, although there are still tensions, and her relationships are improving. The city of Vancouver also seems to be recovering. But major political shifts among the nonhuman races have echoes and ramifications, and after a kidnapped singer and a murdered dwarf, Sarah is pulled back into the turmoil. If you liked the world-building in Black Blade Blues, Honeyed Words is more of the same, but with more variety. Pitts adds elves to dwarves and dragons, shows considerably more of the supernatural world beneath the surface of human cities, and introduces Sarah to some new types of danger. There are a few more glimmers of the history of Sarah's sword and the nature of the gods in this universe, but still nothing conclusive. Pitts broadens the world-building rather than deepening it. But I liked the new characters. Teenage elves with obsessions make for great characters, and the demon who shows up about halfway through is delightful. As with the first book, this is mostly a story of things happening to Sarah while she tries to muddle through as best she can. There isn't much in the way of grand strategy or of seizing control of the plot. She just tries to do the right thing and gets lots of help from lots of friends. But the plot moves right along, and it's mixed with a healthy sense of normality that I find refreshing. She needs work as a blacksmith, she's trying to support a friend without having much money of her own, the living arrangements this forces are a bit of a strain for her own relationships, and her broken relationship with her family continues to lurk. Much urban fantasy is built around the premise of a regular person thrust into a fantasy world, but much of it then discards normal life once the supernatural starts. I think Pitts does a particularly good job of keeping his protagonist's life grounded in the ongoing mundane problems of living. It's also more fun to read about Sarah when she isn't quite as angstful. Over the course of Honeyed Words, she seems to be growing into a person who can accept who she is and feel secure about her place in the world. That's nice to see. The downside to this book is that, while it's full of quite a lot of stuff, not all of it coheres that horribly well. It has a certain "middle book" feel. The major plot resolves, but with some significant lingering consequences and something of a cliff-hanger last chapter. Most of the threads left open by the first book are still open here. There's character growth, and setup for some resolution, but not a lot of actual resolution. I still like this series, though. I like urban fantasy that isn't about vampires and werewolves; berserker blacksmiths and Norse legend has much more appeal, and I want to read more of it. Pitts's writing is a bit clunky and probably won't win many rewards, but it's serviceable and I think improving. If you enjoyed the first book, it's worth looking for this one as well. Followed by Forged in Fire. Rating: 7 out of 10

Packt Publishing sent me Krzysztof Niki ski's Instant Haml to do a short blog review... Thing is, and those who read me know, I cannot write short reviews ;-) So lets see how this goes. Packt's Instant series First, as for the format this book follows and the series it fits in: They are very short books. I was the tech reviewer for their Instant Debian Build a Web Server , and when I opened it for the first time, I thought I had received only the first chapter. This series' motto is Short Fast Focused so, yes, the books are ~50 pages long, with ample margins and many big screenshots. I told the editors I was quite unsure about the value this kind of publications add, as they are very similar to online tutorials, but given they have so many books in this series (printed and electronic), there must be a market for them. They surely know their business better than me! There is one other detail regarding the publisher I'll stick in this section, although it is not specific to their Instant series: I do most of my reading in my Kindle. Packt seems to have chosen to specify a very small font for their texts. Besides the two books mentioned, I also have their Object Oriented JavaScript book. For all of the other books I read, I use the smallest font setting. Packt books in mobi format (what they present as Amazon format in their web site) are basically illegible without bumping up the font size through the roof. As I understand it, publishers should use industry-wide default sizes, and let the reader adjust their reading experience. Yes, books of technical nature (such as Packt's) often benefit from having more displayed information than what an ebook reader normally shows... But that's for me to decide! And no, this is not a Kindle-specific glitch: I also get a similar experience when opening it from Calibre. And as for the book... I am a long-time Haml convert/fan; I packaged its Ruby implementation for Debian (and should update it soon :-/ as new versions have appeared), the corresponding Emacs Lisp mode, and I use it for basically all of my static HTML sites and Ruby-based Web systems. Haml is a great templating language, meant to make dynamic HTML generation leaner and easier. It is a clean and elegant technology, and can clearly sell itself. And the project webpage has very good documentation, ranging from a short tutorial and a full API documentation for people to contribute to the project. The first part of this book, of course, covers the "why is Haml so great?" part, presenting the basic equivalences with HTML. In my opinion, the book could be structured a bit differently and would generate a stronger following. Why? Because, even given that Haml is greatly benefited from its Ruby integration and has many hooks to be tightly integrated in a Rails workflow, it can also be used by people who don't want to do Ruby. I have some static websites that are just a bunch of Haml files, compiled by a very simple Makefile and calling the haml command from the shell. I think there would be some value in presenting this invocation before learning how to call it from the Gems environment (i.e. bundle install). Using Haml as a standalone tool is only presented almost as an appendix ("Using HAML outside of Rails" section). Even more so if the author presents four different static-site solutions (probably more non-programmer-friendly than my trusty Makefile). Oh, and this little section uses the (old? obsolete?) uppercase HAML notation for the project's name instead of the now-standard Haml. Dirty screenshots About the screenshots presented in this first section (and I refer mainly to the scaffolding examples), they are meant to show how easy things are, but have an important stylistic mistake: Terminals with transparency might look cool (I hate using them, but then again I'm a very boring person when it comes to my computer habits ;-) ), but transparencies have no place in a book (be it printed or e-book), as the images appear very uncrisp. In the Kindle, they just seemed dirty Now that I'm browsing the PDF from my desktop, I see that the screenshot in "Step 1" has a photo of Earth from space behind the terminal, and even worse, in "Step 2" it is shown over a Web page detailing the installation! These elements are just distracting, and the text would be much cleaerer if it were presented typeset just as a regular code fragment. Opening many of those screenshots in the Kindle was plain useless. Editor integration It is perfectly clear from this text that the author strongly prefers Sublime Text, sometimes uses TextMate, and has just heard reports that Haml has some support in Vim, Rubymine, Emacs and Coda. Of course, devoting a full page (as it is for the first case) for each editor would be out of proportion for this size of book... I just felt the imbalance too big, specially given that most editors will sport approximately the same feature set. Text redundancy If the Instant series is about making short books, I would expect them to be loaded with easy to follow content (as they are), but free of repetition. I felt, however, a large chunk of step 1 of the "Quick start" chapter to be repeated with the the first section of the "Top 6 features" chapter. Yes, the second one adds some depth, but very little. Still, in the later parts of this "Top 6 features" chapter, a more thorough explanation could be used for the Filters, Multiline attributes/code/strings and helpers and extensions sections. It seems the author was on a hurry trying to get it finished, as it is barely explained Explaining a bit more deeply would surely benefit readers. Wrapping up Reading my review, I'm mostly talking about negative issues. IMO, the book is not excellent, but it is clearly not bad. Specially given it fits very well the format for which it was designed. There are many items that could be fixed, and I hope some of them can be fixed (i.e. the screenshots) before the book goes to print If I understand Packt's page correctly, it is currently only available as ebook, contrary to most of the Instant series' titles.

Qt and KDE will be present at FOSDEM, the largest open-source event in Europe. One more year, we will be sharing the Desktops DevRoom with Gnome, Unity, Enlightenment, LXQt and Hawaii (a Qt Quick desktop environment). We recently published the schedule for the devroom, which will be also available in the printed booklet available at the front desk. For the 2014 edition, the FOSDEM organization wants to achieve 100% recording of presentations. That means every presentation, in every room (devroom, lightning talk, main conference, etc) must be recorded. That s hundreds of talks. While the FOSDEM and devrooms organization teams comprise a lot of people, we are far too busy already with the organizative stuff and cannot spend time doing the actual recordings. Good thing is, you can help! Do you want to join the FOSDEM Video Team and receive the t-shirt? We are now looking for volunteer cameramen (and camerawomen, of course ). FOSDEM will provide you with equipment and training, you only need to start recording, focus, make sure nobody gets between the camera and the speaker/stage, etc. You do NOT need to record the whole track, even one talk recording would help. More details on what will be required from you are available in this e-mail from Wouter. Please contact me (pgquiles at elpauer dot org) if you are interested in recording one or more presentations from the Desktops DevRoom.

In November I resigned from Bytemark. In December I started working for a local company, here in Edinburgh, in a real office (rather than working from home). Unfortunately today I resigned from that new job, meaning I'm currently unemployed. I plan to take a 1-2 week vacation, then look for another job as a matter of some urgency. (I can live off savings for the next half-year, or so, if I need to, but I'd go crazy if I had nothing to do for that long.) It is unfortunate to have to resign from a new job after only five-six weeks, but much more honest to do so now than pretend everything was OK and do it at the point I'd passed my probationary period (of three months). The people were lovely, the office was lovely, the coffee machine was excellent, the work was interesting, but the nature of a large corporate job with the associated beaurocracy made it a less good fit for me than it looked on paper. I shall pretend that the next week or two of down-time is our honeymoon ;)

After playing around with How secure is my password, I dediced to start changing the handful of passwords I have committed to memory to being very long phrases rather than shorter, concentrated collections of random-seeming characters (another interesting blog post would be to understand the algorythm behind that site but for now I've just taken it at face value). Despite my usual practice of introducing new passphrases by changing my laptop login (which can always be reset if necessary), I went ahead and changed by GPG key instead to a new 50 character phrase. Fifteen minutes passed before I needed access to the key and... I couldn't repeat the passphrase. I tried to be clever. I opened a text file and typed in this passphrase over and over again, ultimately generating 49 unique typo-ridden versions of the passphrase. Then, I wrote a simple script that took each of these versions and generated additional versions by:

• Changing one character (from a preselected character list of letters, numbers and punctuation within key range of the actual letters in the passphrase).
• Deleting one character
• Adding one character
• Transposing all characters

In the end I had about 350,000 versions of my passphrase. None of them worked. What in the world did I type?? Could I just ask the NSA? In any event, my new OpenPGP key fingerprint is:

1F9C30CB3CFC5DA9987FA035A014C05A607B7535

Please note: I've changed the "real name" part of my User Id from "Jamie McClelland" to "James McClelland" becuase James is on my government issued id. I'll be working on collecting new signatures for the rest of my life. Some interesting information on how this made my life difficult:

• I use the Monkeysphere for accessing the servers I maintain using by OpenPGP key. Fortunately, the key was loaded into my ssh agent, so I have still been able to access the servers. If my computer crashes or gets restarted I'll be locked out until I get my new key in place. I'll need to ask another May First/People Link administrator to sign a puppet release with my new fingerprint.
• May First/People Link uses keyringer to store shared passwords, encrypted to a small collection of administrator OpenPGP keys, including my old one. I'll need to ask another admin to update the list of keys.
• I have hundreds of (mostly) web-site passwords stored in assword and encrypted to my old key. They are all gone. I'll be doing lots of password resets now.
• I OpenPGP sign my emails. That has been turned off between the time I lost the passphrase and generated a new key.
• I am subscribed to a few email lists run via Schleuder. I can't access those emails now. I'll need to send my new key to the list.
• I've lost access to every encrypted message that has been sent to my old key.

---

Postscript: Thanks to Jordan Uggla, I was directed to passe-partout, a tool that can extract an RSA private key out of memory when ssh-agent is running with the key loaded. Since I am using the Monkeysphere, my OpenPGP key has an authentication-capable subkey, which is fed to an ssh-agent and used to grant me access to remote computers using ssh. After losing my passphrase, I was not able to access that subkey. With the help of pass-partout, and the fact that the subkey was loaded into ssh-agent before I lost the passphrase, I was able to extract the private RSA key from my ssh-agent and save it to a file. Then, I could restart my ssh-agent with my ''new'' authentication-capable OpenPGP subkey, thus allowing me to access servers that have granted me access via my new key and the handful of servers that still only grant me access via my old key.

Back in the mists of Internet time (2002) when spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri there were few hosting options for a budding open source project.

Despite the meteoric rise of many dot com companies in the early noughties a project either had to go it alone by running everything themselves or pick from a small selection of companies that had not yet worked out how to turn free hosting into money.

One such company, arguably the first, was VA Research with their SourceForge system. Many projects used their platform including a small niche web browser called NetSurf. For years the service was great, there were rough edges but nothing awful.

Over time the NetSurf projects requirements grew beyond what SourceForge could provide and service after service was migrated away eventually all that was left was the bug tracking system. This remained the state of affairs until mid 2013 when SourceForge forced a migration to their new platform which made them unsuitable for the projects use case.

Aside from the issue trackers questionable user interface SourceForge had started aggressively placing advertising throughout their platform. Some of the placements so inadvisable that projects started taking the decision to leave.

While I appreciate that SourceForge had to make money to provide a service they appear to have sown discontent within a large part of their user base without understanding that there are a number of alternatives solutions with a much less onerous funding model.

NetSurf used this as an opportunity to move the remaining issue tracking service to our own infrastructure. Rob Kendrick proceeded to evaluate several solutions and in December 2013 I finally found the time to migrate an XML dump of the old data from SourceForge into MantisBT.

Migrating data from one database into another via incompatible formats took me back to my roots. My early career started with programming tasks moving historical business data from ancient large systems which were about to be scrapped to modern Sparc based systems. Later I would be in a role where financial data needed to be retrieved from obsolete proprietary systems and moved into databases on x86 servers.

My experience in this field was not really stretched as it turns out that modern systems can process a few tens of megabytes in seconds rather than the days for a run in my youth! So some ugly perl scripts and a few hours later I had a nice shiny SQL database filled with NetSurfs bugs and MantisBT instance configured to use them.

Then came the hard bit, triaging all the open bugs, fixing up all the bugs submitted as an anonymous user but with email addresses in, removing duplicates and checking every open bug was still valid took almost two weeks of tedious drudge.

I set up an initial bug workflow within the system that the project developers are still fine tuning to better suit their needs but overall Mantis is proving a very flexible tool. The main deficiencies center around configuration for the projects useage, especially removing unused fields from filters and making the workflow more intuitive..

The resulting system is now getting bug reports submitted again where the sourceforge system had had three in the six months since the forced migration.

The issue tracker is once more a useful tool for the developers allowing us to focus on areas actually causing problems for our users and allowing us to see progress we are making fixing issues.

Overall this was a successful migration and provides a platform the NetSurf project can control where we can offer guarantees to our users about their personal information usage and having a clean rapid interface with no advertisements.

After reading Fran ois article about running your own XMPP server, I thought I d give it a go myself. A comment from Steven suggested that prosody would be a better and simpler alternative so I went with that. It is actually really simple with about 5 lines to add to the config, as well as firewall changes. The only trick is if you have a chained SSL certificate like mine you need to cat the certificate and the CA one together, otherwise you get SSL errors. So I m up and going with a XMPP contact the same as my email address which is csmall and then the domain enc.com.au
It just doesn t have anyone as contacts yet I ll be adding the DNS records for it shortly.

remctl is the middleware layer that we use everywhere at Stanford. It's a simple GSS-API-authenticated network service that supports running commands with ACLs. There are client bindings available for a wide variety of programming languages. This release fixes a couple of irritating bugs: the client library leaked memory when remctl_set_ccache was used (which was affecting mod_webkdc from WebAuth), and Net::Remctl::Backend didn't validate argument counts correctly when one of the arguments came from standard input (which affected krb5-sync). I also worked around a bug in RHEL 5's Module::Build, and added sanity checking to Net::Remctl and related classes to ensure that the object argument wasn't undef. Also new in this release are support in the remctld server for systemd startup notification and socket activation, and (via the -Z) flag) support for upstart's expect stop synchronization method. This is mostly a "clearing the decks" release in advance of more significant work. The next release will replace the server event loop with libevent, in preparation for further improvements in how the server can handle persistent worker children. You can get the latest release from the remctl distribution page.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (147.56 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook I spent a good chunk of December on the book. First finalizing the English version and getting it out (BTW, just for the launch, there s a 10% discount on the paperback that lasts only until January 9th!). Then working on updating the French translation. Eyrolles will publish a new edition of the French book based on this translation. Expect some further news about this during January! Debian France I contributed to many discussions within Debian France. Starting with a complaint that most events are organized in Paris, I proposed to map the location of Debian France members. We added new fields in the membership management page so that members can add their GPS coordinates and Fr d ric Decou made some experiments with Openstreetmap. Someone else (Kiriarat) volunteered to write the required glue code. A manual map is currently maintained on the website. In the discussions about the setup of the Debian France shop, I suggested to update our logo with a nicer looking one. We got a few suggestions and after further discussions with Alexandre Delano and Sylvestre Ledru, we organized a small contest to entice designers to submit a logo proposal to us (the winner earns a set of Debian goodies). We got 46 proposals (see my favorite on the right)! The board is currently pre-selecting the logos and setting up the final vote for our members. The winner shall be announced at the end of the upcoming mini-debconf in Paris. I also continued the work to finalize new bylaws and new internal rules. They shall be adopted during the next general assembly which will happen during the mini-debconf. Misc Debian Work WordPress maintenance. I mentored Pablo Vasquez to do his first small contribution to the WordPress packaging. I really appreciate this but he s not yet ready to assume maintenance of a big package like WordPress on his own. I got multiple other offers of help and pinged them all while filing #733726 to coordinate the work on the new upstream version. But I got no reply Handing over packages to new maintainers is hard Init system discussion. The technical committee has the hard task of picking the default init system that will replace the traditional System V init (see #727708). I followed this huge discussion closely and contributed a bit where I add something meaningful to say. Final decision is expected sometimes in January. FWIW, I share entirely Russ Allbery s point of view in those discussions. I have been running systemd on some of my computers for a few months already. Fixing lxc in stable. The lxc package in stable has a non-working debian template. I really dislike documenting that things are broken so instead of doing that in the Debian Administrator s Handbook, I opted to do something about it. I prepared a non maintainer upload for stable (see #680469 for the problem and #732358 for the stable update request). Misc stuff. I sponsored a tcpdf upload. I filed an enhancement request on Publican to have it keep processing instructions present in translations. I uploaded new versions of publican-librement and debian-handbook. I filed #732678 against git-buildpackage because it failed to properly call lintian when given the -A dpkg-buildpackage argument. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Following the release of Debian Wheezy, I have (finally) updated my set of Debian QEMU images for both Squeeze (6.0.8) and Wheezy (7.3). The following images are now available:

• amd64 (Squeeze and Wheezy)
• armel (Squeeze and Wheezy)
• armhf (Wheezy)
• i386 (Squeeze and Wheezy)
• mips (Squeeze and Wheezy)
• mipsel (Squeeze and Wheezy)
• powerpc (Squeeze and Wheezy)

Each of these directories contains a GPG signed README.txt file with the md5sums all files, detailed instructions how to run these images and especially the minimum QEMU version to use. The requirements to run the default desktop environment have increased a lot between Squeeze and Wheezy. An accelerated graphics card is now needed to be able to use Gnome (unless you use the fallback mode), which is not something provided by QEMU (actually there is now a QXL para-virtualized video card, but the driver is only in unstable). In addition GDM now needs more than 128MiB to start, while this is the default amount of memory provided for virtual machines. I have therefore decided to switch the default desktop environment on the Wheezy images to Xfce and the display manager to LightDM. Both Gnome and GDM are still installed on the images, and the original default can easily be restored using the following commands:

• update-alternatives --auto x-session-manager
• echo /usr/sbin/gdm3 > /etc/X11/default-display-manager

Beside this the new images only contain minor changes. The filesystems have been tweaked to not run fsck after a certain amount of days, and locales-all and openssh-server are now installed in all images. For MIPS and MIPSEL, 64-bit kernels are now also installed and provided, so that it is possible to choose between a 32-bit or a 64-bit kernel (see the README.txt for more details). There is no Debian Wheezy SPARC image available, as QEMU does not fully support SPARC64 yet (it is actually possible to run it, but then the VM crashes often), and Debian Wheezy now only supports 64-bit kernels. I will also invest time to build an S390X image, but so far I haven t been successful on that. The following images are still available at the same location, though they haven t been updated:

• sparc (Etch)
• SH4 (Sid from a few years ago)

The Ceilometer project supports various database backend that can be used as storage. Among them are MongoDB, SQLite MySQL, PostgreSQL, HBase, DB2 All Ceilometer's code is unit tested, but when dealing with external storage services, one cannot be sure that the code is really working. You could be inserting data with an incorrect SQL statement, or in the wrong table. Only having the real database storage running and being used can tell you that. Over the months, we developed integration testing on top of our unit testing to validate that our storage drivers are able to deal with real world databases. That is not really different from generic integration testing. Integration testing is about plugging all the pieces of your software all together and running. In what I call "database integration testing", the pieces will be both your software and the database system that you are going to rely on. The only difference here is that one of the module is not coming from the application itself but is an external project. The type of database that you use (RDBMS, NoSQL ) does not matter. Taking a step back, what I will describe here could also apply to a lot of other different software modules, even something that would not be a database sytem at all. Writing tests for integration Presumably, your Python application has unit tests. In order to test against a database back-end, you need to write a few specific classes of tests that will use the database subsystem for real. For example:
This code will try to fetch the database URL to use from an environment variable, and then will rely on SQLAlchemy to create a database connection.
You can then add as many tests as you want using the connection stored in self.engine. If no test database URL is, the tests will be skipped; however that decision is up to you. You may want to have these tests always run and fail if they can't be run. In the setUp() method, you may also need to do more work, like create a database and delete a database.
This will make sure that the database you need is clean and ready to be used to testing. Launching modules, a.k.a. databases The main problem we encountered when building integration testing with databases, is to find a way to start them. Most users are used to start them system-wide with some sort of init script, but when running sandboxed tests, that is not really a good option. Browsing the documentation of each storage allowed us to find a way to start them in foreground and control them "interactively" via a shell script. The following is a script that you can use to run Python tests using nose and is heavily inspired by the one we wrote for Ceilometer.
If you use tox to automatize your test run, you can use this scripts (I call it run-test.sh) in your tox.ini file.
Most databases are able to be run in some sort of standalone mode where you can connect to them using a either a Unix domain socket, or a fixed port. Here are the snippet used in Ceilometer to run with MongoDB and MySQL:

The mechanism is always the same. We create a fifo with mkfifo, and then run the database daemon with the output redirected to that fifo. We then read from it until we find a line stating the the database is ready to be used. At that point, we can continue and start running the tests. You have to read continuously from the fifo, otherwise the process writing to it will block. We redirect the output to /dev/null, but you could also redirect it to a different log file, or not at all. One step further: using parallelism and scenarios The described approach is quite simple, as it only support one database type. When using an abstraction layer, such as SQLAlchemy, it would be a good idea to run all these tests against different RDBMS, such as MySQL and PostgreSQL for example. The snippet above allows to run both RDBMS in parallel, but the classic approach of unit tests does not allow that. Using one scenario for each database backend would be a great idea. To that end, you can use the testscenarios library.

To speed up tests run, you could also run the test in parallel. It can be intesting as you'll be able to spread the workload among a lot of different CPUs. However, note that it can require a different database for each test or a locking mechanism to be in place. It's likely that your tests won't be able to work altogether at the same time on only one database. (Both usage of scenarios and parallelism in testing will be covered in The Hacker's Guide to Python, in case you wonder.)

The primary addition in this release is the Autoconf macros and header wrapper that I developed for lbcd to add systemd support. These are inspired by the code in systemd's daemon(7) man page but adjusted for my normal software conventions. This is just preliminary and doesn't yet stub out all of the functions, only the ones I've needed so far. Also in this release, I fixed a bug with process_start_fakeroot, and hence with remctld_start_fakeroot, introduced in the last release when I rewrote the TAP process management add-on. You can get the latest release from the rra-c-util distribution page.