Coverity is a nice tool to find problems in C, C++ and Java code using static source code analysis. It can detect a lot of different problems, and is very useful to find memory and locking bugs in the error handling part of the source. The company behind it provide check of free software projects as a community service, and many hundred free software projects are already checked. A few days ago I decided to have a closer look at the Coverity system, and discovered that the gnash and ipmitool projects I am involved with was already registered. But these are fairly big, and I would also like to have a small and easy project to check, and decided to request checking of the chrpath project. It was added to the checker and discovered seven potential defects. Six of these were real, mostly resource "leak" when the program detected an error. Nothing serious, as the resources would be released a fraction of a second later when the program exited because of the error, but it is nice to do it right in case the source of the program some time in the future end up in a library. Having fixed all defects and added a mailing list for the chrpath developers, I decided it was time to publish a new release. These are the release notes: New in 0.16 released 2014-01-14:

• Fixed all minor bugs discovered by Coverity.
• Updated config.sub and config.guess from the GNU project.
• Mention new project mailing list in the documentation.

You can download the new version 0.16 from alioth. Please let us know via the Alioth project if something is wrong with the new release. The test suite did not discover any old errors, so if you find a new one, please also include a test suite check.

Liberation vs expectations I feel like I found a key struggle: liberation vs expectations. Many ideas liberate me, like poliamory, atheism, anarchism, nonviolent communication, sexual liberation, discordianism, free software, nonjudgemental environments, E-prime, sillyness, love, pataphysics, dada, constructivism, rolling naked in the mud, fantasy, bdsm exploration, pansexuality, this very idea that i'm trying to describe, crying, laughing, mindfulness, what have you. Sometimes however they become an end rather than a mean, they give me expectations and judgements, and take some other freedom away. I feel like I found where to draw a line: It's the point inbetween freeing myself of my limits and making myself some new ones. It's one of those epiphany moments. Weeeeeee!

Today (well, technically yesterday by now), the Media were reporting of an open letter by '5 belgian university professors from the IT field' without calling them by name about how Belgian primary education should introduce programming into the curriculum, starting at age 5. Unsurprisingly, this did not go well with a majority of the Belgian populace. Arguments like "what should the school drop to make place for that, then" and "why would people ever need to learn programming" were rampant on some internet fora that I had a short look at. That being said, though, I think these five unnamed professors are correct. Understanding how a tool works is essential to proficient usage of said tool; and while it is possible to teach particular common and popular use cases of a given tool without providing enough background, doing so will only result in confusion when the computer "is acting up" an oxymoron if there ever was one. After all, a computer can't "act up"; it can only follow instructions. Whenever your seems to be "acting up", what's really happening is that someone (most likely, you) gave it the wrong instructions. Or, perhaps, someone evil on the Internet gave it the right instructions (for their goals, anyway) and infected your computer with a virus. But that's unlikely. Computer education currently mostly consists of "teaching software": rather than explaining how a computer does what it does, people are taught that if you click this button, the text will be bold, or if you click that button, the printer will start buzzing. That's all nice and dandy, until the actual computer people encounter when off the school bench and at work has this other piece of software, where all the buttons are in other places and look all "wrong". Because then they get all confused about the fact that things aren't where they're supposed to be. So, yes, it is my opinion that any computer education should help people understand how a computer does the things it does. And what better way to do that than to actually explain things to a computer, the way computer programmers do such things? Note that programming a computer doesn't have to be hard, and it can be something which kids can understand and will find fun to do. For instance, there's this scratch thing from MIT, which was made for teaching programming in such a context. Yes, adding programming to the primary school curriculum will require that some other things are dropped instead, and I could see that being a problem. However, in our modern world computers have become so pervasively important that we would be doing our kids a disservice not to explain them how to understand computers, as opposed to just learn how to use them...

cjac@foxtrot:/usr/src/deb/strongswan-5.1.0$ ping6 -c 5 google.com tail -3
google.com ping statistics
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 65.235/74.957/103.039/14.122 ms go go gadget ipsec

I have written a number of scripts to create Debian foreign architecture (mostly armel and armhf) rootfs images for SD cards or NAND flashing. I started with putting Debian on my Openmoko gta01 and gta02 and continued with devices like the qi nanonote, a marvel kirkwood based device, the Always Innovating Touchbook (close to the Beagleboard), the Notion Ink Adam and most recently the Golden Delicious gta04. Once it has been manufactured, I will surely also get my hands dirty with the Neo900 whose creators are currently looking for potential donors/customers to increase the size of the first batch and get the price per unit further down. Creating a Debian rootfs disk image for all these devices basically follows the same steps:

1. create an disk image file, partition it, format the partitions and mount the / partition into a directory
2. use debootstrap or multistrap to extract a selection of armel or armhf packages into the directory
3. copy over /usr/bin/qemu-arm-static for qemu user mode emulation
4. chroot into the directory to execute package maintainer scripts with dpkg --configure -a
5. copy the disk image onto the sd card

It was not long until I started wondering why I had to run all of the above steps with superuser privileges even though everything except the final step (which I will not cover here) was in principle nothing else than writing some magic bytes to files I had write access to (the disk image file) in some more or less fancy ways. So I tried using fakeroot+fakechroot and after some initial troubles I managed to build a foreign architecture rootfs without needing root priviliges for steps two, three and four. I wrote about my solution which still included some workarounds in another article here. These workarounds were soon not needed anymore as upstream fixed the outstanding issues. As a result I wrote the polystrap tool which combines multistrap, fakeroot, fakechroot and qemu user mode emulation. Recently I managed to integrate proot support in a separate branch of polystrap. Last year I got the LEGO ev3 robot for christmas and since it runs Linux I also wanted to put Debian on it by following the instructions given by the ev3dev project. Even though ev3dev calls itself a "distribution" it only deviates from pure Debian by its kernel, some configuration options and its initial package selection. Otherwise it's vanilla Debian. The project also supplies some multistrap based scripts which create the rootfs and then partition and populate an SD card. All of this is of course done as the superuser. While the creation of the file/directory structure of the foreign Debian armel rootfs can by now easily be done without superuser priviliges by running multistrap under fakeroot/fakechroot/proot, creating the SD card image still seems to be a bit more tricky. While it is no problem to write a partition table to a regular file, it turned out to be tricky to mount these partition because tools like kpartx and losetup require superuser permissions. Tools like mkfs.ext3 and fuse-ext2 which otherwise would be able to work on a regular file without superuser privileges do not seem to allow to specify the required offsets that the partitions have within the disk image. With fuseloop there exists a tool which allows to "loop-mount" parts of a file in userspace to a new file and thus allows tools like mkfs.ext3 and fuse-ext2 to work as they normally do. But fuseloop is not packaged for Debian yet and thus also not in the current Debian stable. An obvious workaround would be to create and fill each partition in a separate file and concatenate them together. But why do I have to write my data twice just because I do not want to become the superuser? Even worse: because parted refuses to write a partition table to a file which is too small to hold the specified partitions, one spends twice the disk space of the final image: the image with the partition table plus the image with the main partition's content. So lets summarize: a bootable foreign architecture SD card disk image is nothing else than a regular file representing the contents of the SD card as a block device. This disk image is created in my home directory and given enough free disk space there is nothing stopping me from writing any possible permutation of bits to that file. Obviously I'm interested in a permutation representing a valid partition table and file systems with sensible content. Why do I need superuser privileges to generate such a sensible permutation of bits? Gladly it seems that the (at least in my opinion) hardest part of faking chroot and executing foreign architecture package maintainer scripts is already possible without superuser privileges by using fakeroot and fakechroot or proot together with qemu user mode emulation. But then there is still the blocker of creating the disk image itself through some user mode loop mounting of a filesystem occupying a virtual "partition" in the disk image. Why has all this only become available so very recently and still requires a number of workarounds to fully work in userspace? There exists a surprising amount of scripts which wrap debootstrap/multistrap. Most of them require superuser privileges. Does everybody just accept that they have to put a sudo in front of every invocation and hope for the best? While this might be okay for well tested code like debootstrap and multistrap the countless wrapper scripts might accidentally (be it a bug in the code or a typo in the given command line arguments) write to your primary hard disk instead of your SD card. Such behavior can easily be mitigated by not executing any such script with superuser privileges in the first place. Operations like loop mounting affect the whole system. Why do I have to touch anything outside of my home directory (/dev/loop in this case) to populate a file in it with some meaningful bits? Virtualization is no option because every virtualization solution again requires root privileges. One might argue that a number of solutions just require some initial setup by root to then later be used by a regular user (for example /etc/fstab configuration or the schroot approach). But then again: why do I have to write anything outside of my home directory (even if it is only once) to be able to write something meaningful to a file in it? The latter approach also does not work if one cannot become root in the first place or is limited by a virtualized environment. Imagine you are trying to build a Debian rootfs on a machine where you just have a regular user account. Or a situation I was recently in: I had a virtual server which denied me operations like loop mounting. Given all these downsides, why is it still so common to just assume that one is able and willing to use sudo and be done with it in most cases? I really wonder why technologies like fakeroot and fakechroot have only been developed this late. Has this problem not been around since the earliest days of Linux/Unix? Am I missing something and rambling around for nothing? Is this idea a lost cause or something that is worth spending time and energy on to extend and fix the required tools?

Valve Software s Steam is the number one digital game distribution service, with more than 65 million registered accounts. Steam runs on Windows, Mac OS, and Linux x86/amd64 computers, and provides access to several thousand games, at varying price points an enormous growth from less than a dozen games for Windows only about a decade ago. Valve s latest endeavour has been to bring their storefront into the living room, with a three-pronged attack: a new game controller, a programme of licensed x86 consoles to plug into the family TV, and an OS to run on the Steam Machines to tie it all together. December saw the first public release of their SteamOS which, as it turns out, is basically just a preconfigured desktop Linux. Specifically, it s a Debian Wheezy derivative, comprising a subset of 502 source packages from Wheezy; 8 of Valve s own source packages; and 51 source packages which have been either patched compared to Wheezy, backported from post-Wheezy, or both. For example, the compiler used by default is gcc-4.7 (rather than Wheezy s 4.6) and the libc version postdates Wheezy too. Valve s official instructions and installer release concerned quite a few people who had planned to try SteamOS on an older PC, by mandating a large (500GB) hard disk and a PC with UEFI firmware. Very quickly a number of instructions started appearing from people trying to fix what people felt were real issues specifically provision for BIOS-based computers, and installation from optical media. After being assured that redistribution of derivatives of SteamOS were entirely authorized (and, in fact, encouraged) I decided to produce my own variant, calling upon my own experience with debian-installer modification from past and present jobs, as well as calling upon the skills and experience of the UK Debian community as needed. The end result is Ye Olde SteamOSe. This weekend saw the third release of Ye Olde SteamOSe, a derivative designed to greatly widen the pool of computers capable of running Valve s OS. And unlike the first two releases, the public response this time has been crazy. Like, totally crazy. Combined score across several subreddits totals about 1700. 7 pages of Google search results. Hundreds of tweets. Mentions in dozens of blogs around the world. Coverage on the Linux Action Show. Unilaterally added to Softpedia s list of distributions. Almost 2000 views of a video installation walkthrough I posted on YouTube. Crazy. Sadly the idea to try and track visitors to the page didn t occur to me until long after the initial rush subsided, but 1000 visitors on Tuesday for a news story which landed on Sunday is still pretty hot in my book. It s also interesting to observe the demographics of site visitors the #1 referrer on Tuesday was Dutch PC site Tweakers.net, and StumbleUpon outranks reddit for referrals. About 66% of visitors interested in installing my Debian Wheezy derivative derivative came using Windows (20% using Linux), which suggests there s a lot of potential Linux users out amongst the Windows gamer masses. So what s the purpose of this self-congratulatory blog post? Just dick-waving? Well, there s an element of that (I m only human), but I think it might be nice to alert the audience on Planet Debian/Ubuntu, many of whom are not big gamers, to the next big thing in embedded Linux except this time a real GNU/X.org/sysVinit distribution, not some NIH thing like Android. So now you know!

Packt Publishing sent me Krzysztof Niki ski's Instant Haml to do a short blog review... Thing is, and those who read me know, I cannot write short reviews ;-) So lets see how this goes. Packt's Instant series First, as for the format this book follows and the series it fits in: They are very short books. I was the tech reviewer for their Instant Debian Build a Web Server , and when I opened it for the first time, I thought I had received only the first chapter. This series' motto is Short Fast Focused so, yes, the books are ~50 pages long, with ample margins and many big screenshots. I told the editors I was quite unsure about the value this kind of publications add, as they are very similar to online tutorials, but given they have so many books in this series (printed and electronic), there must be a market for them. They surely know their business better than me! There is one other detail regarding the publisher I'll stick in this section, although it is not specific to their Instant series: I do most of my reading in my Kindle. Packt seems to have chosen to specify a very small font for their texts. Besides the two books mentioned, I also have their Object Oriented JavaScript book. For all of the other books I read, I use the smallest font setting. Packt books in mobi format (what they present as Amazon format in their web site) are basically illegible without bumping up the font size through the roof. As I understand it, publishers should use industry-wide default sizes, and let the reader adjust their reading experience. Yes, books of technical nature (such as Packt's) often benefit from having more displayed information than what an ebook reader normally shows... But that's for me to decide! And no, this is not a Kindle-specific glitch: I also get a similar experience when opening it from Calibre. And as for the book... I am a long-time Haml convert/fan; I packaged its Ruby implementation for Debian (and should update it soon :-/ as new versions have appeared), the corresponding Emacs Lisp mode, and I use it for basically all of my static HTML sites and Ruby-based Web systems. Haml is a great templating language, meant to make dynamic HTML generation leaner and easier. It is a clean and elegant technology, and can clearly sell itself. And the project webpage has very good documentation, ranging from a short tutorial and a full API documentation for people to contribute to the project. The first part of this book, of course, covers the "why is Haml so great?" part, presenting the basic equivalences with HTML. In my opinion, the book could be structured a bit differently and would generate a stronger following. Why? Because, even given that Haml is greatly benefited from its Ruby integration and has many hooks to be tightly integrated in a Rails workflow, it can also be used by people who don't want to do Ruby. I have some static websites that are just a bunch of Haml files, compiled by a very simple Makefile and calling the haml command from the shell. I think there would be some value in presenting this invocation before learning how to call it from the Gems environment (i.e. bundle install). Using Haml as a standalone tool is only presented almost as an appendix ("Using HAML outside of Rails" section). Even more so if the author presents four different static-site solutions (probably more non-programmer-friendly than my trusty Makefile). Oh, and this little section uses the (old? obsolete?) uppercase HAML notation for the project's name instead of the now-standard Haml. Dirty screenshots About the screenshots presented in this first section (and I refer mainly to the scaffolding examples), they are meant to show how easy things are, but have an important stylistic mistake: Terminals with transparency might look cool (I hate using them, but then again I'm a very boring person when it comes to my computer habits ;-) ), but transparencies have no place in a book (be it printed or e-book), as the images appear very uncrisp. In the Kindle, they just seemed dirty Now that I'm browsing the PDF from my desktop, I see that the screenshot in "Step 1" has a photo of Earth from space behind the terminal, and even worse, in "Step 2" it is shown over a Web page detailing the installation! These elements are just distracting, and the text would be much cleaerer if it were presented typeset just as a regular code fragment. Opening many of those screenshots in the Kindle was plain useless. Editor integration It is perfectly clear from this text that the author strongly prefers Sublime Text, sometimes uses TextMate, and has just heard reports that Haml has some support in Vim, Rubymine, Emacs and Coda. Of course, devoting a full page (as it is for the first case) for each editor would be out of proportion for this size of book... I just felt the imbalance too big, specially given that most editors will sport approximately the same feature set. Text redundancy If the Instant series is about making short books, I would expect them to be loaded with easy to follow content (as they are), but free of repetition. I felt, however, a large chunk of step 1 of the "Quick start" chapter to be repeated with the the first section of the "Top 6 features" chapter. Yes, the second one adds some depth, but very little. Still, in the later parts of this "Top 6 features" chapter, a more thorough explanation could be used for the Filters, Multiline attributes/code/strings and helpers and extensions sections. It seems the author was on a hurry trying to get it finished, as it is barely explained Explaining a bit more deeply would surely benefit readers. Wrapping up Reading my review, I'm mostly talking about negative issues. IMO, the book is not excellent, but it is clearly not bad. Specially given it fits very well the format for which it was designed. There are many items that could be fixed, and I hope some of them can be fixed (i.e. the screenshots) before the book goes to print If I understand Packt's page correctly, it is currently only available as ebook, contrary to most of the Instant series' titles.

Qt and KDE will be present at FOSDEM, the largest open-source event in Europe. One more year, we will be sharing the Desktops DevRoom with Gnome, Unity, Enlightenment, LXQt and Hawaii (a Qt Quick desktop environment). We recently published the schedule for the devroom, which will be also available in the printed booklet available at the front desk. For the 2014 edition, the FOSDEM organization wants to achieve 100% recording of presentations. That means every presentation, in every room (devroom, lightning talk, main conference, etc) must be recorded. That s hundreds of talks. While the FOSDEM and devrooms organization teams comprise a lot of people, we are far too busy already with the organizative stuff and cannot spend time doing the actual recordings. Good thing is, you can help! Do you want to join the FOSDEM Video Team and receive the t-shirt? We are now looking for volunteer cameramen (and camerawomen, of course ). FOSDEM will provide you with equipment and training, you only need to start recording, focus, make sure nobody gets between the camera and the speaker/stage, etc. You do NOT need to record the whole track, even one talk recording would help. More details on what will be required from you are available in this e-mail from Wouter. Please contact me (pgquiles at elpauer dot org) if you are interested in recording one or more presentations from the Desktops DevRoom.

Ok, so I'm back in Mexico! This year, the best fare I found for travelling to spend the Winter^WSummer season with Regina's family had an oddity: I usually have a layover at either Santiago de Chile or Lima (Per ) of between 45 minutes and 2 hours, clearly less than enough to do anything. But this time, I had a massive 10 hours layover in Santiago. And spending 10 hours in an airport is far from fun. Specially when you have a good group of friends in town! I visited Chile in 2004 for Encuentro Linux (still before the time I had a digital camera: Those photos are all taken by Martin Michlmayr), and I have stayed in touch with a group of systems administrators since then. So, I mailed the list, and we managed to get eight people to have lunch together. In the order we appear in the photo:

• Victor Hugo dos Santos
• Mauricio Troncoso
• lvaro Herrera
• Ricardo Lemus
• Marcelo Riquelme
• Carlos Sep lveda
• Gunnar Wolf
• Pablo Silva

Some of them, even living in the same city, had never met in person before So, of course, we had a table reserved at the restaurant to the name of Dennis Ritchie. And having had nice, fun, sometimes-technical talks... Well, a tiny bit of his spirit was there. Of course, we can only trust he was there, as no Ouija boards were used and no null pointers were dereferenced (just to make sure not to disturb him). Victor Hugo and lvaro took me for a short Santiago city trip before lunch, we had a very nice time. Thanks! :-)

remctl is the middleware layer that we use everywhere at Stanford. It's a simple GSS-API-authenticated network service that supports running commands with ACLs. There are client bindings available for a wide variety of programming languages. This release fixes a couple of irritating bugs: the client library leaked memory when remctl_set_ccache was used (which was affecting mod_webkdc from WebAuth), and Net::Remctl::Backend didn't validate argument counts correctly when one of the arguments came from standard input (which affected krb5-sync). I also worked around a bug in RHEL 5's Module::Build, and added sanity checking to Net::Remctl and related classes to ensure that the object argument wasn't undef. Also new in this release are support in the remctld server for systemd startup notification and socket activation, and (via the -Z) flag) support for upstart's expect stop synchronization method. This is mostly a "clearing the decks" release in advance of more significant work. The next release will replace the server event loop with libevent, in preparation for further improvements in how the server can handle persistent worker children. You can get the latest release from the remctl distribution page.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (147.56 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. The Debian Administrator s Handbook I spent a good chunk of December on the book. First finalizing the English version and getting it out (BTW, just for the launch, there s a 10% discount on the paperback that lasts only until January 9th!). Then working on updating the French translation. Eyrolles will publish a new edition of the French book based on this translation. Expect some further news about this during January! Debian France I contributed to many discussions within Debian France. Starting with a complaint that most events are organized in Paris, I proposed to map the location of Debian France members. We added new fields in the membership management page so that members can add their GPS coordinates and Fr d ric Decou made some experiments with Openstreetmap. Someone else (Kiriarat) volunteered to write the required glue code. A manual map is currently maintained on the website. In the discussions about the setup of the Debian France shop, I suggested to update our logo with a nicer looking one. We got a few suggestions and after further discussions with Alexandre Delano and Sylvestre Ledru, we organized a small contest to entice designers to submit a logo proposal to us (the winner earns a set of Debian goodies). We got 46 proposals (see my favorite on the right)! The board is currently pre-selecting the logos and setting up the final vote for our members. The winner shall be announced at the end of the upcoming mini-debconf in Paris. I also continued the work to finalize new bylaws and new internal rules. They shall be adopted during the next general assembly which will happen during the mini-debconf. Misc Debian Work WordPress maintenance. I mentored Pablo Vasquez to do his first small contribution to the WordPress packaging. I really appreciate this but he s not yet ready to assume maintenance of a big package like WordPress on his own. I got multiple other offers of help and pinged them all while filing #733726 to coordinate the work on the new upstream version. But I got no reply Handing over packages to new maintainers is hard Init system discussion. The technical committee has the hard task of picking the default init system that will replace the traditional System V init (see #727708). I followed this huge discussion closely and contributed a bit where I add something meaningful to say. Final decision is expected sometimes in January. FWIW, I share entirely Russ Allbery s point of view in those discussions. I have been running systemd on some of my computers for a few months already. Fixing lxc in stable. The lxc package in stable has a non-working debian template. I really dislike documenting that things are broken so instead of doing that in the Debian Administrator s Handbook, I opted to do something about it. I prepared a non maintainer upload for stable (see #680469 for the problem and #732358 for the stable update request). Misc stuff. I sponsored a tcpdf upload. I filed an enhancement request on Publican to have it keep processing instructions present in translations. I uploaded new versions of publican-librement and debian-handbook. I filed #732678 against git-buildpackage because it failed to properly call lintian when given the -A dpkg-buildpackage argument. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Following the release of Debian Wheezy, I have (finally) updated my set of Debian QEMU images for both Squeeze (6.0.8) and Wheezy (7.3). The following images are now available:

• amd64 (Squeeze and Wheezy)
• armel (Squeeze and Wheezy)
• armhf (Wheezy)
• i386 (Squeeze and Wheezy)
• mips (Squeeze and Wheezy)
• mipsel (Squeeze and Wheezy)
• powerpc (Squeeze and Wheezy)

Each of these directories contains a GPG signed README.txt file with the md5sums all files, detailed instructions how to run these images and especially the minimum QEMU version to use. The requirements to run the default desktop environment have increased a lot between Squeeze and Wheezy. An accelerated graphics card is now needed to be able to use Gnome (unless you use the fallback mode), which is not something provided by QEMU (actually there is now a QXL para-virtualized video card, but the driver is only in unstable). In addition GDM now needs more than 128MiB to start, while this is the default amount of memory provided for virtual machines. I have therefore decided to switch the default desktop environment on the Wheezy images to Xfce and the display manager to LightDM. Both Gnome and GDM are still installed on the images, and the original default can easily be restored using the following commands:

• update-alternatives --auto x-session-manager
• echo /usr/sbin/gdm3 > /etc/X11/default-display-manager

Beside this the new images only contain minor changes. The filesystems have been tweaked to not run fsck after a certain amount of days, and locales-all and openssh-server are now installed in all images. For MIPS and MIPSEL, 64-bit kernels are now also installed and provided, so that it is possible to choose between a 32-bit or a 64-bit kernel (see the README.txt for more details). There is no Debian Wheezy SPARC image available, as QEMU does not fully support SPARC64 yet (it is actually possible to run it, but then the VM crashes often), and Debian Wheezy now only supports 64-bit kernels. I will also invest time to build an S390X image, but so far I haven t been successful on that. The following images are still available at the same location, though they haven t been updated:

• sparc (Etch)
• SH4 (Sid from a few years ago)

The Ceilometer project supports various database backend that can be used as storage. Among them are MongoDB, SQLite MySQL, PostgreSQL, HBase, DB2 All Ceilometer's code is unit tested, but when dealing with external storage services, one cannot be sure that the code is really working. You could be inserting data with an incorrect SQL statement, or in the wrong table. Only having the real database storage running and being used can tell you that. Over the months, we developed integration testing on top of our unit testing to validate that our storage drivers are able to deal with real world databases. That is not really different from generic integration testing. Integration testing is about plugging all the pieces of your software all together and running. In what I call "database integration testing", the pieces will be both your software and the database system that you are going to rely on. The only difference here is that one of the module is not coming from the application itself but is an external project. The type of database that you use (RDBMS, NoSQL ) does not matter. Taking a step back, what I will describe here could also apply to a lot of other different software modules, even something that would not be a database sytem at all. Writing tests for integration Presumably, your Python application has unit tests. In order to test against a database back-end, you need to write a few specific classes of tests that will use the database subsystem for real. For example:
This code will try to fetch the database URL to use from an environment variable, and then will rely on SQLAlchemy to create a database connection.
You can then add as many tests as you want using the connection stored in self.engine. If no test database URL is, the tests will be skipped; however that decision is up to you. You may want to have these tests always run and fail if they can't be run. In the setUp() method, you may also need to do more work, like create a database and delete a database.
This will make sure that the database you need is clean and ready to be used to testing. Launching modules, a.k.a. databases The main problem we encountered when building integration testing with databases, is to find a way to start them. Most users are used to start them system-wide with some sort of init script, but when running sandboxed tests, that is not really a good option. Browsing the documentation of each storage allowed us to find a way to start them in foreground and control them "interactively" via a shell script. The following is a script that you can use to run Python tests using nose and is heavily inspired by the one we wrote for Ceilometer.
If you use tox to automatize your test run, you can use this scripts (I call it run-test.sh) in your tox.ini file.
Most databases are able to be run in some sort of standalone mode where you can connect to them using a either a Unix domain socket, or a fixed port. Here are the snippet used in Ceilometer to run with MongoDB and MySQL:

The mechanism is always the same. We create a fifo with mkfifo, and then run the database daemon with the output redirected to that fifo. We then read from it until we find a line stating the the database is ready to be used. At that point, we can continue and start running the tests. You have to read continuously from the fifo, otherwise the process writing to it will block. We redirect the output to /dev/null, but you could also redirect it to a different log file, or not at all. One step further: using parallelism and scenarios The described approach is quite simple, as it only support one database type. When using an abstraction layer, such as SQLAlchemy, it would be a good idea to run all these tests against different RDBMS, such as MySQL and PostgreSQL for example. The snippet above allows to run both RDBMS in parallel, but the classic approach of unit tests does not allow that. Using one scenario for each database backend would be a great idea. To that end, you can use the testscenarios library.

To speed up tests run, you could also run the test in parallel. It can be intesting as you'll be able to spread the workload among a lot of different CPUs. However, note that it can require a different database for each test or a locking mechanism to be in place. It's likely that your tests won't be able to work altogether at the same time on only one database. (Both usage of scenarios and parallelism in testing will be covered in The Hacker's Guide to Python, in case you wonder.)

Recently I wrote about docker, after a brief diversion into using runit for service management, I then wrote about it some more. I'm currently setting up a new PXE-boot environment which uses docker for serving DHCP and TFTPD, which is my first "real" usage of any note. It is fun, although I now discover I'm not alone in using docker for this purpose. Otherwise life is good, and my blog-spam detection service recently broke through the 11 million-rejected-comment barrier. The Wordpress Plugin is seeing a fair amount of use, which is encouraging - but more reviews would be nice ;) I could write about work, I've not done that since changing job, but I'm waiting for something disruptive to happen first.. ObQuote: Dune. (film)

Dear World, On the morning of December 24th I felt an unusual pain in my left hand between the thumb and forefinger. The pain increased and in the afternoon I got a high fever, at some point above 40 degrees Celsius or 104 degree Fahrenheit. I went to the emergency department and was hospitalized during the night between the 24th and 25th of December. On the afternoon of December 26th I underwent surgery to find out what was happening, and was then diagnosed with Necrotizing Fasciitis (the wikipedia article on NF gives a fair summary), caused by the common streptococcus bacteria (again see wikipedia article on Streptococcus). A popular name for the disease is flesh-eating bacteria. Necrotizing Fasciitis is a rare and aggresive infection, often deadly if left untreated, that can move through the body at speeds of a couple of centimeters per hour. I have gone through 6 surgeries, leaving wounds all over my left hand and arm. I have felt afraid of what the disease will do to me, anxiety over what will happen in the future, confusion and uncertainty about how a disease like this can exist and that I get the right treatment since so little appears to be known about it. The feeling of loneliness and that nobody is helping, or even can help, has also been present. I have experienced pain. Even though pain is something I m less afraid of (I have a back problem) compared to other feelings, I needed help from several pain killers. I ve received normal Paracetamol, stronger NSAID s (e.g., Ketorolac/Toradol), several Opiate pain-killers including Alfentanil/Rapifen, Tramadol/Tradolan, OxyContin/OxyNorm, and Morphine. After the first and second surgery, nothing helped and I was still screaming with pain and kicking the bed. After the first surgery, I received a local anesthetic (a plexus block). After the second surgery, the doctors did not want to masquerade my pain, because sign of pain indicate further growth of the infection, and I was given the pain-dissociative drug Ketamine/Ketalar and the stress-releasing Clonidine/Catapresan. Once the third surgery removed all of the infection, pain went down, and I experienced many positive feelings. I am very grateful to be alive. I felt a strong sense of inner power when I started to fight back against the decease. I find joy in even the simplest of things, like being able to drink water or seeing trees outside the window. I cried out of happiness when I saw our children s room full of toys. I have learned many things about the human body, and I am curious by nature so I look forward to learn more. I hope to be able to draw strength from this incident, to help me prioritize better in my life. My loving wife sa has gone through a nightmare as a consequence of my diagnosis. At day she had to cope with daily life taking care of our wonderful 1-year old daughter Ingrid and 3-year old boy Alfred. All three of them had various degrees of strep throat with fever, caused by the same bacteria and anyone with young kids know how intense that alone can be. She gave me strength over the phone. She kept friends and relatives up to date about what happened, with the phone ringing all the time. She worked to get information out from the hospital about my status, sometimes being rudely treated and just being hanged up on. After a call with the doctor after the third surgery, when the infection had spread from the hand to within 5cm of my torso, she started to plan for a life without me. My last operation were on Thursday January 2nd and I left hospital the same day. I m writing this on the Saturday of January 4rd, although some details and external links have been added after that. I have regained access to my arm and hand and doing rehab to regain muscle control, while my body is healing. I m doing relaxation exercises to control pain and relax muscles, and took the last strong drug yesterday. Currently I take antibiotics (more precisely Clindamycin/Dalacin) and the common Paracetamol-based pain-killer Alvedon together with on-demand use of an also common NSAID containing Ibuprofen (Ipren). My wife and I were even out at a restaurant tonight. Fortunately I was healthy when this started, and with bi-weekly training sessions for the last 2 years I was physically at my strongest peak in my 38 year old life (weighting 78kg or 170lb, height 182cm or 6 feet). I started working out to improve back issues, increase strength, and prepare for getting older. Exercise has never been my thing although I think it is fun to run medium distances (up to 10km). I want thank everyone who helped me and our family through this, both professionally and personally, but I don t know where to start. You know who you are. You are the reason I m alive. Naturally, I want to focus on getting well and spend time with my family now. I don t yet know to what extent I will recover, but the prognosis is good. Don t expect anything from me in the communities and organization that I m active in (e.g., GNU, Debian, IETF, Yubico). I will come back as energy, time and priorities permits.

I'm not a strong swimmer, and never have been. I was never confident in the water as a child. In my teenage years, I had some skin issues that I was very self-conscious about. Going to a boy's school where I was already bullied, I didn't want to add material, so I avoided swimming. To this day, I can barely swim 50 metres freestyle. To think I once had aspirations of getting into the police, which required a bronze medallion. As as result of my atrocious swimming abilities, it's been very important to me that Zoe be able to swim well. Purely for her own personal safety if nothing more. She's been doing "swim classes" of various sorts since she's been 6 months old. In the US, they were group parent/child water familiarisation classes. When we moved back to Australia and turned three, she was eligible to start doing small-group instructor-led classes with Hampton Swim School. It felt like such an milestone just to no longer have to be in the pool with Zoe. I was initially quite skeptical of Hampton's methods, compared with what I'd observed at the Betty Wright Swim School (which subsequently had to close due to aging facilities). Having seen how well Zoe's swimming has progressed in 3 quarters of classes is just amazing. The last few times I've taken Zoe to a public pool for some non-class swim time, her confidence in the water has been fantastic. I've bought some "sinkies", toys that sink to the bottom, and she'll happily dive down to the bottom to retrieve them (with assistance from me getting down there). Today, to escape the 40 plus degree heat, I took her to the Sleeman Sports Centre, because they have a really cool slide, and the whole thing's indoors. Zoe was confidently swimming between me and the side of the pool in water that was over her head, repeatedly. So to have her go from being fine in the water as long as she was being held, to actively asking me to go further back from the wall, in under a year has been really excellent progress. She's also "diving" in off the side (it's more like a bad belly flop, but she has brilliant form at the start, with her hands over her head). We got to the pool today around 10:30am, and it turned out the slide didn't open until 1pm, so we decided to stick around for lunch and wait until the slide opened. To kill time after lunch, we went to the 50 metre pool, where some other kids were taking turns to jump in off the starting blocks. Fearless Zoe wanted to have a go too. This was seriously deep water. It was really inspiring watching this tiny little girl climb up onto the starting blocks of the 1982 Commonwealth Games 50 metre pool and "dive" into the water, flounder over to me, who was furiously treading water in the middle of the lane, and then flounder over to the side, climb out unassisted and do it all over again. It was a seriously "proud father" moment, while I was trying not to drown myself in the middle of the lane. I wish I'd had someone there with a camera to capture the moment. I have great hopes for the future of her swimming, and I think she's ready to go up a level in her swim class.

A new release of the BH package is now on CRAN and its mirrors. BH provides (a sizeable subset of) the Boost library for C++, particularly the (large) parts delivered as pure template headers not requiring linking. See the BH page for more details. This release comes from a rewritten build script for the package. We no longer drive the initial set of Boost libraries by the requirements of the bigmemory (and synchronicity) packages, but rather explicitly enumerate an equivalent set of Boost libraries. This ensures that these libraries are included completely. We also move the project repository to this GitHub repo. The complete list changes follows below.

Changes in version 1.51.0-4 (2014-01-01)

• Rewritten main package creation script to no longer scan for what bigmemory and synchronicity use, but rather explicitly copy over an (equivalent) list of explicitly-enumerated Boost libraries
• Repository moved from R-Forge to GitHub, scripts and layout adjusted accordingly
• Besides the implicitly expanded coverage by including the complete libraries, we also expanded from math/distributions to all of math.

Courtesy of CRANberries, there is also a diffstat report for the most recent release. Comments and suggestions are welcome via the mailing list or issue tracker still available via the package page at R-Forge.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Implementing software to replace legacy systems is always a challenge, especially when you're dealing with a system with as much legacy as sendmail, which was first introduced as delivermail in 1979.^ref

Each UNIX vendor, it seems, rewrote or heavily customized sendmail. This has lead to sometimes conflicting implementations.
Case in point: -tNormally, you invoke sendmail(8) with a series of arguments indicating the subject of a message, the recipients, etc. When invoked this way, the command expects a message on standard input, waits for EOF, and then sends your message along.

However, sometimes you don't want to have to fiddle with command-line parameters; you've already written a perfectly fine message with headers.

-t is generally passed to sendmail when you want to build a message envelope from an already-formatted message, with headers, etc. For example, if you had a file foo.txt with a body like this:

From: Luke Faraone 
To: John Smith 
Subject: Hello, world!

Hi there.

you could send the message with a simple invocation of cat foo.txt sendmail -t. The system would take care of ensuring a Message-id was appended if appropriate, and queue the message to be sent. However, it is when you do slightly more complex invocations of sendmail that things get ambiguous.

It turns out that implementations differ on what exactly it means when you use -t in combination with naming destination addresses after the arguments to sendmail. exim4's documentation describes the situation in greater detail:

Thus, there's basically no mechanism for a program to know which behaviour to expect. God forbid two programs are installed on a system that expect different behaviours!

It appears that the default behaviour of Ruby is the opposite of what exim4 (Debian's default mail client) expects. This has resulted in numerous bug reports. Some replies suggest changing exim4's defaults, while others advocate overriding ActionMailer and friends to use sendmail -i instead, without -t.

That said, its not really clear who's wrong here; at no point does there appear to have been a definitive specification for sendmail, and as such we can hope for defined behaviour by common custom at best, and a sea of incompatibility bugs at worst. Amusingly, POSIX standards have nothing to say on this subject of sendmail at all; it defines that a mailx command must exist, but says that its sending mode may be implementation-specific.

As Matthew Garrett writes, there's not enough gin in the world.