Search Results: "will"

4 July 2013

Julien Danjou: OpenStack meets Lisp: cl-openstack-client

A month ago, a mail hit the OpenStack mailing list entitled "The OpenStack Community Welcomes Developers in All Programming Languages". You may know that OpenStack is essentially built using Python, and therefore it is the reference language for the client libraries implementations. As a Lisp and OpenStack practitioner, I used this excuse to build a challenge for myself: let's prove this point by bringing Lisp into OpenStack!
Welcome cl-openstack-client, the OpenStack client library for Common Lisp! The project is hosted on the classic OpenStack infrastructure for third party project, StackForge. It provides the continuous integration system based on Jenkins and the Gerrit infrastructure used to review contributions. How the tests works OpenStack projects ran a fabulous contribution workflow, which I already talked about, based on tools like Gerrit and Jenkins. OpenStack Python projects are used to run tox, to build a virtual environment and run test inside. We don't have such thing in Common Lisp as far as I know, so I had to build it myself. Fortunately, using Quicklisp, the fabulous equivalent of Python's PyPI, it has been a breeze to set this up. cl-openstack-client just includes a basic shell script that does the following: I just run the test using SBCL, though adding more compiler on the table would be a really good plan in the future, and should be straightforward. You can admire a log from a successful test run done when I proposed a patch via Gerrit, to check what it looks like. Implementation status For the curious, here's an example of how it works:
* (require 'cl-openstack-client)
* (use-package 'cl-keystone-client)
* (defvar k (make-instance 'connection-v2 :username "demo" :password "somepassword" :tenant-name "demo" :url "http://devstack:5000"))

K

* (authenticate k)

((:ISSUED--AT . "2013-07-04T05:59:55.454226")
(:EXPIRES . "2013-07-05T05:59:55Z")
(:ID
. "wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM")
(:TENANT (:DESCRIPTION) (:ENABLED . T)
(:ID . "1774fd545df4400380eb2b4f4985b3be") (:NAME . "demo")))

* (connection-token-id k)

"wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM"

Unfortunately, the implementation is far from being complete. It only implements for now the Keystone token retrieval. I've actually started this project to build an already working starting point. With this, future potential contributors will be able to spend efforts on writing code, and not on setting up the basic continuous integration system or module infrastructure. If you wish to help me and contribute, just follow the OpenStack Gerrit workflow howto or feel free to come by me and ask any question (I'm hanging out on #lisp on Freenode too). See you soon, hopping to bring more Lisp into OpenStack!

EvolvisForge blog: Fun with ssh pubkey auth

Okay, so imagine this: you just generated an SSH RSA key and threw its public part on system B into ~foo/.ssh/authorized_keys and its private part on system A into ~bar/.ssh/id_rsa but can t login. Why? Automated processes (Jenkins *cough*) often need you to ssh(1) manually once, to accept the remote host s server key. Do that. The id_rsa file on system A must be owned by the user bar and chmod 0600 or 0400 (similarily, the .ssh directory has strict permission checks, and everything in the path until there). Check those. And, the most surprising one of the day: if there s an id_rsa.pub it will be used for offering a key to the remote host (B) even if it does not match the secret key. Deleting A:~bar/.ssh/id_rsa.pub apparently makes OpenSSH generate the public part from the secret key each time (or just put the correct pubkey there), but if one s there, it seems to like to use them. (That was the only part of this post that was news to even me, of course ) And, as bottom line: hello to Planet Debian from mirabilos at work , too. I ll occasionally tag posts so they show up here, if I think they re of interest, since I m doing Debian work at the dayjob, too.

Julien Danjou: OpenStack meets Lisp: cl-openstack-client

A month ago, a mail hit the OpenStack mailing list entitled "The OpenStack Community Welcomes Developers in All Programming Languages". You may know that OpenStack is essentially built using Python, and therefore it is the reference language for the client libraries implementations. As a Lisp and OpenStack practitioner, I used this excuse to build a challenge for myself: let's prove this point by bringing Lisp into OpenStack!
Welcome cl-openstack-client, the OpenStack client library for Common Lisp! The project is hosted on the classic OpenStack infrastructure for third party project, StackForge. It provides the continuous integration system based on Jenkins and the Gerrit infrastructure used to review contributions. How the tests works OpenStack projects ran a fabulous contribution workflow, which I already talked about, based on tools like Gerrit and Jenkins. OpenStack Python projects are used to run tox, to build a virtual environment and run test inside. We don't have such thing in Common Lisp as far as I know, so I had to build it myself. Fortunately, using Quicklisp, the fabulous equivalent of Python's PyPI, it has been a breeze to set this up. cl-openstack-client just includes a basic shell script that does the following: I just run the test using SBCL, though adding more compiler on the table would be a really good plan in the future, and should be straightforward. You can admire a log from a successful test run done when I proposed a patch via Gerrit, to check what it looks like. Implementation status For the curious, here's an example of how it works:
* (require 'cl-openstack-client)
* (use-package 'cl-keystone-client)
* (defvar k (make-instance 'connection-v2 :username "demo" :password "somepassword" :tenant-name "demo" :url "http://devstack:5000"))

K

* (authenticate k)

((:ISSUED--AT . "2013-07-04T05:59:55.454226")
(:EXPIRES . "2013-07-05T05:59:55Z")
(:ID
. "wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM")
(:TENANT (:DESCRIPTION) (:ENABLED . T)
(:ID . "1774fd545df4400380eb2b4f4985b3be") (:NAME . "demo")))

* (connection-token-id k)

"wNFQwNzo1OTo1NS40NTQyMthisisaverylongtokenwNFQwNzo1OTo1NS40NTQyM"

Unfortunately, the implementation is far from being complete. It only implements for now the Keystone token retrieval. I've actually started this project to build an already working starting point. With this, future potential contributors will be able to spend efforts on writing code, and not on setting up the basic continuous integration system or module infrastructure. If you wish to help me and contribute, just follow the OpenStack Gerrit workflow howto or feel free to come by me and ask any question (I'm hanging out on #lisp on Freenode too). See you soon, hopping to bring more Lisp into OpenStack!

Petter Reinholdtsen: The Thinkpad is dead, long live the Thinkpad X230

Half a year ago, I reported that I had to find a replacement for my trusty old Thinkpad X41. Unfortunately I did not have much time to spend on it, but today the replacement finally arrived. I ended up picking a Thinkpad X230 with SSD disk (NZDAJMN). I first test installed Debian Edu Wheezy as a roaming workstation, and it worked flawlessly. As I write this, it is installing what I hope will be a more final installation, with a encrypted hard drive to ensure any dope head stealing it end up with an expencive door stop. I had a hard time trying to track down a good laptop, as my most important requirements (robust and with a good keyboard) are never listed in the feature list. But I did get good help from the search feature at <ahref>Prisjakt, which allowed me to limit the list of interesting laptops based on my other requirements. A bit surprising that SSD disk are not disks, so I had to drop number of disks from my search parameters. I am not quite convinced about the keyboard, as it is significantly wider than my old keyboard, and I have to stretch my hand a lot more to reach the edges. But the key response is fairly good and the individual key shape is fairly easy to handle, so I hope I will get used to it. My old X40 was starting to fail, and I really needed a new laptop now. :) I look forward to figuring out how to turn off the touch pad.

3 July 2013

Matthew Garrett: Explaining Intel Rapid Start Technology

Recent Intel-based systems often implement something called Intel Rapid Start Technology. Like many things with the word "Technology" in the name, there's a large part of this that's marketing. The relatively small amount of technical documentation available implies that it's tied to your motherboard chipset and CPU, but as far as I can tell it's entirely implemented in firmware and could work just as well on, say, a Cyrix on a circa 1996 SIS-based motherboard if someone wrote the BIOS code[1]. But since nobody has, we're stuck with the vendors who've met Intel's requirements and licensed the code.

The concept of IRST is pretty simple. There's a firmware mechanism for setting a sleep timeout. If you suspend your computer and this timeout expires, it'll resume. However, instead of handing control back to the OS, the firmware just copies the entire contents of RAM to a special partition and turns the computer off. Next time you hit the power button, the firmware dumps the partition contents back into RAM and resumes as if nothing had changed. This takes a few seconds longer than resume from S3 but is far faster than resume from hibernation since it starts the moment the system gets power.

At a more technical level, it's a little more complicated. The first thing to know about this feature is that it's entirely invisible unless your hard drive is set up correctly. There needs to be a partition that's at least the size of your system's physical RAM. For GPT systems, this needs to have a type GUID of D3BFE2DE-3DAF-11DF-BA-40-E3A556D89593. For MBR systems, you need a partition type of 0x84[2]. If the firmware doesn't find an appropriate partition then the OS will get no indication that the firmware supports it. Boo.

(The second thing is that it seems like it really does have to be on an SSD, and if you try to do this on spinning media your firmware will ignore it anyway)

If all the prerequisites are in place, an ACPI device with an HID of INT3392 will exist. It has four methods associated with it: GFFS, SFFS, GFTV and SFTV. GFFS returns an integer representing the events that will cause the system to wake up from S3 and suspend to SSD. The system will wake after the timeout expires if bit 0 is set, and will wake when the battery becomes critically low if bit 1 is set. The other bits appear to be unused at the moment[3]. SFFS sets the wakeup events, using the same bit values as GFFS. GFTV returns an integer containing the current wakeup timeout in minutes. SFTV sets it. Values above 1440 (ie, 24 hours) seem to be considered invalid - if I set them the value instead ends up as 10 and the timeout flag gets cleared from the wakeup events field.

I've submitted a patch that adds a sysfs interface for setting these values, and unless anyone objects it'll probably end up in 3.11. There's still the remaining question of how userspace should make use of these, and also how installers should behave when it comes to systems that support IRST. As previously mentioned, there's no obvious indication to the OS that the feature is supported unless the appropriate partition already exists. The easiest way to deal with this is for installers to default to retaining any partitions with the magic IDs, but I'm still looking into whether it's possible to get the firmware to cough up some more information so it can be created automatically even if the drive's entirely blank.

And now, having got this working on a test machine, I just need to split my Thinkpad's swap partition in half and make sure it works here as well. Woo.

[1] Note: I am not going to do this.
[2] Conveniently, the same as the partition type that APM systems used for suspend to disk back when dubstep hadn't been invented yet
[3] At least, if you attempt to set them they get ignored.

comment count unavailable comments

Ian Campbell: Emesinae -- Yet Another Bug Tracker

Quite a while ago at a Xen hackathon (I think the Cambridge one in 2011, has it really been 2 years?) the topic of bug tracking came up. The developers who were present (myself included) expressed a general dislike for web based bug trackers, and in particular the existing bugzilla wasn't working for us for a variety of reasons (primarily lack of resource for care and general tending of it). They also liked the existing work flow based around mails to the Xen mailing lists, there was a feeling that this fits well with the demographics of the bugs which get reported to Xen: a sizable number of the issues are configuration errors or misunderstandings which can usually quickly be dealt with through a small email thread (followed by a request to please update the wiki). The project also gets quite a few bug reports which are initially "unactionable" due to of lack logs or a coherent description, in these cases a little bit of back and forth can quickly determine whether the reporter is going to provide the necessary additional information (this is something we are also trying to fix with clearer guidance and an ongoing effort to provide a reportbug-like tool for Xen). However there were concerns that bigger issues which couldn't be dealt with quickly were falling through the cracks. We kicked around a few ideas for how a bug tracker that met our needs might work and that was mostly that. Since then I've kicked the ideas around in my head a bit and searched for an existing system which would fit what was required. In the end I didn't find anything which quite fitted the bill and about six months ago I was looking for an interesting hacking project (perhaps something a bit different to my day job, hypervisor and kernel hacking) so I decided to bite the bullet, dust of my rusty Perl skills and hack something together. Yes, that's right, I've written Yet Another Bug Tracker(tm). I confess I feel a bit dirty for doing so. I've taken a fair bit of inspiration from debbugs (the Debian bug tracker) but I've turned the concept a little on its head. Debbugs is effectively instantiating a new mailing list for each bug report (each NNNNNN@bugs.debian.org can be thought of as a mailing list plus some additional bug specific metadata). Instead I implemented a model which takes an existing mailing list and acts as a list archive which provides a mechanism to track individual threads (or subthreads) as bugs (each bug is effectively a root Message-Id plus a little bit of metadata). In common with debbugs bugs are created and subsequently manipulated by sending mail to a special control@ address. Commands are available to create bugs as well as to graft and prune subthreads onto existing bugs (to cope with people who break threading or offtopic subthreads etc). There is also a read-only web interface for browsing the bugs. Internally the system is subscribed to the list and records details of each message and their relationships (e.g. References headers etc) in a sqlite database. A set of CGI scripts (using the Perl CGI module) provide an interface to browse bugs. Since the system tracks bugs based on mailing list threads I've named it Emesinae which is a "thread-legged bug" (yes, I spent longer than is strictly necessary browsing entomological websites). I've made the code available on gitorious under a GPLv2 license, see https://gitorious.org/emesinae. I announced an alpha instance of Emesinae back in May which tracks the xen-devel mailing list. You can find the bug tracker at http://bugs.xenproject.org/xen/. Since then I've been slowly fleshing out some more useful features (such as tagging of affected branches). It's not seen a great deal of uptake from the Xen developers yet but there are already some bugs being tracked and I'm hopeful that it will prove useful in the long run. In any case it was a fun hacking project for a few rainy evenings (and a couple more train and plane journeys).

2 July 2013

Ond&#345;ej &#268;ert&iacute;k: My impressions from the SciPy 2013 conference

I have attended the SciPy 2013 conference in Austin, Texas. Here are my impressions.

Number one is the fact that the IPython notebook was used by pretty much everyone. I use it a lot myself, but I didn't realize how ubiquitous it has become. It is quickly becoming the standard now. The IPython notebook is using Markdown and in fact it is better than Rest. The way to remember the "[]()" syntax for links is that in regular text you put links into () parentheses, so you do the same in Markdown, and append [] for the text of the link. The other way to remember is that [] feel more serious and thus are used for the text of the link. I stressed several times to +Fernando Perez and +Brian Granger how awesome it would be to have interactive widgets in the notebook. Fortunately that was pretty much preaching to the choir, as that's one of the first things they plan to implement good foundations for and I just can't wait to use that.

It is now clear, that the IPython notebook is the way to store computations that I want to share with other people, or to use it as a "lab notebook" for myself, so that I can remember what exactly I did to obtain the results (for example how exactly I obtained some figures from raw data). In other words --- instead of having sets of scripts and manual bash commands that have to be executed in particular order to do what I want, just use IPython notebook and put everything in there.

Number two is that how big the conference has become since the last time I attended (couple years ago), yet it still has the friendly feeling. Unfortunately, I had to miss a lot of talks, due to scheduling conflicts (there were three parallel sessions), so I look forward to seeing them on video.

+Aaron Meurer and I have done the SymPy tutorial (see the link for videos and other tutorial materials). It's been nice to finally meet +Matthew Rocklin (very active SymPy contributor) in person. He also had an interesting presentation
about symbolic matrices + Lapack code generation. +Jason Moore presented PyDy.
It's been a great pleasure for us to invite +David Li (still a high school student) to attend the conference and give a presentation about his work on sympygamma.com and live.sympy.org.

It was nice to meet the Julia guys, +Jeff Bezanson and +Stefan Karpinski. I contributed the Fortran benchmarks on the Julia's website some time ago, but I had the feeling that a lot of them are quite artificial and not very meaningful. I think Jeff and Stefan confirmed my feeling. Julia seems to have quite interesting type system and multiple dispatch, that SymPy should learn from.

I met the VTK guys +Matthew McCormick and +Pat Marion. One of the keynotes was given by +Will Schroeder from Kitware about publishing. I remember him stressing to manage dependencies well as well as to use BSD like license (as opposed to viral licenses like GPL or LGPL). That opensource has pretty much won (i.e. it is now clear that that is the way to go).

I had great discussions with +Francesc Alted, +Andy Terrel, +Brett Murphy, +Jonathan Rocher, +Eric Jones, +Travis Oliphant, +Mark Wiebe, +Ilan Schnell, +St fan van der Walt, +David Cournapeau, +Anthony Scopatz, +Paul Ivanov, +Michael Droettboom, +Wes McKinney, +Jake Vanderplas, +Kurt Smith, +Aron Ahmadia, +Kyle Mandli, +Benjamin Root and others.


It's also been nice to have a chat with +Jason Vertrees and other guys from Schr dinger.

One other thing that I realized last week at the conference is that pretty much everyone agreed on the fact that NumPy should act as the default way to represent memory (no matter if the array was created in Fortran or other code) and allow manipulations on it. Faster libraries like Blaze or ODIN should then hook themselves up into NumPy using multiple dispatch. Also SymPy would then hook itself up so that it can be used with array operations natively. Currently SymPy does work with NumPy (see our tests for some examples what works), but the solution is a bit fragile (it is not possible to override NumPy behavior, but because NumPy supports general objects, we simply give it SymPy objects and things mostly work).

Similar to this, I would like to create multiple dispatch in SymPy core itself, so that other (faster) libraries for symbolic manipulation can hook themselves up, so that their own (faster) multiplication, expansion or series expansion would get called instead of the SymPy default one implemented in pure Python.

Other blog posts from the conference:

Bits from Debian: all Debian source are belong to us

This is a verbatim repost from Stefano Zacchiroli's post TL;DR: go to http://sources.debian.net and enjoy.
Debsources is a new toy I've been working on at IRILL together with Matthieu Caneill. In essence, debsources is a simple web application that allows to publish an unpacked Debian source mirror on the Web. You can deploy Debsources where you please, but there is a main instance at http://sources.debian.net (sources.d.n for short) that you will probably find interesting. sources.d.n follows closely the Debian archive in two ways:
  1. it is updated 4 times a day to reflect the content of the Debian archive
  2. it contains sources coming from official Debian suites: the usual ones (from oldstable to experimental), *-updates (ex volatile), *-proposed-updates, and *-backports (from Wheezy on)
Via sources.d.n you can therefore browse the content of Debian source packages with usual code viewing features like syntax highlighting. More interestingly, you can search through the source code (of unstable only, though) via integration with http://codesearch.debian.net. You can also use sources.d.n programmatically to query available versions or link to specific lines, with the possibility of adding contextual pop-up messages (example). In fact, you might have stumbled upon sources.d.n already in the past few days, via other popular Debian services where it has already been integrated. In particular: codesearch.d.n now defaults to show results via sources.d.n, and the PTS has grown new "browse source code" hyperlinks that point to it. If you've ideas of other Debian services where sources.d.n should be integrated, please let me know. I find Debsources and sources.d.n already quite useful but, as it often happens, there is still a lot TODO. Obviously, it is all Free Software (released under GNU AGPLv3). Do not hesitate to report new bugs and, better, to submit patches for the outstanding ones. Acknowledgements PS in case you were wondering: at present sources.d.n requires ~381 GB of disk space to hold all uncompressed source packages, plus ~83 GB for the local (compressed) source mirror

Stefano Zacchiroli: introducing sources.debian.net

all Debian source are belong to us TL;DR: go to http://sources.debian.net and enjoy.
Debsources is a new toy I've been working on at IRILL together with Matthieu Caneill. In essence, debsources is a simple web application that allows to publish an unpacked Debian source mirror on the Web. You can deploy Debsources where you please, but there is a main instance at http://sources.debian.net (sources.d.n for short) that you will probably find interesting. sources.d.n follows closely the Debian archive in two ways:
  1. it is updated 4 times a day to reflect the content of the Debian archive
  2. it contains sources coming from official Debian suites: the usual ones (from oldstable to experimental), *-updates (ex volatile), *-proposed-updates, and *-backports (from Wheezy on)
Via sources.d.n you can therefore browse the content of Debian source packages with usual code viewing features like syntax highlighting. More interestingly, you can search through the source code (of unstable only, though) via integration with http://codesearch.debian.net. You can also use sources.d.n programmatically to query available versions or link to specific lines, with the possibility of adding contextual pop-up messages (example). In fact, you might have stumbled upon sources.d.n already in the past few days, via other popular Debian services where it has already been integrated. In particular: codesearch.d.n now defaults to show results via sources.d.n, and the PTS has grown new "browse source code" hyperlinks that point to it. If you've ideas of other Debian services where sources.d.n should be integrated, please let me know. I find Debsources and sources.d.n already quite useful but, as it often happens, there is still a lot TODO. Obviously, it is all Free Software (released under GNU AGPLv3). Do not hesitate to report new bugs and, better, to submit patches for the outstanding ones. Acknowledgements PS in case you were wondering: at present sources.d.n requires ~381 GB of disk space to hold all uncompressed source packages, plus ~83 GB for the local (compressed) source mirror

Lars Wirzenius: Consistency in backups

A backup run takes time. During a backup run the live data may change. This can mean that the snapshot of the live data may not be internally consistent. For example, suppose you're writing an e-mail when the backup starts automatically. If the timing goes wrong in exactly the right way, the backup might capture both the draft e-mail you were writing when the backup started, and the sent e-mail, when you finished the draft and sent the mail before the backup ended. Or, worse, the backup might miss both the draft and the sent e-mail. The next backup run will, certainly, get the sent e-mail, but what if you have a disaster before that happens? Internal consistency of data can be quite difficult to achive. Some programs, such as MySQL, don't do a good job of keeping the data consistent on-disk, making it difficult for the backup program to get a proper setup. (In that case, a standard trick is to "dump" or "export" the data in a consistent form, and back that up instead.) In an ideal situation, when the backup starts, the system would have a way to tell all running programs to store all their state on disk in a recoverable manner (save all open documents, dump data base content, etc), and once that is done, the backup program creates a snapshot of the filesystem. Unfortunately, I don't know of any operating system that can request consistent state from running applications. Snapshotting, however, is easy-peasy. It can be achieved by the filesystem (e.g., btrfs), or using LVM. Doing the snapshots requires root privileges, but that can probably be arranged. However, snapshotting like this, without getting all programs to commit their state to disk first, is not a guarantee of consistency. A thorough way to be consistent is to shut down the computer, and then reboot it into a special backup mode (similar to single-user mode), where no services or applications are running. However, this is, for most people, much too heavyweight a procedure to be worth it. Does consistency really matter? It matters a lot when the live data is being changed actively and consistency of the data is critical. For example, a bank's database of the amount of money in each account is quite critical: if data is not backed up consistently, money can be lost or created spontaneously, and that's probably a bad thing. Most people, however, don't need to worry a lot, and merely need to be aware of the issue, and perhaps mitigate it by backing up when the computer is mostly idle, and run backups often enough. Oh dear. This blog post contains too few jokes. I seem to have run out of jokes to make, so that will conclude this series of blog posts. I hope it was useful to some.

1 July 2013

Michael Stapelberg: Survey answers part 2: the transition

This blog post is the second of a series of posts dealing with the results of the Debian systemd survey. I intend to give a presentation at DebConf 2013, too, so you could either read my posts, or watch the talk, or both :-). It seems that it is unclear how Debian s transition to systemd is intended to work. By transition , we mean going from the current state (sysvinit is the default and fully supported) to systemd is fully supported. Then by merely installing systemd by default and letting it provide /sbin/init, we can make it the default init system. If and when that happens is a different matter and it s not necessary for all packages to have systemd support. sysvinit compatibility systemd natively supports sysvinit scripts, meaning your existing package will work as-is but you cannot utilize all the features that systemd provides. The sysvinit support works very well, as you can try in a fresh Debian wheezy VM. In the output of systemctl list-units , every entry which has an LSB: prefix is actually a sysvinit script. The mechanism with which systemd decides whether to use an init script or a service file is by looking whether a service file with a corresponding name exists. That is, if e.g. apache2.service exists, systemd will prefer it over /etc/init.d/apache2. To make this crystal clear: it is not necessary to ship service files for all services in some kind of flag day. systemd supports a mixed installation where some services use init scripts and some services use service files. Adding systemd support to your package In a nutshell, it usually works like this:
  1. Install a service file to /lib/systemd/system/foo.service.
    Often, upstream already provides and installs a .service file.
    If not, you can place your file at debian/package.service
    Make sure that your service file name corresponds to the sysvinit script name
    (e.g. apache2.service for /etc/init.d/apache2)
  2. Ensure your service file(s) are enabled and started.
    We strongly recommend you to use our package dh-systemd.
    If you use dh(1), add --with=systemd in debian/rules and Build-Dep on dh-systemd
  3. Test your package, see the next section.
For details see wiki.debian.org/Systemd/Packaging. Testing systemd We carefully made sure that you can install the systemd Debian package on your machine alongside sysvinit without breaking anything. The systemd package does not conflict with any other packages, it will not replace /sbin/init and systemd will not be enabled right away. It is only after you specify the kernel parameter init=/bin/systemd in /etc/default/grub that you switch to systemd. In case you want to go back, simply boot without this kernel parameter. Conclusion In conclusion, the transition is straight-forward and the necessary infrastructure is in place. systemd is available in Debian and can be used today. Packages can add systemd support whenever their maintainer(s) feel like it. There is no need for a flag day. We can switch the default whenever we think we are ready.

Russ Allbery: Review: Dark Lies the Island

Review: Dark Lies the Island, by Kevin Barry
Publisher: Greywolf
Copyright: 2012
Printing: 2013
ISBN: 1-55597-651-4
Format: ARC
Pages: 185
This is a literary short story collection by a mainstream Irish writer, rather far from my normal fare. It was the second book in a shipment of Powell's Indiespensable, which is generally (when there is one) a proof or Advanced Reading Copy of something relatively obscure. I'm not really the target audience, not being much of a short story aficionado, but the whole point of doing this Indiespensable experiment is to stretch my reading range. First off, as I would generally expect by mainstream fiction, the quality of the writing is excellent. One thing I particularly like about Barry is that he's refreshingly and unhesitatingly profane. His characters curse like I expect people to curse, some of them at the drop of a hat, and for me it gives a feel of solid realism to his dialogue. He's also very good with eye dialect, giving the impression of accents without making the text hard to read. Some of these stories I liked a great deal, rather more than I had expected. But if there was a general undermining flaw for me, it was that I prefer stories that go somewhere, that have a plot or lead up to a definite point. Some of Barry's do, but a lot of them seem more vignettes or just scenes, sketches of characters, that drift off without resolution. I can admire those as writing exercises, but for me they don't fill the desire I have when I sit down with a story. I'm not sure how useful these reviews will be, given that I don't have a lot of mainstream fiction background to compare them with, but here are some reactions, for whatever they're worth. "Across the Rooftops": This is one of those sketches, but it's one of the best of them. It's a single moment on the rooftops, an attempt to capture the moment of possibility around the first romantic overture. It's sadly quite conventional in gender roles (nervous man initiating, woman as relationship gatekeeper), but apart from that it's very well-written. This kind of thing is normally not to my taste, but Barry does such a great job capturing the sense of uncertainty and the sharp focus on detail that I couldn't help but like it. (7) "Wifey Redux": This is by far my favorite story of this book. I think it's both hilarious and brilliant, right from the starting line.
This is the story of a happy marriage but before you throw up and turn the page let me say that it will end with my face pressed hard into the cold metal of the Volvo's bonnet, my hands cuffed behind my back, and my rights droned into my ear this will occur in the car park of a big-box retail unit on the Naas Road in Dublin.
It's the story of a marriage and how it changes over time, but also the story of being a father and his baffled, flailing reactions to his daughter's sexual explorations. But that's not what makes the story. What makes the story is what leads up to the arrest at the end, which had me laughing out-loud in delight. Barry is great at writing a sort of devilish humor; I wish more of the collection was like this. (9) "Fjord of Killary": This is another one of the ones I enjoyed. It's about an author who buys a hotel (with a bar) on the fjord of Killary thinking that the local culture and contact along with the wild beauty would inspire his poetry. At the start of the story, he's thoroughly sick of both the climate and the people, and the collision between his idyllic dreams and the reality, as well as his acerbic first-person complaints, is quite entertaining. A huge storm and resulting flood provides the conflict and impetus for change, and a few moments of humor in the understated local reactions. I thought the closing epiphany was a bit too obvious and expected, and the ending didn't quite work for me, but it has some good moments. (7) "A Cruelty": Here's where the collection started losing me. This is still well-written and closely observed, but in the case of this story, I wish it weren't, since I found it extremely uncomfortable. The viewpoint character is vulnerable to an encounter of unalloyed nastiness. I'm sure that's the point, but, similar to how I avoid horror, it's the sort of story I'd rather not read. (3) "Beer Trip to Llandudno": This was my second-favorite story of the collection. It's about a group of men who take periodic beer-tasting trips and rate beers. They're just normal men, from a variety of backgrounds and professions, and I thought Barry did a great job capturing the sort of camaraderie that comes from long association with a hobby. That includes, here, the awkward limits on what one talks about and the tendency to use the hobby as a shield from anything that drifts into less certain territory. I thought the story suffered somewhat from not having much of a plot or conclusion, but that's also necessary from the setup. This group of people isn't actually going to solve problems, but will be there for each other in their own way. (8) "Ernestine and Kit": Another very dark story, although that's not quite as obvious at the start. As with "A Cruelty," it's well-written but disturbing and lacks any sort of positive resolution. Here, I think Barry went a bit far into portraying people who are a large part of the popular imagination but who are exceptionally rare in reality. It's the kind of story that, at least for me, plays badly with my brain's threat analysis: too memorable for the level of actual risk, and feeling like it was playing a bit too much with popular boogeymen. (4) "The Mainland Campaign": This one was just too subtle for me. Some of the summaries for this book hint at a meaning for the story that I didn't pick up at all on my first reading. On re-reading, I suspect that analysis is right, but even re-reading the story with that interpretation in mind (this is the sort of story where I'm not sure the average American would pick it up on first reading, but others might), I still don't understand exactly what happened. I think some mainstream writing, and some short stories, tends to err on the side of subtle. (4) "Wistful England": Another one that's way over on the sketch of a moment side of stories. Unlike "Across the Rooftops," it failed to make any impression on me whatsoever; I had to skim it just now to remind myself of any details at all, and by the time I edited this review for posting, I'd forgotten it again. I think it was aiming at capturing a mood, but the mopey viewpoint character didn't mean anything to me. (3) "Doctor Sot": This is another odd one, and I'm not quite sure what I think of it. Either it's another character sketch without much of a plot, or the plot was too subtle. The protagonist is a rather unlikable drunk who doubles as a small town's incompetent doctor. The story involves his encounter with a local sort of hippie encampment, which is moderately interesting even though I couldn't stand the man. Then it leads to an encounter that clearly meant something, but which was entirely lost on me. (4) "The Girls and the Dogs": And from an absence of plot back to the deeply disturbing. This story is about a moderately unlikable drug dealer who gets refuge from an extremely unlikable crazy person in a weird relationship with a woman and her sister. (As is sadly typical, polyamory only makes an appearance when the author wants to paint a scenario that's utterly fucked up and psychotic.) There's a pseudo-fantasy element here in that the characters talk about spells, but I think the only spell involved is authorial fiat to create a deeply broken and sick game that plays into a host of negative stereotypes. Despite myself, I did get drawn into the suspense of the story (Barry's undoubtedly a good writer), but it left me feeling dirty on several levels. (4) "White Hitachi": Finally, another story that I liked, although not as good as the ones earlier in the collection. This one follows a con man and two-bit thief as he gets his brother out of lockup and tries to resolve, or at least stay ahead of, various problems in his life. Again, no real ending, but I thought Barry did a good job at taking a snapshot of a life from a perspective that I don't read much in literature (along with all the sexism and racism that comes along with it). It sort of drifts off rather than ends (although the last sentence is a great bit of characterization), but I found it oddly enjoyable. (6) "Dark Lies the Island": I'm not sure what to make of this. It's another character sketch, a moment of deep psychological significance for a character, floating somewhat alone but described in detail. But the character is a girl struggling with cutting, from a male author whose other stories did not fill me with confidence in his handling of female characters. Barry makes this psychologically believable, but I'm a little nervous about trusting his portrayal of that mindset. It's also, again, just a bit too subtle; I had a hard time getting an emotional feel for how the events of the story connect. (5) "Berlin Arkonaplatz My Lesbian Summer": The final story of the collection is fairly typical of the problem I had with most of it. It seems reasonably well-written, I had a hard time connecting with the protagonist or understanding his motivations. It looks at a slice of life with which I have no familiarity at all (and is interesting for that reason), but I don't quite trust the story enough to submerge in it. Barry seems to write a lot of protagonists who seem aimless, befuddled, or just largely out of control, and I like a bit more agency in the primary character. Silvija is an interesting character, even shown from another's perspective, but the story is almost purely observational: showing her without comment, without obvious motive, and mostly without plot. It's strangely compelling, but not quite compelling enough that I'd actually seek it out. Which, really, is the whole book in a nutshell. (6) Rating: 6 out of 10

Russ Allbery: release 1.46

Just a quick note about a new release of the script I use to publish new software releases. I'm moving more packages to Module::Build, and I noticed I'd been pre-generating release tarballs for Perl modules and then using the release script to do the rest of the work. It supports building a release tarball from the Git repository, though, and by plugging into that machinery I could ensure that I run the test suite with the released code. That's now implemented for the Module::Build build system. Although I do think it's odd that ./Build disttest doesn't actually build a distribution, just acts like it will and then runs a test. You can get the latest version from my miscellaneous scripts page.

Russ Allbery: Pod::Thread 1.01 and faq2html 1.32

Julien LIE pointed out a rendering problem with one of my pages, and some further investigation revealed that the changes that I made to faq2html when moving away from Subversion caused all sorts of problems. I started testing a new version of it, and then found a bunch of other problems with pod2thread. One I already knew about, but others I'd not seen before. Since I was working on this anyway, I decided to fix all the problems I knew about. Pod::Thread 1.01 now correctly encodes its output for UTF-8, which will fix some long-standing problems with ISO 8859-1 characters used to render non-breaking spaces. It also fixes some minor formatting issues and adds a workaround for the inability to format formatting codes in headings when generating an artificial table of contents. faq2html 1.32 now supports multiple subheadings at the top of a page provided that the start of the second subheading is still a centered line. I use this format in a bunch of my package README files, all of which had been misformatted due to not having this support. I did a large test generation of my web pages, and I don't think I added any new problems. (faq2html is inherently tricky, since it's converting plain text to HTML and has to do a bunch of markup guessing. I suppose one of these days I should just give up and switch over to writing README files in Markdown or something.) While I was at it, I also modernized Pod::Thread's module packaging and switched it to Module::Build, which is the direction I'm (slowly) going for all of my Perl module work. The generic parts of the test suite also got a refresh to the latest versions of my generic test suite code (which I'm currently maintaining in rra-c-util for lack of a better location). That had one interesting wrinkle. Pod::Thread was the one Perl distribution where I'd played with using a module filter program (something ExtUtils::MakeMaker supports) to set the module $VERSION automatically during the build. This had the advantage of letting me maintain the distribution version in one place (Makefile.PL) no matter how many Perl modules there are. However, Module::Build doesn't support that. So, this afternoon I wrote Module::Build::SetVersion, which overrides the processing of *.pm files to set the $VERSION string to whatever one sets dist_version to in Build.PL. This lets you just put placeholder strings in all of the modules, and the process of building them into blib will substitute in the current distribution version. Currently, this module is only in this package and subclasses Module::Build (it really should be a mixin of some kind, but I didn't work out a good way of doing that). I'm going to experiment with copying it into my various module packages, but I suppose at some point I should wrap it up in its own distribution, add a test suite, and release it properly. (If you would find this useful, drop me email and I'll prioritize releasing it properly on CPAN somewhat higher.) Note that you can do the same thing with the existing Module::Build::PM_Filter module, as well as a bunch of other things, but that requires you write and maintain a Perl script at the top level of the distribution that does the desired transformations and seemed unnecessarily complex for what I was trying to do. I really wanted something I could just drop in and forget about. You can get the latest releases of Pod::Thread and faq2html from my web tools distribution page.

30 June 2013

Russell Coker: Links June 2013

Cory Doctorow published a letter from a 14yo who had just read his novel Homeland [1]. I haven t had anything insightful to say about Aaron Swartz, so I think that this link will do [2]. Seth Godin gave an interesting TED talk about leading tribes [3]. I think everyone who is active in the FOSS community should watch this talk. Ron Garrett wrote an interesting post about the risk of being hit by a dinosaur killer [4]. We really need to do something about this and the cost of defending against asteroids is almost nothing compared to defence spending. Afra Raymond gave an interesting TED talk about corruption [5]. He focussed on his country Trinidad and Tobago but the lessons apply everywhere. Wikihouse is an interesting project that is based around sharing designs for houses that can be implemented using CNC milling machines [6]. It seems to be at the early stages but it has a lot of potential to change the building industry. Here is a TED blog post summarising Dan Pallotta s TED talk about fundraising for nonprofits [7]. His key point is that moral objections to advertising for charities significantly reduce their ability to raise funds and impacts the charitable mission. I don t entirely agree with his talk which is very positive towards spending on promotion but I think that he makes some good points which people should consider. Here is a TED blog post summarising Peter Singer s TED talk about effective altruism [8]. His focus seems to be on ways of cheaply making a significant difference which doesn t seem to agree with Dan Pallotta s ideas. Patton Oswalt wrote an insightful article about the culture of stand-up comedians which starts with joke stealing and heckling and ends with the issue of rape jokes [9]. Karen Eng wrote an interesting TED blog post about Anthony Vipin s invention of HAPTIC shoes for blind people [10]. The vibration of the shoes tells the person which way to walk and a computer sees obstacles that need to be avoided. David Blaine gave an interesting TED talk about how he prepared for a stunt of holding his breath for 17 minutes [11].

29 June 2013

Steve Kemp: So I have a new desktop..

So I have a new desktop computer. I installed Wheezy on it via a USB stick, and everything worked. All the hardware. Yay. I guess we take it for granted when things like sound, disks, and network cards just work these days. I remmeber fighting with distros in the past, where such things were not necessarily straightforward. The only minor complication is the graphics card. I bought a cheap/random GeForce card for the new machine ( 30):
$ lspci -nn   grep VGA
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF119 [GeForce GT 610] [10de:104a] (rev a1)
Booting up I get a working X.org and GNOME 3.x, but the open graphics driver is "too bad" so I get fallback GNOME; with "Applications" & "Places" menus. Installing the proprietry driver gave me a full GNOME 3.x experience. But I didn't like it so for the moment I'm running: The plan was to install awesome, or similar, but I'm just a creature of habit and I'm still cloning git/mercurial repos and selectively restoring backups. My old desktop has been given to my partner to replace the EeeeeePC she's been using for the past year. I'll fettle over the weekend until I'm back up and running properly; but for the moment I'm good. All my videos/music are ported across. I can print, and I have access to the repos I'm currently working on. (Mostly lumail which will have a new release over the weekend.)

Russ Allbery: remctl 3.5

remctl is the middleware layer that we use all over the place at Stanford. It's a simple GSS-API client/server application that provides a very low-weight way of running a specific command using GSS-API authentication and strong ACL checking. The primary point of this release is to fix a very long-standing occasional data corruption issue. Due to the way that the remctl server checked for the completion of a command, a command that produced a lot of output immediately before exiting could have the end of its output cut off when sending it back to the client. That's fixed in this release. Also in this release, I finally figured out the thread-safe way of specifying to the GSS-API layer which Kerberos ticket cache to use for authentication. This means that the remctl_set_ccache API is now thread-safe if there were Kerberos libraries available during the remctl build and gss_krb5_import_cred was supported. I also fixed the way that the Perl modules are versioned, which resulted in a one-time move backwards of the module versions from Perl's perspective (from 3.4 to 3.05). Hopefully that won't be a problem for anyone. The Perl module version will match the package version with an adjustment so that it sorts properly as a number from here on. You can get the latest version from the remctl distribution page.

28 June 2013

Daniel Leidert: Repository key expired VII

The repository-key expiration date has been extended recently. The expiration date of the renewed key has been set to be October 3rd, 2013. You will have to update the key in your apt keyring by either using (e.g.) the pgp.mit.edu keyserver:

apt-key adv --keyserver subkeys.pgp.net --refresh-keys E394D996
or a local keyfile in ASCII format at wgdd_archive_key.asc

wget -O - http://debian.wgdd.de/stuff/wgdd_archive_key.asc apt-key add -
or by updating the wgdd-archive-keyring Debian package. Detailed information can also be found here.

Vincent Bernat: lan o: a task launcher powered by cgroups

A few months ago, I was looking for a piece of software to spawn long-running tasks on behalf of some daemon with the benefit of the tasks not being interrupted when this daemon is restarted. Here are the specifications: The last requirement explains why tasks are not spawned directly by the daemon requesting them: its complexity or the way it needs to be operated may need frequent restarts. Even if it is possible to re-execute a daemon while keeping its children, like stateful re-exec support in Upstart, this is quite difficult: internal state should be serialized and restored. Part of this state can be contained into a third-party library. While Upstart or systemd seemed to be good candidates for this purpose, I didn t find a straightforward way to run arbitrary tasks without any privilege1. Here comes lan o2. It is a very simple task launcher. It can run any tasks, stop them and check if they are still running. It leverages cgroups in recent Linux kernels and avoids the use of any daemon.

Quick tour Before looking at how lan o works, let s have a look at how it can be used. To avoid usage conflicts, each task is run in the context of a namespace that needs to be initialized:
$ sudo lanco testns init -u $(id -un) -g $(id -gn)
This is the only command that needs to be run as root. Subsequent ones can be run as a normal user. Let s run some task:
$ lanco testns run first-task openssl speed aes
$ lanco testns check first-task && echo "Still running"
Still running
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes 
  
The output of the task is logged into a file:
$ head -3 /var/log/lanco-testns/task-first-task.log
Doing aes-128 cbc for 3s on 16 size blocks: 8678442 aes-128 cbc's in 2.85s
Doing aes-128 cbc for 3s on 64 size blocks: 2478283 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 256 size blocks: 628105 aes-128 cbc's in 3.00s
If the task is too long, we can kill it:
$ lanco testns run first-task openssl speed aes
$ lanco testns stop first-task
You cannot run a task that already exists or kill a task that does not exist:
$ lanco testns run first-task openssl speed aes
2013-06-09T22:50:34 [WARN/run] task first-task is already running
$ lanco testns stop second-task
2013-06-09T22:50:45 [WARN/stop] task second-task is not running
Thanks to the use of cgroups, lan o is able to track multiple processes even when they are forking away3:
$ lanco testns run second-task sh -c \
>  "while true; do (sleep 30 &)& sleep 1; done"
$ lanco testns ls
testns
   first-task
      28456 openssl speed aes 
   second-task
      29572 sh -c while true; do (sleep 30 &)& sleep 1; done 
      29575 sleep 30 
      29593 sleep 30 
      29596 sleep 30 
      29599 sleep 30 
      29622 sleep 30 
      29644 sleep 1 
      29645 sleep 30 
  
$ lanco testns stop second-task
$ lanco testns check second-task   echo "Killed!"
Killed!
Also, there is a top-like command (lanco testns top): lanco top

Using cgroups Control groups (cgroups) is a mechanism to partition a set of tasks and their future children into hierarchical groups with a set of parameters.

Hierarchy Let s start with the hierarchical stuff first. To create a new hierarchy, you have to mount the cgroup filesystem on some empty directory. Usually, cgroups hierarchies are setup in /sys/fs/cgroups which is a tmpfs filesystem:
# mount -t tmpfs tmpfs /sys/fs/cgroup -o nosuid,nodev,noexec,relatime,mode=755
Now, we can create our first hierarchy:
# cd /sys/fs/cgroup
# mkdir my-first-hierarchy
# mount -t cgroup cgroup my-first-hierarchy -o name=my-first-hierarchy,none
# ls -1 my-first-hierarchy
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
release_agent
tasks
We ll see later the purpose of none. The most interesting file is tasks. It contains the PIDs of all processes in our group. Since there are currently no sub-group, all processes are part of it. Let s create a sub-group and attach a process to it:
# mkdir first-child
# cd first-child
# ls -1
cgroup.clone_children
cgroup.event_control
cgroup.procs
notify_on_release
tasks
# echo $$ > tasks
# cat tasks
23184
23311
# cat /proc/$$/cgroup 
9:name=my-first-hierarchy:/first-child
8:perf_event:/
7:blkio:/
6:net_cls:/
5:freezer:/
4:devices:/
3:cpuacct,cpu:/
2:cpuset:/
1:name=systemd:/user/bernat/1
We have added our shell to the new cgroup. Moreover, all its children will also be part of this group. This explains why we have two tasks: the shell and cat. The last command is quite interesting: for each hierarchy, it shows which cgroup the task belongs to. For a given hierarchy, each task is exactly in one group. The most useful features of lan o are done using just this: a namespace is a named hierarchy and each task is enclosed into a cgroup so it can be tracked properly.

Subsystems A subsystem makes use of the task grouping provided by cgroups to assign resources to a set of tasks. Each available subsystem can only be in one hierarchy, so the usual way to setup things is to assign a hierarchy for each subsystem4. Let s have a look at the cpuset subsystem: it allows to assign tasks to specific CPUs and memory nodes (for NUMA systems). Let s suppose we have 4 cores and we want to assign the first core to common system tasks and the three remaining ones to nginx:
# cd /sys/fs/cgroup
# mkdir cpuset
# mount -t cgroup cgroup cpuset -o cpuset
# echo 0-3 > cpuset/cpuset.cpus
# echo 0 > cpuset/cpuset.mems
# mkdir cpuset/system
# echo 0 > cpuset/system/cpuset.cpus
# echo 0 > cpuset/system/cpuset.mems
# for task in $(cat cpuset/tasks); do
>    echo $task > cpuset/system/tasks
> done
# mkdir cpuset/nginx
# echo 1-3 > cpuset/nginx/cpuset.cpus
# echo 0 > cpuset/nginx/cpuset.mems
# for task in $(pidof nginx); do
>    echo $task > cpuset/nginx/tasks
> done
The first mount is not needed if something else (like systemd) has setup the appropriate subsystem. Now, even if some system process is going crazy, it won t affect the performance of your webserver. The kernel comes with a more complete documentation if you need additional details on cgroups. There are high level tools to manipulate them, like the tools provided by libcg, but they are currently quite buggy and invasive.

Use in lan o Here are the features used by lan o:
  • For each namespace, a specific named hierarchy is created.
  • By setting the appropriate permissions on the hierarchy, an unprivileged user can create subgroups.
  • Each submitted task has its own sub-group for tracking purpose.
  • Actions can be executed when a task terminates by using the release agent mechanism5.
  • The cpuacct subsystem is used to track CPU usage: a group is created in this subsystem for this purpose.

  1. Both systemd and Upstart are difficult to run without being PID 1 and without being root. They both support user sessions but this requires to use them as PID 1 as well. I discovered later than runit would have been a good fit: it does not require to be PID 1, it does not need to be run as root and the service directory can be specified by an environment variable. It could have been tweaked to meet the above requirements.
  2. Lan o means launcher in esperanto. The unusual diacritic on the letter c is not present in many web fonts, so the rendering may be a little odd.
  3. Linux cgroups do not provide any facility to kill a whole group. We need to iterate through the processes and kill them one by one. lan o does not try to freeze processes before killing them and therefore may be inefficient against violent fork bombs.
  4. The cpu and cpuacct subsystems are the notable exception. There seems to be little use to keep them in separate hierarchies, so they are part of the same hierarchy.
  5. Unfortunately, only one global release agent per hierarchy can be used. Since lan o provides with the ability to execute an arbitrary command when a task terminates, the command needs to be stored on the filesystem for the release agent of the namespace to execute it.

Niels Thykier: Optimizing autodie

After converting Lintian to using autodie, I was a bit trouble by the effects it had on Lintian s (start-up) performance. This overhead accumulated to at least a 1s when checking packages. Indeed, I am not the first to complain about this problem. For those who have not heard of it, autodie is a Perl pragma that basically alters subs and built-ins from a check return value for errors -sub (e.g. like C) to throws exception on error -sub (e.g. like Python). So, you can replace open( ) or die with open( ) when autodie is in effect. autodie achieves this by exporting a little sub that wraps the target sub (or built-in). This little wrapper is compiled[1] into the importing package. The problem with this solution is that it ends up compiling these subs once per importing package. Secondly, all of these wrappers are wrapped in something called a leak guard . As autodie is a lexical pragma it is not permitted to leak across file boundaries. The leak guards take care of that by doing a runtime check of the caller and reverting to the original sub/built-in if a leak occurred. These leak guards were also compiled into the importing package. It is probably not much of a surprise to anyone that autodie spent quite a bit of its runtime compiling these wrappers and their leak guards. Together with upstream, I have addressed some of these problems. The first optimization was to allow some wrappers for built-ins to be reused that does not take glob or bareword arguments[2]. Secondly, we started using closures to generate the leak guards. Combined these two effectively cut the load time in half for subsequent loads of autodie, though it did have a negative effect on the call overhead for these wrappers (~1s for 1M calls). That landed in autodie v2.18. The extra runtime overhead of v2.18 was basically caused by the leak guard being split into two (a small closure, which called the real leak guard with an extra argument). We managed to merge these two parts into one without (too much) loss of code maintainability. With that, the call overhead of these wrappers dropped by ~3s (for 1M calls) and thus the wrappers in autodie v2.20 were even faster than the ones in v2.17. Beyond these improvements, I have sent upstream several patches (still pending review) that will improve the performance of autodie even further. If merged, we are looking at another 1.5s (per 1M calls) removed and almost a factor 4 reduction on the load time of autodie. This factor 4 comes from lazily compiling the wrapper on demand, so some of the price is moved to the call overhead. That said, it is a net gain since it is highly unlikely that every module using autodie will use every single wrapped built-in (how often do you use both read and sysread in the same module)? [1] It is compiled via a call to eval using some template code. [2] For the rest of the built-ins, it could probably just use Symbol::qualify to qualify bareword filehandles. Perhaps that will come in a future version of autodie. :)

Next.

Previous.