This is a bug-fix release to my POD translator package, which contains Pod::Man and Pod::Text and is used by Perl core to format POD documentation. This is not, alas, the long-awaited overhaul of UTF-8 handling; that's still coming at some point in the future. The primary changes in this release are various performance patches from Dagfinn Ilmari Manns ker from work on profiling all the Perl core modules. I learned from merging those packages that reversing a long string can make regex replacement operations on the tail faster, even with the time required to reverse the string twice. I'm surprised. This release also wraps the Pod::Simple parse_lines and parse_string_document to set the default output to STDOUT, similar to what was already done with parse_file, since some of the one-liners that people were testing would have produced confusing error messages without this. It fixes handling of POD documents with error sections and no other content, thanks to a bug report by Andreas Koenig. And it fixes typos and other documentation issues, thanks to Andreas, David Steinbrunner, and Brian Gottreu. You can get the latest release from the podlators distribution page.

Review: Throne of the Crescent Moon, by Saladin Ahmed

Series:	Crescent Moon Kingdoms #1
Publisher:	DAW
Copyright:	January 2013
ISBN:	0-7564-0778-8
Format:	Mass market
Pages:	367

Doctor Adoulla Makhslood is the last ghul hunter in Dhamsawaat. He's an old man, and tired, and increasingly uncertain whether he wants to continue putting his life in danger to fight one more fight. But there are no replacements; his young and idealistic dervish apprentice, despite obvious skill, is too inexperienced and uninterested in the knowledge and investigation that makes Adoulla so good at what he does. So, when a young boy shows up on his doorstep, a survivor of a vicious ghul attack, he goes hunting once again. And both finds something much darker and stronger than he expected and meets an unexpected ally: a shape-shifting girl who is herself a survivor of something far worse than ghuls. Throne of the Crescent Moon is a first-novel fantasy that has made quite a splash, including a Hugo nomination (although I thought a win was unlikely, and I was right). One of the more interesting comments I've read about it was the observation by Jo Walton that if it had come out in 1980, it would have been an amazing, unusually wonderful, non-normative book. That it isn't is evidence of the changes in fantasy since then. The field has grown. As you might guess from the publisher's insistence on comparing it with The Arabian Nights, and from the names mentioned above, this book is set in an unspecified Middle Eastern culture and draws heavily from its mythology. It does not contain any western characters; the entire story is native to its region. The same is true of the mythology, which gave it a refreshing originality for me. And while I'm not the best one to judge, the presentation of that setting seemed free of the tint of the exotic. People live here, there are factions and multiple cultures, and the supernatural is a backdrop to most of the city. This would be very unusual decades ago. It's a bit less unusual now, and the story itself is a familiar ensemble fantasy, if refreshingly free of extended map exploration journeys. Ahmed brings together five characters with very different backgrounds and throws their varying talents at a problem. A few of those elements are a bit too familiar: female sexuality as a threat to male power is a world-building element that I could do without, for example. I also found the villain completely inadequate in its unidimensional evil (reinforced with a few more torture scenes than I needed to read), and even the attempted complication felt too predictable to me. Except for one touch of perspective, I'd call Throne of the Crescent Moon a competent, enjoyable, but somewhat forgettable fantasy novel, albeit one that's refreshingly concerned with a city instead of countrysides and travel. That touch is in the age of several of the protagonists. We get sections following several of the main characters in tight third person, but Adoulla is the first protagonist and the focus of much of the book. And Adoulla is neither young nor inexperienced, and is not going through the typical fantasy bildungsroman. He's old, experienced, a bit set in his ways, cranky, cantankerous, and deeply in love with his city. In tone, this shifts the story towards Star Wars as seen through the eyes of Obi-Wan Kenobi instead of Luke Skywalker. And that change I really liked, and would be happy to see become more common in fantasy. There was a bit of a downside for me in three of the five main characters being older. At times, Throne of the Crescent Moon has an elegiac tone, a feeling of life, power, and time slipping away and cultures and ways of life being lost. That's not my favorite thematic element. I tend to find it more depressing than I really want in my reading. But, here, the mature perspective more than makes up for some occasional negativity. I particularly liked how the older characters treated the likely budding young romance, which would have been the center of many books in this genre, with bemused detachment and playful encouragement without giving it more significance than it deserves. There's another romance that ends up being more significant, and which needs the weight of time and well-lived lives to become so. I don't think this book deserves a Hugo, but it is an enjoyable fantasy with some interesting characters and a refreshing perspective and setting. The villain really isn't adequate to the story, there's a bit too much torture for my liking, and I could have done without a few of the tropes, but it's a solid entry in its genre and a pleasant few hours of reading. And despite being advertised as the first of a series, it comes to a satisfying conclusion. Rating: 7 out of 10

On a regular basis, I find myself the odd-man-out when it comes to talking about how to work with Python on Debian systems. I'm going to write this and post it so that I might be able to point people at my thoughts without having to write the same email in response to each thread that pops up. Turns out I don't fit in with the Debian hardliners (which is to say, the mindset that pip sucks and shouldn't exist), nor do I fit in with the Python hardliners (which is to say apt and dpkg are out of date, and neither have a place on a Development machine). I think our discourse on this topic has become petty and stupid in general. Let's all try to step back and drop a bit of the attitude. pip doesn't suck, and neither does apt. The truth is, both sides are wrong. As with any subject, the real answer here is much more nuanced than either side presents it. I'm going to try and present my opinion on this, in the way that both my Pythonista self and my Debianite self see the issue. Hopefully I can keep this short, to the point, and caked with logic. The case for dpkg (the Debianite in me) In defense of dpkg and apt, imagine having to install python-gnome2 on all your systems when you install. It'd be hell on earth. Imagine having a user try to do this. It's insane to assume that end-users will be using pip for this purpose. pip is fun and all, but it's also installing 100% untrusted code to your system (perhaps as root, if you're using pip with sudo for some reason), and hasn't been reviewed for software freeness, which is something Debian (and Debian users) take seriously. This isn't even to mention the hell that pip wreaks on dpkg controlled files / packages. Try to remember how much of your system running (yes, right now) is running because of Python or Python modules. Try to imagine how much of a pain in the ass it'd be if you couldn't boot into GNOME to use nm-applet to connect to wifi to pip install something. I'm sure even the most extreme pip'er understands the need for Operating System level package management. Debian also has a bigger problem scope - we're not maintaining a library in Debian for kicks, we're maintaining it so that end user applications may use the library. When we update something like Django, we have to make sure that we don't break anything using it (although, to be honest, the fact that we package webapps is an entire rant for later) before we get to update it to the newest release. Hell, with a few coffees, I could automate the process of releasing a .deb with a new upstream release, 100% unattended. I won't, however, since this is an insane idea. Let's go over a brief list of things I do before uploading a new package:

1. Review the entire codebase for simple mistakes.
2. Review the entire codebase for license issues.
3. Review the entire codebase for files without source, and track down (and include source for) any sourceless files (such as pickle files, etc).
4. Get to know the upstream, get to know open bugs, write something using the lib, in case I need to debug later.
5. Install the package.
6. Test the package.
7. Work out any Debian package issues (this is easy).

Now, a brief list of things I do before I update a package:

1. Review the changes between the last uploaded version (in diff format, if it's sane, otherwise get the VCS and review), ensure all the above are still OK.
2. Review for Debian-local issues (such as how it will upgrade, using piuparts, and adequate, etc).
3. Check to make sure it won't break any reverse dependencies.
4. Review for bugfixes that I might need to bring back to the stable release.
5. Figure out if I should (or even can) backport the package, if API is stable.
6. Review for bugs (upstream or in Debian) that I need to mention in the debian/changelog.

Clearly, this isn't a quick-and-dirty task. It's not a matter of getting a package updated (technically), it's a much more detailed process than that. This is also why Debian is so highly regarded for its technical virtuosity, and why the ISS decided to deploy Debian in space, despite other commercial distros such as Red Hat, or Ubuntu, and community distros, such as Fedora or Arch. It's also not Debian's job to package the world in the archive. This is an insane task, and it's not Debian's place to do it. We introduce libraries as things need them, not because we wrote some new library that someone may find slightly useful at some point in the future. maybe. Upstream developers and language communities (not only Python here) tend to lose sight of why we're doing this in the first place, which is our users. This isn't some sort of technical pissing contest to see who can distribute the software in the best way. Debian-folk always keep end users as our highest priority. I quote the Debian Social Contract, when I say that Our priorities are our users and free software. No one's trying to get developers to use dpkg to create software. In fact, as you'll see below, I actively discourage using system modules for development. The case for pip (the Pythonista in me) In defense of pip, the idea that Debian will keep the latest versions of packages is insane. The idea that we can keep pace with upstream releases is nuts, and the idea that every upstream release on pypi is ready to ship is bananas. b-a-n-a-n-a-annas. As a developer, I don't want to support every release, and I surely don't want other people depending on some random snapshot. Often times, I'll put stuff up on pypi as a preview, or to release often, and solicit feedback without having to give out instructions on using a git checkout (it's also easier to have them try a version from pypi so I can cross-ref the git tag to reproduce issues when they file them) pypi is easy, ubiquitous and works regardless of the platform, which means less of my development time is spent packaging stuff up for platforms I don't really care about (Arch, Fedora, OSX, Windows), even though I value feedback from users on those systems. The effort it takes to release something is limited to python setup.py sdist upload, and it's in a place (and in a shape) that anyone can use it without having 10 sets of platform-local instructions. Even ignoring all the above, when I'm writing a new app or bit of code, I want to be sure I'm targeting the latest version of the code I depend on, so that future changes to API won't hit me as hard. By following along with my dependencies' development, I can be sure that my code breaks early, and breaks in development, not production. Upstreams also tend to not like bug reports against old branches, so ensuring I have the latest code from pypi means I can properly file bugs. Lastly, I prefer virtualenv based setups for development, since I'm usually working on many things at once. This often means version mismatches in libraries, which brings in API changes (another whole rant here as well). I don't want to keep installing and uninstalling packages to switch between the two projects, and using a chroot(8) means a lot of overhead and that it's disconnected from my development environment / filesystem, so I resort to virtualenv to isolate my Development environment. Final notes I don't want to keep arguing about this. Just accept that the world's a big place and that there exist use-cases that both apt and pip need to exist and work in the way they're working now. At the very least, try and understand there exist smart people on both sides, and no one is trying to screw anyone over or keep their own little private club to themselves. Hopefully, going forward, we can make sure that the integration between these two tools gets better, not worse. Help make this dream a reality. Contribute to a productive tone, not a destructive one. In short:

• Use pip without sudo always. Don't tell people to use sudo.
• Use apt or dpkg when deploying system-wide.
• Understand people are going to package, and they will be more concerned about software using your library then keeping your library up to date.
• Understand Debian Developers and package maintainers have to do a lot of work when updating or sponsoring an upload.
• Understand upstream developers can't be bothered to fix every issue with every release (release early, release often) with some snapshot you introduced into unstable.
• Use pip and virtualenv in development setups, so we can upgrade your app when we upgrade the lib.

I have just bought a wireless phone charging system based on the Qi Inductive Power Standard. I bought a charging device which connects to a standard micro-USB cable and receivers for the Samsung Galaxy Note 2 and Samsung Galaxy S3 phones I own. Both those phones have contacts in the back of their case that are designed for wireless charging so you can install a charging device inside them. The charging devices make the case fit a little tight, and the charging device is stuck to the phone battery with contact adhesive, this makes it impractical to change the battery on a phone with such a device and makes it a little more difficult to swap out a battery case. One nice feature of the Nexus 4 is that it has Qi charging built in, that saved me $19 and was also more convenient. I believe that the main advantage of a wireless charger is to avoid the risk of damage to the phone if it s dropped while connected to a USB charger. This allows the phone to be charged in situations where you might need to quickly or regularly unplug it to go somewhere. One example of how I might use it is when working at an office so I could charge my phone while at my desk and then quickly take it with me if I had to go to a meeting (sadly I have worked in many offices where they have so many meetings). Another example is for sysadmin work where I have to frequently visit devices to fix them. The wireless charging mat that I bought from Kogan connects to a standard micro-USB plug, the good thing about this is that it s easy to find cables and it can take power from any PC. The bad thing about this is that the resistance of the USB cable is a factor that limits the power that a phone can receive, when using wireless charging you have the limit of the cable resistance as well as some power loss from the wireless transmission. After any extended period of charging the charging mat feels warm to the touch and the phone that s been resting on it feels warmer than usual. The warmth is an indication of energy loss which means longer charging times, a longer charging time isn t necessarily a problem as the convenience of wireless charging can allow longer charging times, but if you want to charge your phone in a hurry before you go somewhere then wireless isn t a good choice. In the past I ve discovered that the battery in a Samsung Galaxy S3 can t be charged if the phone is at 46C [1]. 46C might seem extremely hot to people in some parts of the world (EG northern Europe and Canada) but the temperature in even southern parts of mainland Australia can get that hot and it can be hotter in central and northern parts, so phone temperature can be a real issue. Currently my house is at 21C according to a digital thermometer, the Galaxy S3 and the Note 2 are being charged from USB and report temperatures of 27C and 23C respectively. While the thermometer in my house and those in the phones probably aren t really accurate it seems reasonable to assume that the battery of a relatively idle smart-phone that s being charged will be a few degrees warmer than the ambient temperature. The Qi charger makes things a lot worse as it even feels warm to the touch. So maybe a phone on a Qi charger would be 8 degrees warmer than the ambient temperature or more. That implies that in Australian summer weather a Qi charger won t be useful outside or in any building that lacks air-conditioning. So I think we can give up on the idea of using Qi devices to charge phones at a BBQ. The final problem I have is that the Qi device is quite small, I took the above picture with my phone face-down because no part of the charger is visible in normal use. With that size I can t just dump a phone like a Note 2 on top of the charging mat and expect it to work. I have to carefully place it so that it balances and so that the wireless receptor inside the phone matches the transmitter in the mat, if the phone isn t placed correctly then the Qi mat won t detect it and won t supply full power to the transmitter. Conclusion I m fairly disappointed in this device. The waste heat makes it unsuitable for Australian summer conditions and slows charging. The difficulty of correctly placing the phone reduces the convenience which is one of the major features. The price was $19 for each charging card for the Note 2 and the S3 and $29 for the charging mat to give a total of $67. I think it s worth the money for me to cover the risk of one of my phones having it s USB port damaged. Using a Qi charger on occasion will decrease the probability of such damage and allow the phone to be used after receiving certain types of damage. The prices of those phones nowadays are $389 for a Galaxy S3 (Kogan price), $250 for a Nexus 4 (when it was on sale in the Google store), and probably about $500 for a Galaxy Note 2 (last time Kogan offered them). So by paying $67 for Qi charging I believe that I m getting some degree of damage insurance for just over $1100 worth of phones. It seems likely that the Nexus 5 will ship with Qi charging support and that the Galaxy Note 3 will also support an optional Qi charging card (which will probably also be $19 or some similar price) so the charging mat should be useful for a long time. While I m disappointed I don t regret buying the device. But I would be hesitant to recommend it to other people and definitely wouldn t recommend it to someone who doesn t have a significant interest and investment in smart phones.

• [1] http://etbe.coker.com.au/2013/01/17/cooling-phones/

all your ctag (and checksum) are belong to us A few months after the initial announcement, here are some news about the sources.d.n service. I've been late in blogging this, but most of it has been implemented by myself and Matthieu Caneill during DebConf13, which has been a great DebConf, totally exceeding my expectations (and they were already fairly high!). First, you might have noticed some user-visible changes:

• there is now an advanced search page, which complements the already existing regex code search with the possibility of searching source files by their sha256, or the ctags defined therein
• on the same topic, when browsing through a package and using regex search, you'll now search by default within that package, allowing to focus your searches more easily than before. (You can easily override this by editing the search box and removing the package: predicate.)
• for the data geeks (or the wannabe host), there are now disk usage stats (note that they don't include the database size, though, see below for that)
• the website also got a significant facelift, as part of which we have moved the detailed explanations of what the service is about out of your way. You now immediately get to the various browsing options.

On the other hand, under the hood:

• to implement ctags and sha256 searches we needed a serious DBMS, so we switched from SQLite to PostgreSQL. Again, for the data geek: storing ctags/sha256 for all of sources.d.n content with decent indexes takes about 37 GB, for about 160 million rows in the ctags table and 20 million rows in the checksums one. (Currently filenames are duplicated between the two tables so, probably, the DB disk size might be reduced some.)
• together with the switch to a serious DBMS, the update logics has been completely rewritten in Python (from Bash...), and should now be entirely transactional.
• ... and given it was going to be Python anyhow, better to enjoy what it has to offer, no? So there is now a plugin mechanism that makes it easier to add extra data extractors, triggering them at each package update. Currently there are plugins for sha256sum, ctags, and sloccount (even though the latter is not yet exposed via the web interface). An added benefit of this is that if you want to deploy debsources elsewhere, you can easily disable the most time consuming extractors: running ctags and sha256sum on the fabulous 3 chromium/libreoffice/linux is not for the faint of disks...
• we now receive push updates from the Debian mirror network, so that you'll get updates on sources.d.n as soon as a package hits Debian mirrors (+ processing time, which is about 15-20 minutes on the average update run). Many thanks to Simon Paillard and Adam Lackorzynski for their help in setting this up.
• thanks to a suggestion by kugel we have adopted Geany's conventions for filetype detection, and we now take into account both file extensions and shebang lines (when available)

As you usual, your bug reports (and patches!) are more than welcome, just check BUGS before reporting to avoid duplicates.
That's all!

I love to say that Python is a nice subset of Lisp, and I discover that it's getting even more true as time passes. Recently, I've stumbled upon the PEP 443 that describes a way to dispatch generic functions, in a way that looks like what CLOS, the Common Lisp Object System, provides. What are generic functions If you come from the Lisp world, this won't be something new to you. The Lisp object system provides a really good way to define and handle method dispatching. It's a base of the Common Lisp object system. For my own pleasure to see Lisp code in a Python post, I'll show you how generic methods work in Lisp first. To begin, let's define a few very simple classes.
This defines a few classes: snare-drum, symbal, stick and brushes, without any parent class nor attribute. These classes compose a drum kit, and we can combine them to play sound. So we define a play method that takes two arguments, and returns a sound (as a string).
This only defines a generic method: it has no body, and cannot be called with any instance yet. At this stage, we only inform the object system that the method is generic and can be then implemented with various type of arguments. We'll start by implementing versions of this method that knows how to play with the snare-drum.
Now we just defined concrete methods with code. They also takes two arguments: instrument which is an instance of snare-drum and accessory that is an instance of stick or brushes. At this stage, you should note the first difference with object system as built into language like Python: the method isn't tied to any class in particular. The methods are generic, and any class can implement them, or not. Let's try it.
As you see, the function called depends on the class of the arguments. The object systems dispatch the function calls to the right function for us, depending on the arguments classes. If we call play with instances that are not know to the object system, an error will be thrown. Inheritance is also supported and the equivalent (but more powerful and less error prone) equivalent of Python's super() is available via (call-next-method).
In this example, we define the stick and brushes classes as subclass of the accessory class. The play method defined will return the sound BIIING! regardless of the accessory instance that is used to play the cymbal. Except in the case where it's a brushes instance; only the most precise method is always called. The (call-next-method) function is used to call the closest parent method, in this case that would be the method returning _"BIIING!".
Note that CLOS is also able to dispatch on object instances themself by using the eql specializer. But if you're really curious about all features CLOS provides, I suggest you read the brief guide to CLOS by Jeff Dalton as a starter. Python implementation Python implements a simpler equivalence of this workflow with the singledispatch function. It will be provided with Python 3.4 as part of the functools module. Here's a rough equivalence of the above Lisp program.
We define our four classes, and a base play function that raises NotImplementedError, indicating that by default we don't know what to do. We can then write specialized version of this function with a first instrument, the SnareDrum. We then check for the accessory type that we get, and return the appropriate sound or raise NotImplementedError again if we don't know what to do with it. If we run it, it works as expected:
The singledispatch module looks through the classes of the first argument passed to the play function, and calls the right version of it. The first defined version of the play function is always run for the object class, so if our instrument is a class that we did not register for, this base function will be called. For whose eager to try and use it, the singledispatch function is provided Python 2.6 to 3.3 through the Python Package Index. Limitations First, as you noticed in the Lisp version, CLOS provides a multiple dispatcher that can dispatch on the type of any of the argument defined in the method prototype, not only the first one. Unfortunately, Python dispatcher is named singledispatch for this good reason: it only knows to dispatch on the first argument. Guido van Rossum wrote a short article about the subject that he called multimethod a few years ago. Then, there's no way to call the parent function directly. There's no equivalent of the (call-next-method) from Lisp nor the super() function that allows to do that in Python class system. This means you will have to use various trick to bypass this limitation. So while I am really glad that Python is going toward that direction, as it's a really powerful way to enhance an object system, it really lacks a lot of more advanced features that CLOS provides out of the box. Though, improving this could be an interesting challenge. Especially to bring more CLOS power to Hy. :-)

After years of waiting, Google has finally enabled IPv6 for their email service Gmail. And a few weeks ago, they updated their policy, adding one specific rule: reject email from IP addresses with no reverse name:

% nc -Cv gmail-smtp-in.l.google.com. smtp
Connection to gmail-smtp-in.l.google.com. 25 port [tcp/smtp] succeeded!
220 mx.google.com ESMTP bz2si13656083wjc.108 - gsmtp
HELO boo.example.com
250 mx.google.com at your service
MAIL FROM: <me@example.com>
250 2.1.0 OK bz2si13656083wjc.108 - gsmtp
RCPT TO: <you@gmail.com>
250 2.1.5 OK bz2si13656083wjc.108 - gsmtp
DATA
354  Go ahead bz2si13656083wjc.108 - gsmtp
Subject: Test
From: Me <me@example.com>
To: You <you@gmail.com>
Test.
.
550-5.7.1 [2001:db8:8e3f:43c7::12      16] Our system has detected that this
550-5.7.1 message does not meet IPv6 sending guidelines regarding PTR records
550-5.7.1 and authentication. Please review
550-5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more
550 5.7.1 information. bz2si13656083wjc.108 - gsmtp

That sucks

• It is not a sane anti-spam measure, because not having a reverse name tells absolutely nothing about the email emitter or the message itself: it only indicates that the email emitter has a lame access provider. Specifically:
  • spammers can have perfect reverse names;
  • regular email emitter can have no reverse name because their access provider suck, and have no way of fixing it because all access providers suck the same way.
• While IPv6 is the future of the Internet, this measure discourages its use. Yes, providing no reverse names suck, but it is by no mean the user's fault, and since IPv6 is more complicated to implement than IPv4, Internet actors must tolerate youth errors or people will just keep what works, i.e. IPv4.

By the way, I would love to have a perfect Internet access provider, but that does not exist where I live, and I do not think it exists at all anywhere. Google, instead of complaining about lame access providers, you are welcome to offer me a fibre service with decent upload rate, static IPv4, static IPv6 /64, customizable reverse DNS for IPv4 and IPv6, and full respect of the network neutrality. Until then, I am doing the best I can with the least bad provider available. Workaround As I said, their is an easy workaround. Google does not accept my mail on IPv6? Fine, I will keep using IPv4. Only for Google of course, no need to punish the whole Internet for Google's damn restrictions. With Postfix, you can do that this way (thanks to Christian Skala for his blog post about this problem). In /etc/postfix/master.fr, enable an IPv4-only SMTP client service:

smtp4     unix  -       -       -       -       -       smtp -o inet_protocols=ipv4

Then, in /etc/postfix/main.cf, define a transport map if you do not already use one:

transport_maps = hash:/etc/postfix/transport

Create that transport map /etc/postfix/transport or complete it:

gmail.com smtp4:

Finally, hash that transport map and have Postfix reload its configuration:

# postmap /etc/postfix/transport
# service postfix reload

Today I link you to a video narrated by the legendary Carl Sagan The Frontier is Everywhere. Partial quotes:

We were hunters and foragers. The frontier was everywhere. We were bounded only by the earth, and the ocean, and the sky. The open road still softly calls. Our little terraqueous globe as the madhouse of those hundred thousand millions of worlds. We, who cannot even put our own planetary home in order, riven with rivalries and hatreds; are we to venture out into space? By the time we are ready to settle even the nearest other planetary systems, we will have changed For all our failings, despite our limitations and fallibilities, we humans are capable of greatness How far will our nomadic species have wandered, by the end of the next century, and the next millennium? Our remote descendants, safely arrayed on many worlds through the solar system, and beyond, will be unified, by their common heritage, by their regard for their home planet, and by the knowledge that, whatever other life may be, the only humans in all the universe, come from Earth. They will gaze up and strain to find the blue dot in their skies. They will marvel at how vulnerable the repository of all our potential once was, how perilous our infancy, how humble our beginnings, how many rivers we had to cross, before we found our way.

Can you imagine seeing Earthrise? Incredible.

It seems I'm catching up with the pace of this course I'm following and that is compelling me to go back to posting on my blog, Arte y cultura en circulaci n: crear y compartir en tiempos digitales . This week's lesson is (again, in Spanish) Las fronteras del remix (the boundaries of remixes). An interesting text, open to everybody (regardless of whether you are signed up for the course or, I hope, whether time has passed since the course took place). And this week's homework is to find "our favorite" non-original work (I picked one among my favorite works And, yes, this is partly because I am part of the "club" of deniers of true originality: We cannot create anything without being part of a surrounding culture, without a common heritage and language with which we speak to our audience) and to find something about it, anything considered important or significative as to its antecedents. What do I like about this work, what grabs my attention. Do I consider it to be a true new creative work? Why? I am taking as an example Leo Masliah, an Uruguayan writer and musician (writer and interpreter). I have followed and enjoyed Masliah's work since 1996, and although by far I'm most familiar with his musical works, I have two books (a novel and a series of short stories). Among his facets, I most enjoy the acid, nihilistic/dadaistic streaks. I chose three of his songs to talk about I am linking to anonline resource where possible, but uploading the three songs to this blog to make his work better known, so that people understand what I talk about, and with my best intentions. Of course, if there is any request to remove the material, I will do it right away. I hope this can be seen as fair/academic use, although this blog is somewhat widely read.

La recuperaci n del unicornio

This song was in the first Masliah cassette I came across, Although musically it is an original work, the lyrics are a clear, almost line-by-line reply to the ever-repeated Unicornio song, by Silvio Rodr guez. This song (which I'm sure that every Spanish speaker reading this lines knows, like it or not It defines for me a good deal of Rodr guez hyper-sweetness and clicheness) has been analyzed over and over looking for a meaning. Silvio recorded this song in 1981, and by 1987 Masliah published this answer, mocking each of the lines.
So, this song can be seen as an original creation, as it contains no literal copy neither of the music nor of the lyrics of the original, but it cannot be understood without being familiar with Silvio's lost unicornio. It would just be an almost-dadaistic rant. But every person that has tasted Silvio Rodr guez cloying song will surely laugh with this one.

No necesitamos otro h roe Balderrama

I find this to be a very unique piece. It bonds together Argentinian folklore and USA pop.
Masliah sings the famous Argentinian song Zamba de Balderrama, (Castilla / Leguizam n), a song lamenting that the very popular bar and artist stage Balderrama, in Salta (North-Western corner of Argentina), seemed to be on the brink of closing. Of course, when Mart n, Constanza, Regina and me went to Balderrama in July 2010, the danger of closing this place had long been averted. And basically every Argentinian knows this song by heart.
The interesting part is that Masliah sings this song (as he presents it as the closing piece for one of his concerts) being short on time, and decides to present two acts together: Zamba de Balderrama and We don't need another hero, by Tina Turner, from the Mad Max soundtrack. So, of course, even if it's obvious that Masliah had to stretch bits to make them fit together, he achieved a very funny, interesting and unique blend of two completely unrelated works. He derives from one as well as from the other, but creates something unique and new.

Donna Lee

I was thinking what to write for this assignment , and had already chosen the two other songs. Yesterday, we were visiting our friends Octavio and Claudia, in Guanajuato, and I heard for the first time the original(?) version of Donna Lee, by Miles Davis (although often credited to Charlie Parker). That led me to remember Masliah's interpretation, in his Cl sicos album. This album is basically made up with similar excercises: Taking an instrumental piece, with Masliah singing over it. Some of the lyrics follow the story (i.e. the children stories), others just talk nonsense to illustrate a point (i.e. "La voz del medio", "the middle voice") follows the non-protagonist score trying not to grab much attention to the lyrics but to highlight the often ignored melody)... Donna Lee is just a fun excercise on following the melody asking what was in the head of the author that led him to invent such a strange, hard music.

Masliah is a great music performer, although often it seems he tries to hide it (i.e. by abusing the dissonances, ex-profeso singing off-key, etc), and a very funny and crazy author. Most of his works have a deep satirical tone, and it's common to find either simple winks or complete "borrowings" in a clear remix fashion, but nobody will doubt on the originality of his works.

Attachment	Size
07-Donna Lee.MP3	1.79 MB
16 No necesitamos otro h roe - Balderrama.mp3	3.49 MB
Leo Masl ah-La Recuperaci n Del Unicornio.mp3	1.87 MB

DebConf 13 report (by Andreas Tille) General impression I'm beginning my DebConf report in an unofficial "Scenic Hacklab" right at the edge of the lake in Yverdon. This is the right place to memorise the last days. When I started from this place cycling to Le Camp 12 days ago I was full of great expectations and what should I say - the reality has even beaten these. Once it comes about comparing DebConfs even if it is an unfair comparison due all the differences my secret long term favourite was Helsinki very closely followed by Argentina and also very closely followed by all the other great DebConfs I joined (and I joined all in Europe). Would Le Camp be able to beat it? The short answer is: Yes, it is now my favourite DebConf while I think I do not suffer from the last-Debconf-was-the-best-DebConf-syndrome (and I realised there are others thinking the same). As you might probably know I'm a bit addicted to swimming. While Helsinki had admittedly the better conditions I was at least able to fix the distance issue using my bicycle. (Hey, those Le Camp photographers did a great job in hiding the fact that you can not actually touch the lake right from the meadow of Le Camp.) Being able to have my bicycle at DebConf scored some extra points. However, the really great view of the lake, the inspiring "Scenic Hacklab" which was my favourite place has bumped DebConf13 at first place in my personal ranking. So it comes quite natural to say: "Kudos to the great organisation team!" They did a Swiss-like precise work and perfectly succeeded in hiding any problems (I assume there were some as always) from the attendees so everything went smooth, nice and shiny for the attendees. The local team was even precise in setting up great weather conditions for DebConf. While saying thanks to the local team I would like to also explicitly thank Luca Capello who has quite some share that this DebConf was possible at all (while I have to decrease my DebConf score one point because he was not really there - Luca to bad that you were not able to come full time!) Also thanks to Gunnar and Gannef who helped remotely (another score down because I were missing them this year as well). Even if it was my favourite DebConf I was not able to work down my todo list fully (which was not only uploading one package per day which I at least statistically fullfilled). But that's probably a general feature of todo lists anyway. One item was definitely done: Doing my daily swimming BoF. I actually was able to do the other parts of the triathlon which was skipped by Christian and have done in summary about 150km cycling with 3500m elevation and estimated 7-8km swimming (0m elevation ;-)). Considering the great view at sunrise over the lake I was not hating my "Senile bed escape" disease too much (I was every day waking up at sunset) - it was simply a great experience. I will never forget seeing water drips glimmering like gold inside the morning sun while seeing the Alps panorama in the distant. I hope I was able to help all interested swimmers with the DebConf Beach Map which was just a by-product of my activities in DebCamp. Speaking about OSM: I was astonished that the area was way less covered than I expected. Thanks to several DebConf attendees the situation became better and the map does not only show random trees in the wild but also the tracks leading to these. (Remark: It was no DebConf attendee who is responsible for plastering the map with single trees.) While I had my mapping focus basically close to the edge of the lake I was also able to even map my very own street. :-) I clearly remember one specific mapping tour when I was invited by the DPL: He convinced me to join him on a bicycle tour and since I was afraid to get fired I joined him instead to keep on hacking. Also Sorina was brave enough to join us on the tour and she did quite well. (Sorina, do you remember the agreement about your work on the installer? ;-)) Lucas described the tour as: going uphill on only asphalted roads. Sorina and me were witnessing the mighty DPL powers when we left the wood around Le Camp to reach the described road: The asphalt was just put onto the road - no doubt that it was done on the immediate demand of mighty DPL. :-) DebCamp time was flying like nose dive and a lot of known (and unknown) faces arrived at Le Camp. What I really liked a lot this year was that several really young children has pulled down the average age of DebConf attendees. I clearly remember all the discussion one year ago what to do about children. As always the issue was solved in a typical Debian way: Just do it and bring your children - they had obviously a great time as well. I think the youngest child was 2 months and the oldest "child" above 20. ;-) Actually Baptiste Perrier did great in making the C&W party a success and had obviously a nice time. (I wished my son would have been able to come as well but he needs to write his bachelor s thesis in physics. :-() It was nice to see the kids using all playing facilities and communicating with geeks. Also I would like to point out that even the very young attendees had their share at the success of DebConf: Just think of the three "bell ringing assistants" who helped me ringing the bells for lunch and dinner. I've got this cool job from Didier in the beginning of DebCamp. I must say having some real bells ringing is by far nicer than just the "lunch / dinner starts in 10 minutes" from IRC bot. The only thing I did not understand was that people did not considered ringing the bells at 8:00 for breakfast as a good idea. Regarding the food in general I would also like to send kudos to the kitchen: It was tasty, freshly prepared, regional food with a good change rate. I really liked this. Extra points for having the chance to sit outside when eating. Talks But lets have a look into the conference programme. I'd really recommend watching the videos of the talks Bits from the DPL (video) and Debian Cosmology (video). I considered both talks as entertaining and interesting. I also really hope that the effort Enrico Zini started in Debian Contributors (video) will be successful. I had some talks and BoFs myself starting with Why running a Blend (video) and I admit that (as usual) the number of attendees was quite low even if I think there is some proof (see below) that it is interesting for way more people who should consider working more "blendish" in their team. Do you know how to recruit one developer per year and relax the man power problem in your team? Feel free to watch the video. We have confirmation that ten DDs of our team have considered to join Debian only because Debian Med exists. Admittedly biology and medicine are really leaf topics inside the Debian universe. So if even this topic that has a very tiny share of the Debian users is able to attract this level of attention - how many more people could we win for multimedia, games, GIS and others? So if you feel you are quite overworked with your packaging and you have no time this is most probably wrong. The amount of time is basically a matter of priorities you set for your tasks. Try to put some higher priority onto using the just existing Blends tools I explained in my talk to attract more users and developers to your team and by doing so spread the workload over more people. It works, the prove was given in my main talk. So before you start working on a specific package you should wonder who else could have an even stronger interest to get this work done and provide him with some additional motivation and help to get the common goal done. The interesting thing is that my BoF about How to attract new developers for your team (video) - which was a simple report about some by-product of the Blends work - made it into the main talk room and got way more attention. For me this is the proof that the Blends concept itself is probably badly perceived as something like "a few outsiders are doing damn specific stuff which is not really interesting for anybody else" instead of what is really is: Smoothing the way from specific upstream applications to the end user via Debian. Once you see the video of this BoF you can observe how my friend Asheesh Laroia became more and more excited about the Blends concept and admitted what I said above: We should have more Blends for different fields. Funnily enough Asheesh asked me in his excitement to talk more about Blends. This would have been a really good suggestion ten years ago. At DebConf 3 in Oslo I had my very first talk about Blends (at this time under the name "Debian Internal Projects"). I continuously kept on talking about this (MiniDebConf Peking 2005, DebConf 5, Helsinki (video), DebConf 7, Edinburgh (video), DebConf 8, Mar del Plata (video), DebConf 9, C ceres (video), MiniDebConf Berlin 2010 (video in German), MiniDebConf Paris 2010 (not video recorded), DebConf 11, Banja Luka (video) ... and these are only (Mini)DebConfs my talks page is full of this topic) and every new year I try different ways to communicate the idea to my fellow Debianistas. I'm wondering how I could invent a title + abstract avoiding the term Blends, put "Git", "release" and "systemd versus upstart" in and being able to inform about Blends reasonably by not becoming to off topic with the abstract. I also registered the Debian Science round table. I admit we were lacking some input from remote via IRC which used to be quite helpful in the past. The attendees agreed upon the handling of citations in debian/upstream files which was invented by Debian Med team to create even stronger bounds to our upstream developers by giving their work extra reward and providing users with even better documentation (see my summary in Wiki). As usual I suggested to create some Debian Science offsprings like "Debian Astronomy", "Debian Electronics", "Debian Mathematics", "Debian Physics" etc. who could perfectly leave the Debian Science umbrella to get a more fine grained structure and a more focused team to enhance the contact to our users. Unfortunately there is nobody who volunteers to take over the lead for such Blends. I have given a short summary about this BoF on the Debian Science mailing list. In the Debian Med meeting I have given some status report. No other long term team members were attending DebConf and so I gave some kind of introduction for newcomers and interested people. I touched also the DebiChem topic which maintains some packages that are used by biologists frequently and so we have a good connection to this team. Finally I had registered three BoFs in Blends I'm actually not (or not yet) active part of. My motivation was to turn the ideas I have explained in my main talk into specific application inside these teams and helping them to implement the Blends framework. In the first BoF about Debian GIS I have shown the usual team metrics graphs to demonstrate, that the one packaging team Pkg-OSM is in danger to become MIA. There are only three persons doing actual uploads. Two of them were at DebConf but did not joined the BoF because they do not consider their contribution to Pkg-OSM as a major part of their general Debian work. I will contact the main contributor David Paleino about his opinion to move the packages step by step into maintenance of Debian GIS packaging team to try to overcome the split of two teams that are sharing a good amount of interest. At least if I might become an Uploader for one of the packages currently maintained by Pkg-OSM I will move this to pkg-grass-devel (which is the name of the packaging team of Debian GIS for historical reasons). The attendees of the BoF have considered this plan as sensible. Moreover I talked about my experiences with OSGeo Live - an Ubuntu derivative that tries to provide a full tool chain to work on GIS and OSM problems ... basically the same goal as Debian GIS has just provided by the OSGeo project. I'm lurking on OSGeo mailing list when I asked explicitly I've got the answer that they are working together with Debian GIS and are using common repository (which is IMHO the optimal way of cooperation). However, it seems that several protagonists of OSGeo Live are underestimating the resources provided by Debian. For instance there was a question about Java packaging issues but people were not aware about the existence of the debian-java mailing list. I was able to give an example how the Debian Med team managed to strengthen its ties to BioLinux that is also an Ubuntu derivative for biologists. At our first Debian Med sprint in 2011 we invited developers from BioLinux and reached a state where they are using the very same VCS on Alioth where we are maintaining our packages. At DebConf I was able to upload two packages where BioLinux developers did certain changes for enhancing the user experience. My "work" was just bumping the version number in changelog and so we did profit from the work of the BioLinux developers as well as they are profiting from our work. I plan to dive a bit more into Debian GIS and try to strengthen the connection to OSGeo Live a bit. The next BoF was the Debian Multimedia meeting. It was nice that the current leader of Ubuntu Studio Kaj Ailomaa joined the meeting. When I was explaining my ideas about cooperation with derivatives I repeated my detailed explanation about the relation with BioLinux. It seems every topic you could cover inside Debian has its related derivative. So to me it seems to be quite natural to work together with the developers of the derivative to join forces. I actually consider a Blend a derivative done the right way = inside Debian. The final work for the derivers that might be left for them is doing some shiny customising of backgrounds or something like this - but all the hard work could and should be done in common with the relevant Debian team. My dream is to raise such relevant teams inside Debian ... the Blends. Finally the last BoF of this series was the Debian Games meeting. As always I presented the team metrics graphs and the Debian Games team members who attended the BoF were quite interested. So it seems to be some unknown fact that team metrics are done for several teams in side Debian and so I repeat the link to it for those who are not yet aware of it. As a result of the BoF Debian Games team members agreed to put some more effort into maintaining their Blends tasks. Moreover Miriam Ruiz wants to put some effort into reviving Debian Jr. Regarding Debian Jr. there was an interesting talk about DouDouLinux - in case you might want to watch the video I'd recommend skipping the first 30min and rather watch the nice live demo. There was also an ad hoc BoF about Debian Jr scheduled to bring together all people interested into this cute project and Per Anderson volunteered to take over the lead. I have given a summary about this specific BoF at the Debian Jr list. For some other talks that I'd regard as remarkable for some reasons: I'd regard the talk "Debian-LAN" by Andreas Mundt as some hidden pearl because it did not got a lot of attention but after having seen the video I was quite impressed - specifically because it is also relevant for the Blends topic. Memories I also liked "Paths into Debian" by Moray Allan (and I was only able to enjoy the latter talks thanks to the great work of the video team!) because it also scratched the same topic I was concerned about in my mentoring talk. Related to this was in my opinion also "Women in Debian 2013" were we tried to find out reasons for the lack of woman compared to other projects and how to overcome this issue. Besides the talks I will probably never forget two specific moments that make DebConf so special. One of these moments is recorded on an image that clearly needs no words - just see Geert hovering over the grass. Another strong moment in my personal record was in the DebConf Newbies BoF "First time at DebConf" that unfortunately was not recorded but at least for this statement it would have been very great if we would have some reference better than personal memory. Aarsh Shah a GSoC student from India suddenly raised up and said: "Four months ago I was not even aware that Free Software exists. Now I'm here with so many people who are totally equal. If I will tell my mother at home that I was standing in the same queue where the Debian Project Leader was queuing up for food she will never believe me." He was totally excited about things we are regarding as normal. IMHO we should memorise moments like this that might be part of the key to success in cultures, where Debian is widely unknown and very rarely in use. Amongst these not scheduled great moments the scheduled day trip was also a great thing. I had a really hard time to decide what tour I might join but ended up in the "long distance walking (or should I say running) group". Inspired by the "running Bubulle" who was flashing between the walking groups we went uphill with 5.4km/h which was a nice exercise. Our destination the large cliff was an exciting landscape and I guess we all enjoyed the dinner organised by the "Trout cabal". ;-) So I had a hard time to leave Le Camp and tried hard to make sure my memories will remain as long as possible. Keeping some signs attached to my bicycle, conserving the "Scenic Hacklab" sign for my private "scenic hacklab @ home" was one part. I also have cut some branches of the Buxus sempervirens in Le Camp and have put them in my garden at home (where I create some hedgerow from places where I spent some great time). These will probably build a great part of the hedgerow ... Thanks for reading this longish report. Looking forward to see you all in Germany 2015 (or earlier) Andreas.

... at least until the cgroup interface is fixed. So, what can it do?

• There is tasks and cgroup.procs. There are no thread IDs on Hurd, so cgroupfs works only on a per-process basis, not per-threads. Consequently tasks has the same semantic as cgroup.procs. Seeing that PIDs and TIDs can be used (mostly) interchangeably on Linux I think this is okay to do.
• You can create and destroy cgroups, child processes are properly tracked.
• You can register an release_agent and it is executed whenever the last process in a cgroup dies.
• There is notify_on_release to enable or disable the use of release_agent.
• There is cgroup.clone_children, one can toggle this bit but it is ignored.

So, what's missing?

• There are no controllers. I haven't looked into this and resource accounting is one of Hurds weakest points, but it is fathomable that one could e.g. advise the scheduler inside the Mach kernel based upon the state of the cgroups if the cgroupfs process is sufficiently privileged (did I mention that any user can use cgroupfs?).
• The notification API aka cgroup.event_control. The Hurd lacks eventfd(2), but even if that was implemented, this interface would still be impossible to implement. Rant below.
• A patch for gnumach to make this bulletproof. I made some encouraging progress with that one this week, but there's nothing presentable yet.

So, what's wrong with Linux cgroup API? Well for one thing the whole API is underspecified. Yes, there is Documentation/cgroups/cgroups.txt, but that is not a specification, that's a howto at best. Second, the notification API is not particularly nice:

To register a new notification handler you need to:
 - create a file descriptor for event notification using eventfd(2);
 - open a control file to be monitored (e.g. memory.usage_in_bytes);
 - write "<event_fd> <control_fd> <args>" to cgroup.event_control.
   Interpretation of args is defined by control file implementation;

Seriously? There is a POSIXly way to pass file-descriptors around, but smashing the decimal representation of it into a string is not the way to do that. Linux gets away with this hack because the kernel knows the process who wrote(2) that string in the first place, parse the string into an integer and look it up in the table of file descriptors for that process. Now the trouble for cgroupfs is, that it is not the kernel and even if it were, it wouldn't solve the problem because on Hurd there are no file descriptors (well there are, but that's only to appease all the POSIX programs out there). Instead Hurd has ports, and you can send messages to ports, and this is pretty much everything that you can do on a Mach system. Reading a file works roughly like this:

1. You open a file and get a port X.
2. You send a message like "I'm like really interested in the first Y bytes of that file" to X.
3. Whoever has the receiving end of X (probably the one who gave you X in the first place) answers your request.

Ports look pretty much like file descriptors, they are (usually small) integers, you can make them, destroy them, pass them around easily (yes, ports are first class objects in the Mach messaging system). Everything is implemented atop of this mechanism. It is transport-agnostic, the other end could be on another machine and you wouldn't even know. You can create proxies or filters (in fact, that is exactly how the firewall eth-filter is implemented). It's beautiful and extensible at it's heart, like Lego bricks. So if X were a port to e.g. memory.usage_in_bytes and the cgroups interface would be less braindead^W^Wmore carefully designed so that on Hurd it could be transported like ports usually are, then cgroupfs could in fact use port X' to look up which file the caller is interested in (this is possible because cgroupfs was the one handing out the port in the first place) and generate notifications for that file. This is not possible when X is "serialized for transport" using sprintf because port names are specific for each process, so X != X'. The kernel would do the translation while sending the message, but it obviously cannot do that if the number is carried in a character array. I'm not sure what I'm going to do next week. The gsoc timeline suggests a soft-pencils-down, time to scrub code and write documentation, not sure that this is applicable to me as I have pushed most of my work upstream as early as possible. I guess I will nag Samuel so that he merges the outstanding patches and continue working on my gnumach patch.

One of the very long-standing problems with AFS is that it doesn't have a stable library interface or API. The situation with shared libraries is even worse. Despite that, there is a native XS AFS Perl module that tries to provide a native Perl interface to the various AFS libraries, despite all of their quirks. We've used that off and on for various things over the years. Unfortunately, the historic AFS code used an LWP (lightweight threading) implementation that relied on hairy C internals, and when Perl switched over to being threaded with POSIX pthreads by default, the loadable module started having trouble. Various people managed to keep it limping along through Debian's squeeze release, but as of wheezy it just no longer worked; the thread mismatches and the mismatches between the various AFS libraries were too severe. There's work in progress to fix this by building yet another profile of the AFS libraries, with all their warts, but in the meantime we needed access to the AFS PAG functions for various backend scripts. Everything else the Perl module did could be replaced by running the regular commands and parsing their output, but setpag has to be called in the process that should be affected to work safely. Hence this module. AFS::PAG provides a native Perl interface to the API exposed by libkafs and libkopenafs for PAG manipulation: hasafs, haspag, setpag, and unlog. (Eventually it might provide access to pioctl, but I haven't done that work yet.) It supports any platform that has a libkafs or libkopenafs, or Linux without either (by implementing the pioctl interface itself). You can get the latest version from the AFS::PAG distribution page.

I signed up to take part of an online, massive course Arte y cultura en circulaci n: crear y compartir en tiempos digitales (The circulation of arts and culture: Creating and sharing in digital times). As the activities of us the attendees are to be published by each on their personal blog/space, I will be publishing them here. I hope they are interesting to some of my other regular readers. I am late in joining, and I should already be posting the second activity. Anyway, this text goes towards the first week's lesson: Qu es un autor: la (de)construcci n hist rica del concepto de autor a (What is an author? Historic (de)construction of the concept of authorship). Our homework is to find an example of a current discussion where the notion of authorship is discussed, and share it this way. And I'm somewhat in a hurry right now, so this will be hasty. But I didn't want to delay my (late) submission anymore! In the Debian Ruby mailing list, Hleb Valoshka asks basically how deep should copyright recognition be. Because, yes, while copyright attribution is simple (simpler, at least) in literary / artistic works, computer programs (and even more those developed following the Free Software distributed fashion) are harder to properly attribute. Is a two-line patch to a tens-of-thousands-of-lines project enough to warrant mention in the copyright file? Most of us would agree it is not. Few would contend this were the amount of changes to make up, say, even 1% of that scale of project. In the coding projects where I am the main author (which, yes, are usually very small I cannot recognize myself as a good programmer, and the size of my successful works proves me), I try to take care to mention explicitly each of the contributors, even if their contribution is quite small. But were I to lead a larger development work, with enough following to generate on the order of one submitted patch a day, would it make sense for me to follow up with such detail the authorship information? And as a minor side note: How much does the law require me? A very small patch can fix important functionality issues (think security or performance). If somebody sends a very small patch fixing an intellectually hard to grasp problem... I am sure it should be properly acknowledged! Anyway, I have to flee now. I am just dropping some ideas on the table. Of course, if anybody is willing to discuss them further, I'm interested in any debate that springs off it! (Meanwhile... I expect to spam you with this topic a couple of times. But it should not bug you much, as it's one of my usual blog topics!)

Many thanks to Vincent Meurisse who solved my node.js callback woe. Some history of the blogspam service:

Back in 2008 I was annoyed by the many spam-comments that were being submitted to my Debian Administration website. I added some simple anti-spam measures, which reduced the flow, but it was a losing battle. In the end I decided I should test comments, as the users submitted them, via some kind of external service. The intention being that any improvements to that central service would benefit all users. (So I could move to testing comments on my personal blog too, for example). Ultimately I registered the domain-name "blogspam.net", and set up a simple service on it which would test comments and judge them to be "SPAM" or "OK". The current statistics show that this service has stopped 20 million spam comments, since then. (We have to pretend I didn't wipe the counters once or twice.)

I've spent a while now re-implementing most of the old plugins in node.js, and I think I'll be ready to deploy the new service over the weekend. The new service will have to handle two different kinds of requests:

New Requests

These will be submitted via HTTP POSTed JSON data, and will be handled by node.js. These should be nice and fast.

Legacy Requests

These will come in via XML-RPC, and be proxied through the new node.js implementation. Hopefully this will mean existing clients won't even notice the transition.

I've not yet deployed the new code, but it is just a matter of time. Hopefully being node.js based and significantly easier to install, update, and tweak, I'll get more contributions too. The dependencies are happily very minimal:

• A redis-server for maintaining state:
  • The number of SPAM/OK comments for each submitting site.
  • An auto-expiring cache of blacklisted IP adddresses. (I cache the results of various RBL results for 48 hours).
• node.js

The only significant outstanding issue is that I need to pick a node.js library for performing CIDR lookups - "Does 10.11.12.23 lie within 10.11.12.0/24?" - I'm surprised that functionality isn't available out of the box, but it is the only omission I've missed. I've been keeping load & RAM graphs, so it will be interesting to see how the node.js service competes. I expect that if clients were using it, in preference to the XML-RPC version, then I'd get a hell of a lot more throughput, but with it hidden behind the XML-RPC proxy I'm less sure what will happen. I guess I also need to write documentation for the new/preferred JSON-based API... https://github.com/skx/blogspam.js

I m getting some odd log messages for the apache module modsecurity. Essentially its xml parser is breaking when random places are sending pingbacks. The requests go to xmlrpc.php and the response headers are ok, but the body is binary. The message in the modsecurity log looks like:

Message: XML parser error: XML: Failed parsing document.

After a bit of guessing and sending messages to and fro, I can now see that it is a gziped response. So I m not sure if it is modsecurity not realising that the response is gziped or wordpress not marking it correctly. In any case I can regularly get very similar binary strings using gzip and the usual xml response. So that s half the mystery solved. Most of the requests are spammers so I m not too worried. I think it also impacts legitimate pingbacks because I ve not had any, even from the usual automatic places.

I was introduced to the Freedombox project in 2010, when Eben Moglen presented his vision about serving the need of non-technical people to keep their personal information private and within the legal protection of their own homes. The idea is to give people back the power over their network and machines, and return Internet back to its intended peer-to-peer architecture. Instead of depending on a central service, the Freedombox will give everyone control over their own basic infrastructure. I've intended to join the effort since then, but other tasks have taken priority. But this summers nasty news about the misuse of trust and privilege exercised by the "western" intelligence gathering communities increased my eagerness to contribute to a point where I actually started working on the project a while back. The initial Debian initiative based on the vision from Eben Moglen, is to create a simple and cheap Debian based appliance that anyone can hook up in their home and get access to secure and private services and communication. The initial deployment platform have been the Dreamplug, which is a piece of hardware I do not own. So to be able to test what the current Freedombox setup look like, I had to come up with a way to install it on some hardware I do have access to. I have rewritten the freedom-maker image build framework to use .deb packages instead of only copying setup into the boot images, and thanks to this rewrite I am able to set up any machine supported by Debian Wheezy as a Freedombox, using the previously mentioned deb (and a few support debs for packages missing in Debian). The current Freedombox setup consist of a set of bootstrapping scripts (freedombox-setup), and a administrative web interface (plinth + exmachina + withsqlite), as well as a privacy enhancing proxy based on privoxy (freedombox-privoxy). There is also a web/javascript based XMPP client (jwchat) trying (unsuccessfully so far) to talk to the XMPP server (ejabberd). The web interface is pluggable, and the goal is to use it to enable OpenID services, mesh network connectivity, use of TOR, etc, etc. Not much of this is really working yet, see the project TODO for links to GIT repositories. Most of the code is on github at the moment. The HTTP proxy is operational out of the box, and the admin web interface can be used to add/remove plinth users. I've not been able to do anything else with it so far, but know there are several branches spread around github and other places with lots of half baked features. Anyway, if you want to have a look at the current state, the following recipes should work to give you a test machine to poke at. Debian Wheezy amd64

1. Fetch normal Debian Wheezy installation ISO.
2. Boot from it, either as CD or USB stick.
3. Press [tab] on the boot prompt and add this as a boot argument to the Debian installer:

   url=http://www.reinholdtsen.name/freedombox/preseed-wheezy.dat

4. Answer the few language/region/password questions and pick disk to install on.
5. When the installation is finished and the machine have rebooted a few times, your Freedombox is ready for testing.

Raspberry Pi Raspbian

1. Fetch a Raspbian SD card image, create SD card.
2. Boot from SD card, extend file system to fill the card completely.
3. Log in and add this to /etc/sources.list:

   deb http://www.reinholdtsen.name/freedombox wheezy main
   

4. Run this as root:

   wget -O - http://www.reinholdtsen.name/freedombox/BE1A583D.asc   \
      apt-key add -
   apt-get update
   apt-get install freedombox-setup
   /usr/lib/freedombox/setup
   

5. Reboot into your freshly created Freedombox.

You can test it on other architectures too, but because the freedombox-privoxy package is binary, it will only work as intended on the architectures where I have had time to build the binary and put it in my APT repository. But do not let this stop you. It is only a short "apt-get source -b freedombox-privoxy" away. :) Note that by default Freedombox is a DHCP server on the 192.168.1.0/24 subnet, so if this is your subnet be careful and turn off the DHCP server by running "update-rc.d isc-dhcp-server disable" as root. Please let me know if this works for you, or if you have any problems. We gather on the IRC channel #freedombox on irc.debian.org and the project mailing list. Once you get your freedombox operational, you can visit http://your-host-name:8001/ to see the state of the plint welcome screen (dead end - do not be surprised if you are unable to get past it), and next visit http://your-host-name:8001/help/ to look at the rest of plinth. The default user is 'admin' and the default password is 'secret'.

my tiny last contributions to the perl 5.18 transition around RC bugs, before it hit testing tonight (thanks to the release team for their great work!):

• #720961 src:libimager-qrcode-perl: "libimager-qrcode-perl: FTBFS with perl 5.18: syntax error at inc/Module/Install/XSUtil.pm"
  add patch to fix syntax error, upload to DELAYED/2
• #720973 src:mailagent: "mailagent: FTBFS with perl 5.18: POD issues"
  fix POD syntax in manpage, upload to DELAYED/2
• #720975 src:libdancer-perl: "libdancer-perl: FTBFS with perl 5.18: test failures"
  update build dependencies (pkg-perl)

there are still quite a few bugs left concerning perl 5.18, unfortunately

Yes, it's that time of the year again. A disk in my desktop-replacement laptop with 2 disks and a RAID-1 has died. Time for recovery. This laptop has been running 24/7 for the last 3 years or such, so it's not too surprising that a disk dies. Surprisingly though, for the first time in a long series of dead disks, smartctl -a does indeed show errors for this disk. Here's a short snippet of those:

  $ smartctl -a /dev/sda
  [...]
  Error 1341 occurred at disk power-on lifetime: 17614 hours (733 days + 22 hours)
   When the command that caused the error occurred, the device was active or idle.
   After command completion occurred, registers were:
   ER ST SC SN CL CH DH
   -- -- -- -- -- -- --
   40 41 02 1f c0 9c 40  Error: UNC at LBA = 0x009cc01f = 10272799
   Commands leading to the command that caused the error were:
   CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
   -- -- -- -- -- -- -- --  ----------------  --------------------
   60 f8 08 20 c0 9c 40 00  41d+01:51:50.974  READ FPDMA QUEUED
   60 08 00 18 c0 9c 40 00  41d+01:51:50.972  READ FPDMA QUEUED
   ef 10 02 00 00 00 a0 00  41d+01:51:50.972  SET FEATURES [Reserved for Serial ATA]
   ec 00 00 00 00 00 a0 00  41d+01:51:50.971  IDENTIFY DEVICE
   ef 03 45 00 00 00 a0 00  41d+01:51:50.971  SET FEATURES [Set transfer mode]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed: read failure       90%     20511         156170102
  [...]

The status of the degraded RAID array looks like this:

  $ cat /proc/mdstat
  Personalities : [raid1] 
  md1 : active raid1 sdb7[1]
       409845696 blocks [2/1] [_U]
  md0 : active raid1 sda6[0] sdb6[1]
       291776 blocks [2/2] [UU]

The [_U] means that one of two disks has failed, it should normally be [UU]. There are two RAID-1s actually, a small md0 (sda6 + sdb6) for /boot and the main md1 (sda7 + sdb7) which holds the OS and my data. Apparently (at first at least), only sda7 was faulty and got kicked out of the array:

  $ dmesg   grep kick
  md: kicking non-fresh sda7 from array!

Anyway, so I ordered a replacement disk, removed the dead disk (I checked the serial number and brand before, so I don't accidentally remove the wrong one), inserted the new disk and rebooted. Note: In order for this to work you have to have (previously) installed the bootloader (usually GRUB) onto both disks, otherwise you won't be able to boot from either of them (which you'll want to do if one of them dies, of course). In my case, sda was now dead, so I put sdb into its place (physically, by using the other SATA connector/port) and the new replacement disk would become the new sdb. After the reboot, the new disk needs to be partitioned like the other RAID disk. This can be done easily by copying the partition layout of the "good" disk (now sda after the reboot) onto the empty disk (sdb):

  $ sfdisk -d /dev/sda   sfdisk /dev/sdb

Specifically, the RAID disks/partitions need to have the type/ID "fd" ("Linux raid autodetect"), check if that is the case. Then, you can add the new disk to the RAIDs:

  $ mdadm /dev/md0 --add /dev/sdb6
  $ mdadm /dev/md1 --add /dev/sdb7

After a few hours the RAID will be re-synced properly and all is good again. You can check the progress via:

  $ watch -n 1 cat /proc/mdstat

You should probably not reboot during the resync (though I'm not 100% sure if that would be an issue in practice; please leave a comment if you know). Also, don't forget to install GRUB on the new disk so you can still boot when the next disk dies:

  $ grub-mkdevicemap
  $ grub-install /dev/sdb

And it might be a good idea to use S.M.A.R.T. to check the new disk, just in case. I did a quick run for the new disk via:

  $ smartctl -t short /dev/sdb # Wait a few minutes after this.
  $ smartctl -a /dev/sdb
  [...]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed without error       00%        22         -
  [...]

Looks good. So far.