It's been a long time since I last took some time to read First Monday A great online publication, if you are not familiar with it, that I would categorize (and no, I'm not probably well-informed in it to be authoritative) as dealing with social, psychological aspects of the cultural shifts the online world has brought upon us (often dealing with topics related to Free Software communities, the reason I first met the publication). Firstmonday is an Open Access champion from early on. It follows an approachable but academic format (this means, it is peer-reviewed, its articles give extensive lists of references, and the articles are not the short reads we have got used to finding on the net, but, quoting from their audience profile, English is not the first language of many First Monday readers; A large percentage of First Monday readers are not a part of academia; Cultures, educational backgrounds, and fields of study vary greatly among First Monday readers.) This means, it's at least a great publication for me to follow :) Anyway After a long time not following it, I have just read Reading revolutions: Online digital text and implications for reading in academe, by Barry W. Cull; First Monday, Volume 16, Number 6 - 6 June 2011 Nice, interesting read. As I was planning on telling about the article to a couple of friends more into the subjects than myself, I'll comment+quote some bits on it. Before going any further: The article makes several references to Maryanne Wolf. No relation to her I'm not lulling my (two? are you both still reading?) readers towards her work ;-) The article talks about the differences social, even dips into some physiological aspects that the activity of reading is sustaining due to the shift from an activity done mainly from books (or other similar printed material) to the computer screen. Of course, we all know from our own experience many of the basic traits Shorter attention spans, a different reading pattern (skimming instead of reading; browsing through several related items instead of in-depth reading a single text as a knowledge unit). The article begins with an overview of reading and humankind. Cull quotes Maryanne Wolf's phrase, despite the fact that it took our ancestors about 2,000 years to develop an alphabetic code, children are regularly expected to crack this code in about 2,000 days . An interesting point I never thaught of is the start of reading as a purely mental activity, detaching reason from verbalization, ~1200 years ago:

Interestingly, these early scribes first did their work by reading out loud to themselves. Not until the ninth century did monastic regulations begin requiring silent reading. By the thirteenth century the practice of men reading silently and alone became commonplace. This shift to silent reading was a profound change, one that Darnton suggested involved a greater mental adjustment than the shift to printed text

. One last important point in this older history, that I'm quoting because I know I'll need the reference later on for one of my texts is about reading as a social activity Yes, also related to the quietness I just mentioned:

Interestingly, with the democratization of the printed text, there was a return to reading aloud. Reading was a solitary silent process only for the educated elite who could afford to buy books. For the rest of the population, as Darnton pointed out, reading was a social activity which took place in workshops, barns, and taverns [and] while children played, women sewed, and men repaired tools.

After this introduction (obviously one of the most interesting parts to me), Cull gives numbers showing how reading is evolving (in the USA and Canada), and quotes some prediction on how the future will end up adapting. Of course, I live in a place with a very different society, so I cannot comment much. Then he confronts some studies regarding specifically leisure reading, as it is a much more trustable factor than just literacy (in a world as highly literate as ours is, many people only read when they have to and have never or very seldom experienced the pleasure of reading just for the sake of it), bringing into the discussion the Internet (and computers in general) usage patterns. I found also very interesting the next section, regarding the pattern changes many libraries are facing now, specially academic/research-oriented libraries:

For several millennia, right up until just two decades ago, the central role of a library was to collect and house physical texts: from clay tablets, to scrolls, to printed books (Battles, 2003; Manguel, 2006). While printed text remains essential to most academic libraries, today s libraries have also become a core conduit via which researchers access scholarly texts online. Just within the last few years, Canadian academic libraries, in a situation similar to libraries throughout the Western world, have reached an interesting tipping point librarians now spend the majority of their collections budgets on electronic instead of printed texts.
( )
Libraries are convinced that digital text, now in its infancy, is likely to have a long future. Not only do they purchase electronic texts, but most academic libraries have also become publishers of electronic texts, whether they are digitizing large portions of their book holdings, or focusing on scanning a relatively small number of archival documents from their unique special collections.

And yes, doing some work with our Institute's library, I can confirm this trend. About e-books: I have got quite into that topic since the Kindle won my heart (and my money!) half a year ago. The little device completly changed my reading habits, I have read lots more since I carry it. And yes, I have never considered a full tablet-like device The article talks long about the difference, about the disadvantage that multitasking means to the human brain (oh, do I suffer from it!) I liked this snippet, quoting Steve Jobs a couple of years ago, regarding an Apple e-book reader question:

When Apple was rumoured to be working on an e book reader a few years ago, CEO Steve Jobs expressed his lack of interest: It doesn t matter how good or bad the product is, the fact is that people don t read anymore, he reportedly said, continuing by stating that 40 percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don t read anymore (Markoff, 2008). Nicholas Carr (2010) has summed up Apple s involvement in the tablet phenomenon this way: Jobs is no dummy. As a text delivery system, the iPad is perfectly suited to readers who don t read anymore .

The issue for me is, I do enjoy reading, but I am an information addict. I know that if I have parallel information flows, my attention will surely dilute between them. Of course, as I read this article on-screen, it was hard for me to take the needed discipline not to be distracted by IMs or IRC highlights during the whole reading (which was also as an excercise for myself ;-) ). I am surprised to see this on student preferences (and even more surprised to see this data comes from my university):

In a study of students at the Universidad Nacional Aut noma de M xico (UNAM), the majority of students preferred print, and 63 percent reported that they could bear reading a document on a computer screen for no more than one hour (Ram rez Leyva, 2003). When it comes to course textbooks, a marked student preference for paper over e books has recently been found (Woody, 2010).

As we approach the end, it talks about another important topics I have often tried (and often failed) to communicate to my users: That of paratext, the meaning of the different texts, covers, items, layouts, etc. that are not part of the text itself but do shape the way we face it. To some of us, this seems obvious. To others, it is so hard to understand It closes with two more topics I will refer to. One is the permanent connectivity. All the time, more people are connected virtually all of their waking time. This affects not only learning habits but priorities. Will this near constant access to information interfere with students desire to comprehend and remember information, necessary to the educational process of turning it into knowledge? Author and university business school lecturer Don Tapscott recently suggested that students might not have to stress about the details those you can check Finally, regarding the continuity and ellaboration found in texts that are each time more common He quotes Maryanne Wolf:

I am worried about kids who are immersed in digital culture. They will get to college and they will have been Twittering so much that they won t have the patience to read those really long cognitively convoluted and complex sentences. They may not have developed those rich networks which are required in order to read at a high level of sophistication. The effort is what we are going to lose. They are becoming not so much a lazy reader, but an atrophied reader

As you can see, not only this post is meant to tell my couple-of-interested-friends to read the article, but it's mainly meant as a mental placeholder for myself. I will be surely refering to some of these items.

If you try to move a git submodule with git mv, you ll get the following error message:

$ git mv old/submodule new/submodule
fatal: source directory is empty, source=old/submodule, destination=new/submodule

There s a patch against git to supoort submodule moving, but it doesn t seem to be applied yet, at least not in the version which is currently in Debian Sid. What worked for me to solve this issue was the following (as posted on StackOverflow):

1. Edit .gitmodules and change the path of the submodule appropriately, and put it in the index with git add .gitmodules.
2. If needed, create the parent directory of the new location of the submodule: mkdir new.
3. Move all content from the old to the new directory: mv -vi old/submodule new/submodule.
4. Remove the old directory with git rm --cached old/submodule.

Looked like this for me afterwards:

 # On branch master
 # Changes to be committed:
 #   (use "git reset HEAD <file>..." to unstage)
 #
 #       modified:   .gitmodules
 #       renamed:    var/lib/dokuwiki/tpl -> var/lib/dokuwiki/lib/tpl
 #

Finally commit the changes. HTH.

"kinneko" reported the Debian bug #630000 on Friday June 10th, against.....debian-installer (first time this is against a package I participate to). First time, also, that we don't know the exact real name of the person reporting the bug. Bug #620000 was reported as of March 29th. That again makes 2.5 months for 10,000 bugs, roughly the same time than 610000-620000. And now I need to decide about reassigning that bug report as it either pertains to the D-I "netcfg" component...or to busybox (the DHCP client in D-I is provided by busybox).

I was travelling on a train today, and got a seat at a table, with a power socket. I decided to do some coding to pass the time, and picked the task of measuring the speed of summain, my tool for generating manifests of directory trees. Think md5sum or sha1sum, but printing out much of the inode information: the goal being that if I can diff manifests before a backup and after a restore and see that everything is OK. So I wrote a little script to time any checksummer for a particular set of files, and got the following results:

113 ./summain
112 md5sum
114 sha1sum
121 sha512sum

It doesn't look like I need to actually optimize it at all: it's already as fast as md5sum and sha1sum, even though it's written in pure Python. (That's cause it uses the Python standard library module hashlib, which is, of course, pure C.) Here's a sample output from summain, for the curious:

Name: README
Mtime: 2011-02-13 09:59:09.000000 +0000
Mode: 100644
Ino: 1
Dev: 1
Nlink: 1
Size: 1256
Uid: 1000
Username: liw
Gid: 1000
Group: liw
Sha-1: b7fa5517ea20311d13128312456c0dba3da3b11d

Luckily I had some other things I could work on, so the rest of the train ride wasn't too boring.

Looking through the man page of find there is no obvious way to find broken symbolic links. But there is a simple way involving only find:

$ find -L . -type l
$ find -L . -type l -ls

The option -L (before the path!) causes find to follow symbolic links and the expression -type l causes find to report only symbolic links. But as it follows symlinks, it only reports those it can t follow, i.e. broken ones. The second line also shows where the broken links point to. To easily show that this really works, just use the color indicator of GNU ls instead of find s builtin -ls:

$ find -L . -type l -exec ls -lF --color=yes ' ' +

Et voil , all displayed links show up in red which means they re broken. Kudos to CodeSnippets for showing me the right idea. And thanks to ft of zsh and grml fame for the hint about find -exec command + instead of find -exec command ;. Hint from mika of grml fame: With zsh it is even less code to type:

% ls **/*(-@)
% ls -lF **/*(-@)

Thanks, mika!

Philipp is a Debian developer since 2005 and a member of the release team since 2008. Since he took the responsibility of Stable Release Manager, the process has evolved for the best. I asked him to explain how the release team decides what s fit for stable or not. His work on the buildd infrastructure is also admirable but I ll let him describe that. My questions are in bold, the rest is by Philipp. Who are you? I m a 24 year old Computer Science student, living in Karlsruhe, Germany. As a student assistant I m currently in two jobs: one is taking care of getting our university IPv6 network in shape, or rather generally getting fringe technologies into the network. The second is taking care of a s390x machine in the basement which my faculty got sponsored recently. In my spare time I tend my Debian duties and I m active in the student council (Fachschaft) as a sys-admin, software developer and until recently as treasurer. I got accepted as a Debian Developer in 2005. I m only really active since I was invited to join the Release Team in early 2008 after I contributed rewrites of some scripts that got lost in a disk crash of ftp-master. In late 2008 I also took on wanna-build/buildd duties. What s your notable achievement within Debian? For one I worked on the deployment of a single buildd/sbuild combination over all of our buildds and I rebased it on the sbuild in the archive several times already. All buildds basically look the same on all architectures, with only few variations. The chroots are now also created in a mostly predictable manner. Then we finally got build autosigning after years of constant poking. However the policy decision to allow it was made by the ftp-masters anyway, for which I m grateful, as it eases the workload of the buildd maintainers, the Security Team and the Release Team quite a bit. On the release side there s the integration of volatile into the proper structures of Debian. But there I m guilty for choosing the bad name squeeze-updates is, in comparision with security s squeeze/updates. Let s see if we can improve that for wheezy. The policy for stable updates have changed over time. Can you summarize what kind of updates are allowed nowadays? All the changes that were made to the policy were driven by the aim to keep stable (and to some lesser degree oldstable) usable throughout its lifetime. I have to admit that we got slightly more liberal in what we accept since I took over. The previous stable update policy included the fixes of critical bugs that break the system in interesting ways and security fixes. We also opened up the possibility of including important fixes for annoying bugs on a case-by-case basis. The whole don t update that much part of the stable release management is rooted in let s don t change the behaviour of a running system and let s have as few regressions as possible . We currently try to counter that with a review process that only allows self-contained fixes that were tested in unstable first, if applicable. In the future I d also like to have a feedback process that includes the users of the packages to report problems more early. In fact that s also why we now send calls for testing to debian-stable-announce one week in advance (see this mail as an example). So stable is updated through point releases that happen roughly every two months for stable and roughly every four for oldstable. They are accompanied by announcements and new CD/DVD sets. To be able to push some updates with less delay to our users and to avoid that they have to pick them from proposed-updates, there is stable-updates. The current policy for this suite is available in this post on debian-devel-announce. It boils down to everything urgent like tzdata, regression fixes and volatile packages. We noticed that some packages in stable just weren t useful anymore when they got updated through volatile. Hence those are folded into stable, too. We also relaxed the rules a bit so that leaf packages like clive, that rely on external websites not to change their URLs, can be updated too. If somebody wonders what happened to the and a half releases Those were intended to mark the inclusion of new hardware support (mostly by including a co-installable new kernel version). This kind of flag point release is thankfully no longer needed because our marvellous kernel team now backports certain hardware drivers to the kernel version in stable. They have some trouble soliciting test feedback for them, though. It would be cool if more people using stable could respond to their calls for testing. What are your plans for Debian Wheezy? Luckily the wanna-build/buildd side isn t much dependent on release cycles. The stable release management also just takes care of what gets dumped on them by the testing RMs. The near-term goals are certainly:

• getting d-i to use some development process in the archive that s compatible with the normal flow of packages into testing and
• hopefully getting wanna-build to use a sane SQL layout and hence better performance

Apart from that my work is mainly squashing the bugs on buildd.debian.org for the wanna-build part, so if you want something fixed, you need to report it, I m afraid. You have several roles within Debian, in particular stable release manager and member of the wanna-build team. Which role do you enjoy more and why? So the regular duties of the SRM, apart from setting up policies, are reviewing requests for uploads to proposed-updates, accepting them, chasing builds and scheduling point releases. It s mainly steady work. As for the Release Team it s working in a fabulous team where everyone s doing an awesome job. For me the wanna-build role is holding the things together and fixing up stuff that breaks while the world continues to evolve. And occassionally taking a stab at the ftp-masters. So rather different. My heart is probably siding with the buildd side because it s infrastructure but release work is fun, too. What s the biggest problem of Debian? Mainly that we re not quick in picking up new technologies and that we got slow on innovation. Distro-wide changes are usually a very big effort to coordinate and to finally complete. I guess people should be able to push fixes for release goals more eagerly into the archive. Also it seems to me that many DDs are actually disengaging from the release process, especially during the freeze but also before, which makes me a bit sad. However I m still not sure how we could counter that. As we re all volunteers you can t suddenly make others love the idea of releasing something together, I guess. Do you have wishes for Debian Wheezy? It s getting old, but there s certainly multiarch. Also a usable Python 3 is something I d really appreciate. Is there someone in Debian that you admire for their contributions? Peter Palfrader did an awesome job of getting a nicely working team together for DSA and got many tasks as automated as they are now. It s a pleasure to work with them and setting up a new buildd is, for example, a real breeze. He also developed a sane archive snapshot service, kudos for that.

---

Thank you to Philipp for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

Earlier this year, I agreed to join the board of Eben Moglen's FreedomBox Foundation, and to chair the recently announced Technical Advisory Committee. To date, most of the time I've been able to personally invest in the foundation has gone towards work "behind the scenes", all of which was necessary, but little of which is worthy of external report or attention. I now believe it's time to take a more active role in communicating what we're actually trying to do, and how I think we can get there. To that end, I intend to be more visible in discussions on the freedombox-discuss mailing list. I'm also accumulating a list of interesting technical challenges that I'll articulate here over time, along with status reports I believe are worthy of broad dissemination.

A random co-worker was looking for someone to look after her son's pet gecko for 5 weeks while they went to Hawaii, and I thought it'd be something fun and non-permanent to do, so we now have a female 1 year old Afghan leopard gecko named Speck, and a large supply of crickets in our possession. I note that the Wikipedia page is lacking a photograph. We might have to try and fix that...

This is a very special person. He is a very well known songwriter, at least in German language countries because he sings in German. He was that special kind of person with his lyrics when I was still a kid, and is still around continuing to write his songs in his very own special way. This person is Reinhard Mey, and if you understand German and have missed him so far, you have missed a lot. The songs that I present to you are special in the way that they are all contained in the special compilation titled Mein Apfelb umchen. The dedication he wrote for the album is also very special:

Ich glaube, Kinder zu haben ist das aufregendste Abenteuer, das wir erleben k nnen. Es ist der schwerste Beruf und die gr te Herausforderung, die ich mir denken kann, und die gl cklichste Erfahrung zugleich. Ich bin dankbar daf r! Dies sind die Lieder, die ich bis heute daf r geschrieben habe. Mein Anteil aus dem Erl s dieser Schallplatte gebe ich der Hilfe f r krebskranke Kinder.

Rough translation: I believe that having kids is the most exciting adventure that we can undergo. It is the hardest job and the biggest challenge that I can think of, and at the same time the happiest experience. I'm thankful for it! These are the songs that I wrote up to today for it. My part of the revenues of this record go to Help for children with cancer. So here are the songs:

• Mein Apfelb umchen: The song that gave this compilation its title. Extremely touching, and it manages regularly to wet my eyes... Absolutely lovely.
• Keine ruhige Minute: This is actually a live version of the song with a longer introduction that is worth listening to on its own.
• Menschenjunges: A thoughtful song about the thoughts when seeing your kid for the first time.

Enjoy! And if you feel like it, support these kind of special people.

/music permanent link Comments: 2 Flattr this

I have a WRT-54G. I've had it for some years. It's run a bunch of different firmware variants over that time, but they've all had something in common. There's no way to configure IPv6 without editing text files, installing packages and punching yourself in the face repeatedly. Adam blogged about doing so today, and I suspect he may be in need of some reconstructive surgery now.

I spent yesterday looking at disassembled ACPI tables and working out the sequence of commands the firmware was sending to the hard drive. I'm planning on spending tomorrow writing x86 assembler to parse EFI memory maps. I spend a lot of time caring about stupidly awkward implementation details worked out from staring at binary dumps. The last thing I want to do is have to spend more than three minutes working out how to get IPv6 working on my home network because that cuts into the time I can spend drinking to forget.

Thankfully this is the future and punching yourself in the face is now an optional extra rather than bundled. Recent versions of Tomato USB (ie, newer than actually released) have a nice web UI for this. I registered with Tunnelbroker.net, got a tunnel, copied the prefix and endpoint addresses into the UI, hit save and ever since then NetworkManager has given me a routable IPv6 address. It's like the future.

Because I'm lazy I ended up getting an unofficial build from here. The std built doesn't seem to include IPv6, so I grabbed the miniipv6 one. The cheat-sheet for identifying builds is here. And I didn't edit a single text file. Excellent.

I say this after probably 48 hours of what I estimate is a sustained 60 kB/s of brute force SIP traffic from 61.146.178.173 has been sent my way. I'd very cheerfully sign up for a service that had all Chinese IP addresses null routed, and provided a proxy server for any HTTP access. I have no business needing direct IP connectivity to China, and I certainly do it not want it from China to me. I was pleasantly surprised to be able to call up Comcast's Business Class technical support tonight and ask them if they could null route the above IP address, and they seemed to imply they could (they opened up a ticket for a level 2 person to do something at least). Now if only I didn't have to pick up the phone to interact with Comcast's Business Class technical support, it'd be just lovely. I'm just glad I'm not in Australia where this would be blowing my monthly quota and/or causing me to receive excess usage charges. Bloody UDP. You can firewall it off, but it keeps coming. My initial stopgap Netfilter rule counted 1.5G of traffic before I replaced it with an adaptation of my SSH brute force mitigation rules, and the new rule has seen nearly 5G of traffic now. It's one thing to try to brute force something, but to keep trying after you stop getting responses is just plain stupid.

I've finished importing the usenet archive for oldusenet. The fun part was parsing the dates to put the posts in order. No date format was really required on usenet, and so a wide variery of formats were used. Some posts didn't have a Date, but a guess could be made from their Message-ID. Some posts had absurd dates (ie, 1969, 1995), others had dates that were correct in every way.. except the year was left out (oops). One early post had a date of "_". Still, this excerpt of my code managed to parse the rest and so gives a fairly complete picture of how messy dates can possibly be. Read and weep.

  p anyzone "%d %b %y %T"       "15 Jun 88 02:27:41 GMT"
, p anyzone "%a, %d %b %y %T"       "Thu, 22 Jun 89 20:02:03 GMT"
, p anyzone "%a, %d-%b-%y %T"       "Thu, 15-Jun-89 18:01:56 EDT"
, p anyzone "%d %b %y %T"       "8 Jan 90 14:07:27 -0400"
, p anyzone "%d %b %y %H:%M"        "4 Oct 89 19:56 GMT"
, p anyzone "%a, %d %b %y %H:%M"    "Thu, 23 May 91 02:13 PDT"
, p anyzone "%a, %d %b %Y %T"       "Thu, 23 May 1991 07:07:00 -0400"
, p anyzone "%a, %d %b %Y %H:%M"    "Sat, 18 May 1991 17:28 CDT"
, p anyzone "%d %b %Y %T"       "11 Apr 1991 12:02:01 GMT"
, p anyzone "%d-%b-%y %H:%M"        "24-Mar-90 14:22 CST"
, p anyzone "%d %b %y, %T"      "22 May 91, 16:31:37 EST"
, p anyzone "%d %b %Y %H:%M"        "30 June 1991 17:15 -0400"
, p anyzone "%a, %d %b T  %T"       "Fri, 8 Feb T  09:49:39 EST"
-- special cases
, p (tzconst est) "%a %b %d %T EST %Y"  "Tue Jan 11 12:44:36 EST 1983"
, p (tzconst est) "%a %b %d %T EST %y"  "Tue Jan 11 12:44:36 EST 83"
, p (tzconst edt) "%a %b %d %T EDT %Y"  "Tue Jan 11 12:44:36 EDT 1983"
, p (tzconst edt) "%a %b %d %T EDT %y"  "Tue Jan 11 12:44:36 EDT 83"
, p (tzconst utc) "%a %b %d %T GMT %Y"  "Thu Nov  1 23:14:37 GMT 1990"
, p (tzconst pdt) "%d %b %y %T -7"  "11 Jun 91 15:41:21 -7"
-- dates with no timezone specified are guessed
, p nozone "%d %b %y %T"        "9 Jan 90 09:33:59"
, p nozone "%d %b %Y %T"        "10 APR 1990 05:25:28"
, p nozone "%a %b %d %T %Y"     "Fri Feb  6 00:19:47 1981"
, p nozone "%a %b %d %T %y"     "Fri Feb  6 00:19:47 81"
, p nozone "%Y-%m-%d %T"        "1981-11-12 18:31:01"
, p nozone "%y-%m-%d %T"        "81-11-12 18:31:01"
, p nozone "%a, %d %b %y %T"        "Sat, 13 Apr 91 08:37:57"
, p nozone "%a, %d %b %Y %T"        "Sun, 16 Jun 1991 13:23:02"
, p nozone "%d %b, %Y %T"       "1 May, 1991 00:00:00"
, p nozone "%d %b %y %H:%M"     "8 Jan 88 18:03"
, p nozone "%a, %d %b %y %H:%M"     "Wed, 29 May 91 17:14"
, p nozone "1 %b %d %T %Y"      "1 Jan 08 20:59:08 1991"
-- this has to come near the end, as it matches greedily
, g nozone "%a %b %d %T %Y ("       "Wed Oct 27 17:02:46 1982 (Tuesday)"
, g nozone "%a, %d %b %y %T +"      "Tue, 21 May 91 16:46:01 +22323328"
-- extract date from message-id headers
-- (used for messages with no Date field)
, g nozone "<%Y%b%d.%H%M%S."        "<1989Jul6.214048.28313@jarvis.csri.toronto.edu>"

(Parsing the often ambiguous, malformed, etc timezones was fun all its own too, of course.)

Gearman is a distributed queue with several language bindings.

While Gearman has a nice Python implementation (python-gearman) of the client and worker, I chose to use the libgearman bindings (python-libgearman) directly since they are already packaged for Debian (as python-gearman.libgearman).

Unfortunately, these bindings are not very well documented, so here's the sample application I wished I had seen before I started.

Using the command-line toolsBefore diving into the Python bindings, you should make sure that you can get a quick application working on the command line (using the gearman-tools package).

Here's a very simple worker which returns verbatim the input it receives:

gearman -w -f myfunction cat

and here is the matching client:

gearman -f myfunction 'test'

You can have have a look at the status of the queues in the server by connecting to gearmand via telnet (port 4730) and issuing the status command.

Using the Python libgearman bindingsOnce your gearman setup is working (debugging is easier with the command-line tools), you can roll the gearman connection code into your application.

Here's a simple Python worker which returns what it receives:

#!/usr/bin/python

from gearman import libgearman

def work(job):
   workload = job.get_workload()
   return workload

gm_worker = libgearman.Worker()
gm_worker.add_server('localhost')
gm_worker.add_function('myfunction', work)

while True:
   gm_worker.work()

and a matching client:

#!/usr/bin/python

from gearman import libgearman

gm_client = libgearman.Client()
gm_client.add_server('localhost')

result = gm_client.do('myfunction', 'test')
print result

This should behave in exactly the same way as the command-line examples above.

Returning job errorsIf you want to expose to the client errors in the processing done by the worker, modify the worker like this:

#!/usr/bin/python

from gearman import libgearman

def work(job):
   workload = job.get_workload()
   if workload == 'fail':
       job.send_fail()
   return workload

gm_worker = libgearman.Worker()
gm_worker.add_server('localhost')
gm_worker.add_function('myfunction', work)

while True:
   gm_worker.work()

and the client this way:

#!/usr/bin/python

from gearman import libgearman

gm_client = libgearman.Client()
gm_client.add_server('localhost')

result = gm_client.do('myfunction', 'fail')
print result

LicenseThe above source code is released under the following terms:

To the extent possible under law, Francois Marier has waived all copyright and related or neighboring rights to this sample libgearman Python application. This work is published from: New Zealand.

I was laid off a couple of months ago, and have been thinking about what I want to do next. Actually, I've mostly been writing code for my pet projects, but earning a living again is occasionally on my mind. I would, very much, like to earn a living while deveoping interesting free software that I would like to use myself. There's a bunch of options available:

• I could go work for one of the large companies that use free software, and hope to find a position where I can help develop the tools they use. This would perhaps be the easiest option.
• I could perhaps find a job at a smaller company that specilizes in the development of some parts of the free software stack, or develops a Linux distribution.
• I could join or found a startup that develops free software and thinks it has a business model around it.
• I could learn web programming, and develop interesting online services that people would be willing to pay for.
• I could do consulting gigs, either via my own company (back in Finland), or by getting employed by one of the free software oriented consulting houses that exist. Again, I might need to learn some new skills, since most customers are looking for work in parts of the development stack I'm not experienced in, such as the kernel, or want web or mobile app development, neither of which I know.
• I could try to crowd-fund development of software I think the world needs, and hope to get paid enough to make a living. This would be a special case of doing consulting: I would be selling very small amounts of work to a fairly large number of customers.

In this blog entry, I want to think about that last option. Here's I imagine it would work: I would post a list of things I would like to do, estimates of how much work they'd be, and a hourly price. People would tell me how many hours of work they'd like to pay for. I would then do the work, send everyone their bill, and they'd pay it. In the ideal world, this would be an awesome way to fund development. I could, for example, write some automated tools for system testing of Debian, to help with release management, or I could finish Obnam. And that's just two things I'm already working on: there's no end of things I would like to develop, if I didn't need to worry about my next meal. However, this is not without problems. To start with, it might look like I'm trying to blackmail the world: "pay me, or this software won't be written". That's not a problem in the general case, but it'd easily look like that if the work would be to, say, maintain Debian packages. As a Debian developer, I should keep my packages in good shape whether someone is paying me or not. I would need to be careful to keep things ethical here. Another problem would be the issue of being realistic: would there be enough people willing to pay me, and how long would that last? That partly depends on the pricing. Although my needs are fairly modest, I do live in a country that is fairly expensive to live in. If I started doing this, it'd be easy for someone in a country it is cheaper to live in to compete on price (and they might well be better than I am, too). Then again, since there's no end of things to do, perhaps we could both make a living. Would you pay someone to develop free software by the hour? How much? How often? What should the software do?

Just added a TDeb test build branch for multistrap to test out the changes in the next version of the emdebian-tdeb binary package - specifically:

Move em_installtdeb into the emdebian-grip-server package so that the emdebian-tdeb package can be slimmed down to only contain what will become useful for other maintainers to use in their debian/rules files and for translators when the time comes to implement DEP-4. CDBS rules / patches to follow... bug reports against dpkg and debhelper to follow ...

DEP-4 itself is being restarted and refreshed but with some problems on alioth currently, I'm going to (try and) keep the document up to date via SGML and people.debian.org.

The changes are now in Emdebian SVN and I'm starting testing with my own packages first. Multistrap for PO and po4a-build testing, dpkg-cross for debconf testing, some others for testing with build systems other than CDBS, then the discussions will restart on debian-devel and I'll upload the TDeb support scripts to unstable.

There are a variety of things which need to happen before these scripts become useful, including:

• dpkg support for XC-Package-Type: tdeb
• debhelper support for dh_gentdeb,
• inevitable bug fixing in emdebian-tdeb

Why is the package called emdebian-tdeb? Simple - that's where the scripts have lived until now and the hope is that dpkg-gentdeb can migrate into dpkg-dev and dh_gentdeb into debhelper - eventually.

By all means play with the scripts and follow the example multistrap branch above to see what kind of changes might be needed in your own packages. I'll post again when I've worked out the changes needed in packages like dpkg-cross which only need TDebs for debconf support. (This is the easiest mode of TDeb usage but it's also the most useful during a release cycle - maintainers make a single change and then a nominated i18n boss-person - take a guess who - can update the all your debconf translations in one upload without a binary build - eventually possibly not even needing any bug reports, just a simple email advance warning.

What I've got to work on still is how the generation of an updated TDeb creates a new debian diff / debian.tar.gz to accompany the upload. For that I need XC-Package-Type: tdeb support in dpkg....

Sample output from the viewpoint of a translator updating a package already converted to support a TDeb:

neil@sylvester:tdeb-test$ dpkg-gentdeb -t

Set the changelog message to version 2.1.15+t0 first, closing any relevant bugs!

 dch -v 2.1.15+t0 

neil@sylvester:tdeb-test$ dch -v 2.1.15+t0 TDeb test

neil@sylvester:tdeb-test$ dpkg-gentdeb -t

Building TDeb update: 2.1.15+t0

 (192 entries)

Discard doc/pod/1/da/multistrap (65 of 177 strings; only 36.72% translated; need 50%).

Discard doc/pod/1/da/device-table.pl (5 of 19 strings; only 26.31% translated; need 50%).

Processing untranslated files for pod/multistrap (1) . . .

Processing untranslated files for device-table.pl (1) . . .

Processing de translations for pod/multistrap (1). . .

Processing de translations for device-table.pl (1). . .

Processing fr translations for pod/multistrap (1). . .

Processing fr translations for device-table.pl (1). . .

Processing pt translations for pod/multistrap (1). . .

Processing pt translations for device-table.pl (1). . .

dpkg-gencontrol -pmultistrap-tdeb -Pdebian/multistrap-tdeb -cdebian/control

dpkg --build debian/multistrap-tdeb ../multistrap-tdeb_2.1.15+t0_all.tdeb 

dpkg-deb: building package  multistrap-tdeb' in  ../multistrap-tdeb_2.1.15+t0_all.tdeb'.

neil@sylvester:tdeb-test$ dpkg -c ../multistrap-tdeb_2.1.15+t0_all.tdeb

drwxr-xr-x root/root         0 2011-06-08 21:06 ./

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/pt/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/pt/LC_MESSAGES/

-rw-r--r-- root/root      6427 2011-06-08 21:06 ./usr/share/locale/pt/LC_MESSAGES/multistrap.mo

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/vi/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/vi/LC_MESSAGES/

-rw-r--r-- root/root      6963 2011-06-08 21:06 ./usr/share/locale/vi/LC_MESSAGES/multistrap.mo

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/fr/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/fr/LC_MESSAGES/

-rw-r--r-- root/root      6789 2011-06-08 21:06 ./usr/share/locale/fr/LC_MESSAGES/multistrap.mo

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/da/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/locale/da/LC_MESSAGES/

-rw-r--r-- root/root     16311 2011-06-08 21:06 ./usr/share/locale/da/LC_MESSAGES/multistrap.mo

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/pt/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/pt/man1/

-rw-r--r-- root/root     33963 2011-06-08 21:06 ./usr/share/man/pt/man1/multistrap.1

-rw-r--r-- root/root      4471 2011-06-08 21:06 ./usr/share/man/pt/man1/device-table.pl.1

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/vi/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/fr/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/fr/man1/

-rw-r--r-- root/root     33969 2011-06-08 21:06 ./usr/share/man/fr/man1/multistrap.1

-rw-r--r-- root/root      4708 2011-06-08 21:06 ./usr/share/man/fr/man1/device-table.pl.1

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/da/

drwxr-xr-x root/root         0 2011-06-08 21:06 ./usr/share/man/da/man1/

I'm also going to DebConf11



On the RoadTrip! all the way hrough France, Belgium, Netherlands, Germany, Austria, Slovenia, Croatia, Bosnia-Herzegovina (Banja Luka) in an MX5.

I've got a lot of things to talk to people about @ DebConf this year... you won't be able to tell if I'm just buying you a beer or trying to badger you about your packages!! Just note that TDebs have been already agreed (in terms of how TDebs work in the archive) with ftp-master at a previous meeting in Extremadura and I'm being badgered by a Release Manager and an ex-DPL to get TDebs implemented real soon now, so it is going to happen and hopefully in time for Wheezy - it's just a matter of fixing the bugs. The quicker everyone has a look at the new scripts (which have been completed re-written and immensely simplified since I first tried a dumb conversion of em_installtdeb) and the likely changes needed in packages, the quicker I can get the bugs out of the scripts.

More documentation to follow, I'll be updating the SGML whenever practical and the actual DEP just as soon as I get commit access back.

I just uploaded gdebi_0.8~exp1 into experimental.
This release adds Michael Vogt s support for GTK3 (Thanks a lot, Michael!): Testers are welcome, expecially if you are trying GNOME3 already. Feel free to report any bug you find!

It s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions. Our first milestone The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old. I m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them. One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project. This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian. Conclusions Although we weren t quite able to clear the whole list, we still consider the project to be a success because:

• We ensured that all of the patches received due consideration for inclusion in Debian
• We proved the concept of DEX, with developers from Debian and derivatives cooperating on a common goal and sharing tools
• Most importantly, we learned from the experience

What s next In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX:

• Our second major project, nicknamed big-merges , will begin soon. Our goal is to identify the few packages which are most diverged between Debian and Ubuntu, and work to get them as close to identical as possible. If you have suggestions for packages to focus on, let us know!
• Allison Randal is beginning a DEX project to implement the Python 2.7 transition across Debian and Ubuntu
• Nathan Handler is working on a Summer of Code project to develop specialized tools to help with this kind of cross-distribution teamwork
• Zack is organizing a derivatives BoF at DebConf 11

We re looking forward to seeing DEX develop further. If you d like to get involved, come and join us on the debian-derivatives mailing list or IRC (#debian-derivatives on freenodeOFTC). Matt Zimmerman and Stefano Zacchiroli

The world of augmented reality is quite fashinating.

I recently got interested into Natural Interaction due to the popularity of Microsoft's Kinect and decided to package OpenNI for Debian.

OpenNI is a framework aiming to making natural interaction transparent to the user/developer, publishing a unified API and abstracting the access to different sensors and modules. Currently it supports some Asus sensors and there is a branch on github supporting also Kinect (which is the one we're packaging).

Some cool projects are already using OpenNI: ROS, PointClouds and OpenCV come to my mind immediately.

We'are working on OpenNI package in the Multimedia Maintainers group on Alioth, the package is still not ready to be uploaded to unstable and upstream has some major problems like the lack of SONAME in its produced library, so some communication with upstream is needed but for preliminary use the packages are in a decent state: OpenNI, PrimeSense Sensor, NITE (non-free).

NITE is a closed source module set for advanced body and scene recognition, unfortunately PrimeSense didn't opensource them, thus I decided to use the same approach of flashplugin-nonfree and write a downloader.

For the sake of completeness, unstable already has libfreenect which purpose is similar but specifically oriented to Kinect.

I am studying both and really hope that in the next future some open source project will make (at least) skeleton and hand recognition available under an open license. People are already working on it, according to Google.

Disclaimer: There is a huge discussion going on right now in the debian-devel mailing list, about different ways to rearrange how Debian development happens, and maybe to provide a Debian users with an additional release. This blog entry is not a comment on that discussion, which I am ignoring. The discussion did, however, prompt a dent about "test driven development". That prompted a short exchange of thoughts with Tom Marble, and this blog post is a cleaned up version of my part in that discussion.

Currently Debian development happens roughly like this: all packages are uploaded to a part of the Debian archive called unstable. Once they've been in use for a while without serious problems, they get automatically copied into another part called testing. Every year or two, testing is frozen and any remaining problems are fixed, after which all of testing gets copied into yet another part of the archive called stable, and that's the actual release. There is a little bit of automatic testing, but only a little. Almost all testing is done by people, by them using unstable or testing on their usual computers. If they have problems, they report them. Contrast this with development using Test Driven Development, or TDD, and other modern development methodologies. Here's a rough summary of what I do when I write (much) of my software.

• first write one or more automatic tests (unit or functional ones)
• write the actual code, enough to make all tests pass
• add more tests
• write more code, or change existing code
• repeat this until tests describe all the behavior desired of the code

In addition, I measure coverage to make sure all parts of the code gets tested. I usually aim for 100% coverage, except for those parts that very hard or quite pointless to test. (That's easier to achieve than you'd think, but that's the topic of another blog post.) This all sounds like a lot of bureaucratic nonsense, but what I get out of this is this: once all tests pass, I have a strong confidence that the software works. As soon as I've added all the features I want to have for a release, I can push a button to push it out. (Well, actually there's a little bit more to it.) This does mean I never have bugs in my software. Of course I do. However, there's a lot fewer of them. In fact, there's so few of them, after tests pass, that I would almost be happy to make an automatic release after every successful run of "make check". Another aspect of the way I do development is distributed version control. The relevant feature is powerful branching and merging. Most things I develop happen in short-lived single-purpose branches: whenever I start work on a feature or bugfix, I create a branch for it. (Unless I'm feeling particularly lazy, which happens more often than you'd think.) When I've finished the feature, or fixed the bug, I merge the branch back into the trunk branch. This way, the trunk is always in releaseable state. It might not have all the features I want for the next release, but the features that are there, do work. The branch probably has bugs, but if I've written tests that are good enough, I know the software works well enough. Or that's the idea. Sometimes things go wrong. Then I write more tests and the next time goes better. (It doesn't have to be perfect, as long as it gets better every time.) See the contrast between automatic tests with good coverage, and the Debian style of relying on user feedback? There's no need to wait for user feedback with automatic tests. This speeds up development, makes releasing easier, but most importantly takes away any reason to fear making changes, even big changes. Automatic tests and good test coverage are easy achieve in small projects. For a system as huge as Debian, good test coverage is quite hard to achieve. The thing about automatic tests, though, is that even a little bit of it is helpful, and after you have a few tests, it gets easier to add more. The first test is the hardest to get done, since you need to set up all the infrastructure for it. Debian does do a bit of automatic testing already. (See lintian, piuparts, autopkgtest, edos, etc.) I don't want to belittle that, but I think we could do better. Here's what I would love to see:

• we have a part of the archive that corresponds to a trunk branch, i.e., it is always in a releasable state (the "testing" area was originally meant to be that; we could make it so now)
• releaseable state is determined by two things:
  • an automatic test suite
  • user feedback, particularly in the form of bug reports (as now)
• whenever changes are made, they happen in "branches"
  • the branch is not affected by changes elsewhere in the archive, except by manual synching ("merge from trunk")
  • individual package uploads, as well as groups of packages such as for transitions, are each in their own branch
  • this is sort of similar to a PPA, but more powerful
• when a branch is to be merged into trunk, the automatic tests must first pass (or at least no new failures in them can be introduced)
  • tests can also be run for any other branch, of course, so that those developing the branch know if they're ready to push their changes into trunk
• there's a culture of writing tests for bugs (whenever that is feasible), and for new features
  • particularly release goals should be expressed as automatic test suites
• there's a culture of sharing tests with upstreams, and with other distributions

Since full test coverage is going to be impossible for Debian, some subset should be targeted. Perhaps something like this would suffice to start with:

• a version of debian-installer is generated from a branch
• test installation of a new system from scratch
  • with a large set of tasks selected
• test upgrade from previous stable release to the branch
  • with a large set of packages installed (as many as possible, actually)
• test upgrade from trunk branch to branch to be merged
  • ditto large set of packages
• test with lintian, reject on specific tests failing
• test with piuparts
• test the whole system for specific functionality
  • ssh access from outside
  • sudo access for a logged in user
  • sending mail with SMTP
  • web server
  • possibly test specific web applications
  • possibly test Samba and NFS services
  • possibly test printing services (CUPS)
  • possibly test essential desktop functionality (automatic login, at least)
• if package comes with package specific tests, run those too

These tests would not guarantee that a set of changes would not break Debian, but they would give a high confidence that at least basic stuff would still work after the changes. Now, obviously implementing all of this I'm dreaming of (it is just past midnight, after all) is going to be impossible. There's way too much work to do, there's not enough tools for writing tests, and it would require too much computing power to run, and so on and so forth. But it's late, and I've had a bad day, and I might as well dream of something nice. Anyway, anyone interested in these things should perhaps help drive DEP8.

Today is World IPv6 Day. Please take a moment to test your connectivity, and if you are not IPv6-enabled yet, then send an e-mail to your provider or hoster and ask them for native IPv6 connectivity on your uplink. Do it even if you do not know what I am talking about or you don t care. The reason is quite simply that we re already too late and hence should act without further delay. If IPv6 network effects do not pick up and adoption rate increases, the big players will drive up the prices for everyone. Then you will find yourself locked in and paying. Or you simply won t be able to address individual computers anymore but always be forced to proxy via commercial providers and forced to say how high when they ask you to jump. Remember that they are commercial entities who might claim to act in the interest of their customers, but you are actually second to their profits. Here are some answers to frequently asked questions if you want to know more. PS: Google, having been so vocal about World IPv6 Day, I would have at least expected you to change your logo today! NP: Monkey3: 39 Laps