Search Results: "Martin"

4 October 2011

Michael Banck: 4 Oct 2011

Woodchuck and FrOSCon

At the end of August, I attended FrOSCon in Bonn again, after skipping it last year. The evening before FrOSCon however, I visited Neal Walfield, his wife Isabel and their little son Noam in D sseldorf. Besides having a great time and a lovely dinner, I was most impressed by their collection of Maemo devices (they had at least two N770s, an N900 and, to my jealousy, an N950) which Neal is doing research on these days. He works on woodchuck, which is a project investigating how to improve data availability on mobile devices and our conversation prompted him to implement ATP Woodchuck, which makes smarter decisions when to run APT upgrade on your Maemo device then the standard updater. As part of the research, they also run a user behaviour study which I joined, where one installs a client which records various data off your N900 and sends them anonymized (he seems to be doing a good job at that) to figure out how people use their mobile devices and hopefully enhance the experience. So if you have a N900, you should consider joining the study so they get better data.
The next day, I picked up Martin Michlmayr nearby and we headed for FrOSCon. I was quite impressed by the Makerbot at the Tarent booth, but I still don't know what they are really doing and why they had it on display... In the afternoon, I attended a couple of talks in the PostgreSQL developer room and a talk about a big OpenVPN deployment, before ending the day with the excellent as always social event barbeque. On Sunday, I went to quite a few talks, but I thought that two of them were particularly interesting:
Michael "Monty" Widenius of MySQL gave a talk titled "Why going open source will improve your product" about starting businesses on an open source project, or how business can/should open-source their product. Besides a detailed discussion about the various forms of Open Source licenses and the Open-Core model, he proposed the idea of "Business Source" (see slide 20 of his presentation), where a startup would distribute the source code under a non-commercial (but otherwise open-source) license with the explicit guarantee that the license would be changed to a true FLOSS license at some defined point in the future, giving the company a head start to develop and nurture their project. I asked whether this has been already implemented in practise and how the community could be sure that e.g. lawyers after a hostile takeover would not just remove that part of the copyright notice, as long as a true distribution under a FLOSS license has not happened yet. Monty wasn't aware of any real-word cases, and he did not seem to be concerned about this and said the original intent would be clear in a possible court case. This was the first time I heard about this approach, I wonder how other people think about it, whether it would work in practise and be a useful thing to have?
Second, I attended a talk by Gregor Geiermann, a Ph.D. student in linguistics on "Perceptions of rudeness in Free Software communities". He conducted an online survey about the perceived rudeness of several forum thread posts on Ubuntu Forums. Survey participants were first asked a couple of generic questions about their gender, nationality etc. and were then presented with a series of posts. For each post, they were asked to rate how rude they thought it was on a scale of 1 to 5 and they also had the possibility to highlight the parts of the post they considered rude as well as add comments. He presented a neat web application for analyzing the results, which makes it possible to select different groups (he did male vs. female and Americans vs. Germans in the talk) and have their overall rudeness ratings as well as the highlighted texts visualized as different shades of blue. Comments can be easily accessed. There were quite a few interesting differences e.g. in how Germans perceived rudeness compared to Americans (RTFM comments were considered less rude by Germans for example, IIRC). In response to my question, he said he intended to release the web application as open source and this might be an interesting tool for FLOSS projects to analyze how their public communication channels are perceived by various groups. Unfortunately, I cannot find any other resources about this on the web as of today, so I should try to contact him about it at some point.

30 September 2011

Martin F. Krafft: Archiving web pages revisited

An e-mail by Andreas Schamanek had me revisit the topic of archiving web pages. Andreas pointed me to the MHT format, which bundles HTML pages and their dependencies into a plain text file using MIME. Internet Explorer apparently already handles this format, and UnMHT provides software for the other browsers. As Firefox 6 is not yet supported, I went to try the Mozilla Archive Format extension, which seems to do the same thing and works quite well so well (on first sight) that I wanted to share it with you. NP: Tortoise: Standards

29 September 2011

Martin F. Krafft: Liable for other peoples' debts

Today, German politicians decided, that Germany be liable for up to 211 billion Euros for the debts of other EU countries. Or, put differently: the politicians put the money of the current and future generations on the line for a country that lived way above its capabilities for too many years. When the EU was founded, it was explicitly stated that no country needs ever be liable for any other. I understand that politics is hard, and letting Greece (and others) fall down might carry heavy, unforeseen consequences. Also, I understand that the Greek people are mostly innocent in all this and that the fault lies with their politicians and other corrupt entities in the nation. However, what s happening these days is beyond the comprehensible. If I were in a non-Germany EU country, I d rejoice and continue making debts. It is likely that this won t be the last time that our politicians cave in to pressure by other nations who have a lesser understanding of budgeting and saving. Since I am German, I can only shake my head, look to Berlin and ask myself whether this is the final straw that broke the camel s back. How the heck do the people over there ever want to regain the trust of their people? Politics has become the game of pleasing each other, who cares about the people? And the German politicians are (once again, remember credit default swaps?) at the forefront of this stupidity. To me, there is only one solution to Greece s debts: make sure that what happened can never happen again, and then cut the debts, or slice them in half. Let the banks carry the weight, for it was them who gave out the loans too liberally. And if this forces a bank or two to default, let it happen, for fuck s sake. The consequences might be dire, but they ll subside. And that s surely better than trying to pretend that we can keep juggling this heavily inflated financial system. Instead, the executives, elected to carry the trust of the people, are setting precedents for other countries to follow Greece, for it is likely that they wil be bailed out. By us. That is not the way to teach anyone the basics of economy: you can only spend as much as you earn, without exceptions. Debts will only come around to hurt you. I could puke. NP: Godspeed You Black Emperor!: F# A#

26 September 2011

Jan Hauke Rahm: zsh, tab completion, remote hosts, and collaboration

Using zsh as default shell is perfect. Using grml-zsh stuff to configure it, even better. Using XTaran s config, way better. Using your own config, invaluable. Now, there is one thing that always bothered me: having all hosts I usually deal with in my ~/.ssh/config in order to have tab completion (and short names). I wanted to improve the situation and was kindly pointed in a different direction by Myon, namely to just use the ~/.ssh/known_hosts for tab completion. How about that? I started playing around with the config and, as it turns out, Axel already has something ready. Unfortunately, it didn t fulfill all my needs yet. That is because I have multiple known_hosts files. Martin already filed a bug to have ssh read ~/.ssh/known_hosts.d/* but that isn t resolved yet either, even though upstream is aware of it. I thus had to point tab completion and ssh to multiple files by hand. But that s not too bad for now. Let s have a look: Host *
HashKnownHosts no Host *.your-work.com
User that-is-what-they-call-me-at-work
UserKnownHostsFile ~/.ssh/known_hosts.work Host *.debian.org
UserKnownHostsFile ~/.ssh/known_hosts.debian
 What am I doing? Well, we need to deactivate the hashing of known hosts. Otherwise your known_hosts files aren t readable as needed. Then you define your known_hosts files for the domains you care about. Pretty straight forward. Now, how about tab completion in zsh? Well, easy part actually: [ -f ~/.ssh/config ] && : $ (A)ssh_config_hosts:=$ $ $ $ (@M)$ (f)"$(<~/.ssh/config)" :#Host * #Host :#*\** :#*\?*
[ -f ~/.ssh/known_hosts ] && : $ (A)ssh_known_hosts:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts)" %%\ * %%,*
[ -f ~/.ssh/known_hosts.work ] && : $ (A)ssh_known_hosts_work:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts.work)" %%\ * %%,*
[ -f ~/.ssh/known_hosts.debian ] && : $ (A)ssh_known_hosts_debian:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts.debian)" %%\ * %%,* zstyle ':completion:*:hosts' hosts $ssh_config_hosts $ssh_known_hosts $ssh_known_hosts_work $ssh_known_hosts_debian What's here? We read and parse your ~/.ssh/config for configured hosts, then we parse all your known_hosts files which are for me: ~/.ssh/known_hosts, ~/.ssh/known_hosts.work, and ~/.ssh/known_hosts.debian. And lastly, all is added to zsh completion for hosts. That actually works. :) Axel, that makes a diff for you looking like this: diff --git a/zsh.d/70-completion b/zsh.d/70-completion
index e92e068..5abf5cc 100644
--- a/zsh.d/70-completion
+++ b/zsh.d/70-completion
@@ -8,6 +8,8 @@ [ -f ~/.ssh/config ] && : $ (A)ssh_config_hosts:=$ $ $ $ (@M)$ (f)"$(<~/.ssh/config)" :#Host * #Host :#*\** :#*\?*
[ -f ~/.ssh/known_hosts ] && : $ (A)ssh_known_hosts:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts)" %%\ * %%,*
+[ -f ~/.ssh/known_hosts.work ] && : $ (A)ssh_known_hosts_work:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts.work)" %%\ * %%,*
+[ -f ~/.ssh/known_hosts.debian ] && : $ (A)ssh_known_hosts_debian:=$ $ $ (f)"$(<$HOME/.ssh/known_hosts.debian)" %%\ * %%,* -zstyle ':completion:*:*:*' hosts $ssh_config_hosts $ssh_known_hosts
+zstyle ':completion:*:hosts' hosts $ssh_config_hosts $ssh_known_hosts $ssh_known_hosts_work $ssh_known_hosts_debian Care to merge? Oh, and before I forget... Of course you don't need to check each and every host by yourself. Debian provides ssh keys for all hosts on master. Just do a scp master.debian.org:/etc/ssh/ssh_known_hosts ~/.ssh/known_hosts.debian and I'm sure, your security aware employer has such a file for you as well. Doesn't he? ;)

24 September 2011

Martin-&#201;ric Racine: Proposed updates for CUPS-PDF packages at Ubuntu

For those who use CUPS-PDF on Ubuntu, I have pushed some updated packages into Lucid-proposed, Maverick-proposed and Natty-proposed, mainly to make the package's post-install actions bulletproof, so that automated installs and PPD updates take place in a flawless way. Comments from anyone who tested these package updates are welcome on Launchpad bug #805947. Once these updates have been approved, I'll go through the daunting task of reducing the debdiff down to its utmost essential components and attempt to submit a stable-update to Debian as well.

13 September 2011

Christian Perrier: 10 years being Debian Developer - part 4: NM process

So, this story begins in January 2001, when I applied in the Debian New Maintainer Process. I wanted to maintain Geneweb, which I was using to publish my genealogy research results. And I wanted to keep it up-to-date while upstream was doing a quite fast development. Moreover, I quickly noticed that Geneweb had big trouble in respecting the way data is organized on a Debian system, and respect the FHS. How to have weveral users able to publish their data on the same server without compromizing the overall system security, etc. Upstream development didn't really care about that. Daniel de Rauglaudre is an excellent genealogist and developer but he was not interested in making Geneweb clean "the Debian way". For instance, at that time, it wasn't easy to setup a server, where genealogy databases could be published without having to manually launch a daemon in user mode at each reboot. Arranging this was indeed my first contribution. I contributed a few patches to make it easier to turn Geneweb into something FHS-compliant...and I developed init scripts and an organization allowing one to have shared databases after system reboots. That was rapidly a great introduction to Debian maintainer scripts and even security-related challenges. And all this....because I needed it. Basically, in 2001, the geneweb package adopted the organization it still has in Debian and Ubuntu, 10 years later: While doing so, I was going through the New Maintainer Process. The Debian developer who had signed my key was Sam Tardieu, of of those longstanding "Freenix" dudes I was sometimes hanging around. I don't remember who did write my advocacy. Michael Mattice was my first Application Manager. Interestingly, here are the three first questions he asked my as part of the Policy and Procedures check : Mike actually asked the questions on February 17th and I probably answered immediately. Unfortunately it seems that I lost my answers. Maybe they are in some archive somewhere. Would be interesting to see how I ended giving an answer about the TeX exception.:-) Immediately, Mike moved to the Tasks and Skills step and checked my geneweb and lifelines packages (I already prepared a package for that other genealogy software). From my records, he found a few lintian warnings, which I probably made my best to fix, then later on, on March 12th, he answered me that thigns were OK and he was marking the T&S step as passed...handing things over to the NM database, asking me what username I wanted on debian.org (obviously, you know what I answered to this!). Then nothing happened..:-) I was not that impatient, but finally, around June 27th, I politely mailed NM Frontdesk (I guess) to get news...and I got a very quick answer by Martin Michlmayr. Martin was indeed worried about the situation (he told me I was on his radar....fun as, many years later, as many DDs, Martin probably went on mine because of pending localization work..:-)) Finally, Martin decided he was not entirely happy with the (maybe too short) processing for P&P and proposed me (very politely, you know how Martin is!) to do it again. So, some more questions...:-) Here, I have the answers. Really fun ro read. Hang on, pure bubulle style, very verbose! What a bunch of questions..:-). Reading my answers ten years after, I see that some answers are somehow naive but, after all, they're not so bad. I really remember spending hours in writing them (my English was somehow more shaky at that time) The last question was: "Please tell me about about yourself and what you intend to do for Debian". Here, Martin opened a big can of worms..:-) So, you'll learn about this in the next episode..:-)

Martin Pitt: PostgreSQL 9.1 final packages available for Debian/Ubuntu

Hot on the heels of the PostgreSQL 9.1.0 release I am happy to announce that the final version is now packaged for Debian unstable, the current Ubuntu development version Oneiric , and also in my Ubuntu backports PPA for Ubuntu 10.04 LTS, 10.10, and 11.04. Enjoy trying out all the cool new features like builtin synchronous replication or per-column collation settings for correctly handling international strings, or an even finer-grained access control for large environments. Please see the detailled explanation of the new features. As already announced a few days ago, 9.0 is gone from Ubuntu 11.10, as it is still only a development version and not an LTS. 9.1 will be the version which the next 12.04 LTS will support, so this slightly reduces the number of major upgrades Ubuntu users will need to do. However, 9.0 will still be available in Debian unstable and backports, and the Ubuntu backports PPA for a couple of months to give DB administrators some time to migrate.

9 September 2011

Rapha&#235;l Hertzog: People behind Debian: Enrico Zini, member of the New Maintainer Frontdesk

Enrico ZiniEven though Enrico is not smiling on this picture, he s one of the friendliest Debian person that I know. I always enjoy his presentations because he can t refrain from inserting jokes or other funny tricks. :-) That said he s serious too, there s lots of good stuff that he has developed over the years (starting with Debtags) and he has put a lot of effort in reforming the New Maintainer process. Read on to learn more about his various projects. Raphael: Who are you? Enrico: Hi, I m Enrico Zini, a DD from Italy. I m 35 and I work as a freelance Free Software developer. One of my historical roles in Debian is taking care of Debtags, but that is not all I do: my paid work led me to write and maintain some weather forecast related software in Debian, and recently I gained a Front Desk hat, and then a DAM hat. Raphael: How did you start contributing to Debian? Enrico: It was 2001, I was at uni, I was using Debian. At some point I wanted to learn packaging so I read through the whole Policy from top to bottom. Then I thought: why package only for myself? . There were many DDs at my uni, and it only seemed natural to me to join Debian as well. Evidently this was also natural for Zack [Note from editor: Stefano Zacchiroli], who had become DD 6 months earlier and didn t hesitate to advocate me. I found the Policy and the Developer s Reference to be very interesting things to read, and I welcomed my AM s questions as an excuse to learn more. I completely understand those people who have fun trying to answer all the questions in the NM templates while they wait for an AM. With my super DAM powers I can see that my AM report was submitted on October 16, 2001 by my AM Martin Michlmayr, and that James Troup created my account 9 days later, on October 25. Raphael: You have a special interest in the New Maintainer (NM) Process since you are a Debian Account Manager (DAM) and a member of the NM Frontdesk. Thanks to your work the process is much less academic/bureaucratic than it used to be. Can you remember us the main changes? Enrico: One of the first things I noticed when I become a Front Desk member is that there was a tendency to advocate people too early, thinking by the time they ll get an AM, they ll know enough . Unfortunately, this didn t always work, and once the real NM process started it would turn into a very long and demotivating experience both for the applicant and for the AM. So we tried raising the bar on advocacies, and that seems to have helped a lot. If people join NM when they are ready, it means that NM is quick and painless both for them and their AMs, who are therefore able to process more applicants. We also did a rather radical cleanup of the NM templates , which are a repository of questions that Application Managers can ask to their applicants. We realized that AMs were just sending the whole templates to their applicants, so we moved all non-essential questions to separate files, to drastically reduce the amounts of questions that are asked by default. Other improvements in the NM process came from other parts of Debian: nowadays there are lots of ways to learn and gradually gain experience and reputation inside the project before joining NM, which means that we get many candidates who we can process quickly. For example, packages can now be uploaded via sponsors, and the Mentors project helps new contributors to find sponsors and get their first packages reviewed. Then one can now become Debian Maintainer and take full responsibility of their own packages, gaining experience and reputation. The idea of working in teams also helped: big teams like the Perl, Python, KDE, Ocaml, Haskell teams (and many more) are excellent entry points for people who have something to package. But Debian is not just for packagers, and one could join teams like the Website team, the Press and Publicity team, the Events and Merchandise team or their local translation team. Becoming DDs the non-uploading way is not just for non-technical people: one could enjoy programming but not packaging. An interesting way to get involved in that way is to help writing or maintaining some of the many Debian services. Note that I m not suggesting this as a way to learn how to program, but as a way to get involved in Debian by writing code. Finally, we started to appreciate the importance of having people activities in Debian explicitly visible, which means that the more obviously good work one has done in Debian, the less questions we need to ask. Jan Dittberner s DDPortfolio is an excellent resource for AMs and Front Desk, and I m maintaining a service called minechangelogs that for people who have done lots of work in Debian is able to fully replace the Tasks&Skills parts of the NM process. Raphael: What are your plans for Debian Wheezy? Enrico: For Wheezy I hope to be able to streamline and simplify Debtags a bit more. At Debconf11 I had a conversation with FTP-masters on how to make some tags more official, and I now have to work a bit more on that. I d also like to considerably downsize the codebase behind the debtags package, now that its job is quite clear and I don t need to experiment with fancy features the way I did in the past. I have to say that I enjoy programming more than I enjoy packaging, so most of my plans in Debian are not tied to releases. For example, I d like to finish and deploy the new NM website codebase soon: it would mean to have a codebase that s much easier to maintain, and in which it s much easier to implement new features. I d also like to design a way to allow maintainers to review the tag submission to their own packages instead of having to wait for me or David Paleino to do a regular review of all the submissions. Finally, I d like to promote the usage of apt-xapian-index in more cutting-edge packaging applications. And to design a way to maintain up to date popcon information in one s local index. And improve and promote those services that I maintain, and I tend to often have ideas for new ones. Raphael: If you could spend all your time on Debian, what would you work on? Enrico: If I could spend all my time on Debian, I would do a lot of software development: I love doing software development, but most of my development energy is spent on my paid work. I guess I would start my all your time in Debian by taking better care of the things that I m already doing, and by promoting them better so that I wouldn t end up being the only person maintaining them. After that, however, I reckon that I do have a tendency of noticing new, interesting problems in need(?) of a solution, and I guess I would end up wildly experimenting new ideas in Debian much like a victorian mad scientist. Which reminds me that I most definitely need minions! Where can I find minions? Raphael: You re the author of the Debian Community Guidelines. I have always felt that this document should be more broadly advertised. For example by integrating it in the Developer s Reference. What do you think? Enrico: The DCG was really a collection of tips to improve one s online communication, based on ideas and feedback that I collected from pestering many experienced and well-respected people for some time. Like every repository of common sense, I think it should be widely promoted but never ever turned into law. It wouldn t be a bad idea to mention it in the Developer s Reference, or to package it as a separate file in the developers-reference package. The reasons I haven t actively been pushing for that to happen are two: there isn t much in the DCG that is specific to Debian, and I don t have the resources to do a proper job maintaining it. It d be great if somebody could take over its maintenance and make it become some proper, widespread, easy-to-quote online reference document, like one of those HOWTOs that all serious people have read at some point in their lives. Raphael: What s the biggest problem of Debian? Enrico: It s sometimes hard to get feedback or help if you work on something unusual. That is partly to be expected, and partly probably due to me not having yet learnt how to get people involved in what I do. Raphael: What motivates you to continue to contribute year after year? Enrico: Debian keeps evolving, so there is always something to learn. And Debian is real, so everything I do is constantly measured against reality. What more intellectual stimulation could one possibly want? Raphael: Is there someone in Debian that you admire for their contributions? Enrico: I don t think I could reasonably list everyone I admire in Debian: pretty much in every corner of the project there is someone, sometimes not very well known, who is putting a lot of Quality in what they do. Someone who decided that X should work well in Debian or that Debian should work well for Y or that Z is something Debian people can rely on and makes sure that it is so. Those are the people who make sure Debian is and will be not just a hobby, but a base upon which I can rely for my personal and working life.
Thank you to Enrico for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

Martin Pitt: Dropping PostgreSQL 9.0 packages for Debian/Ubuntu/backports

PostgreSQL 9.1 has had its first release candidate out for some two weeks without major problem reports, so it s time to promote this more heavily. If you use PostgreSQL, now is the time to try it out and report problems. We always strive to minimize the number of major versions which we have to support. They not only mean more maintenance for developers, but also more upgrade cycles for the users. 9.0 has not been in any stable Debian or Ubuntu release, and 9.1 final will be released soon. So we recently updated the current Ubuntu development release for 11.10 ( oneiric ) to 9.1. In Debian, the migration from 8.4/9.0 to 9.1 is making good progress, and there is not much which is left until postgresql-9.0 can be removed. Consequently, I also removed 9.0 from my PostgreSQL backports PPA, as there is nothing any more to backport it from. However, that mostly means that people will now set up installations with 9.1 instead of 9.0, and won t magically make your already installed 9.0 packages go away. They will just be marked as obsolete in the postgresql-common debconf note. If you want to build future 9.0 packages yourself, you can do this based on the current branch: bzr branch lp:~pitti/postgresql/debian-9.0, get a the new upstream tarball, name it accordingly, add a new changelog with a new upstream version number, and run bzr bd to build the package (you need to install the bzr-builddeb package for this). Update 2011-09-09: As I got a ton of pleas to continue the 9.0 backports for a couple of months, and to keep it in Debian unstable for a while longer, I put them back now. I also updated the removal request in Debian to point out that I m mainly interested in getting 9.0 out of testing. I don t mind much maintaining it for a couple of more months in unstable. My dear, I had no idea that my backports PPA was that popular!

8 September 2011

Sylvain Beucler: Modern OpenGL lighting

The Modern OpenGL tutorials series continue - now with a first section on lighting, covering the different kind of lights (sun, hemi, spot, etc.) using a simple diffuse model and clear code.
http://en.wikibooks.org/wiki/OpenGL_Programming#The_lighting_arc
This one is derived from the excellent set of Blender GLSL tutorials by Martin Kraus. As usual, the documentation is CC-BY-SA/GFDL, and the code is in the public domain. We're also expecting another tutorial on scientific visualisation. [EDIT: lighNing->lighing - sorry for the typo :)]

4 September 2011

Sylvain Beucler: Modern OpenGL lightning

The Modern OpenGL tutorials series continue - now with a first section on lightning, covering the different kind of lights (sun, hemi, spot, etc.) using a simple diffuse model and clear code.
http://en.wikibooks.org/wiki/OpenGL_Programming#The_lightning_arc
This one is derived from the excellent set of Blender GLSL tutorials by Martin Kraus. As usual, the documentation is CC-BY-SA/GFDL, and the code is in the public domain. We're also expecting another tutorial on scientific visualisation.

21 August 2011

Chris Lamb: Timezone bingo in debian/changelog files

Tim pointed out that it's worth travelling simply for the new timezone in your debian/changelog entries. We can use AptFs to work out who has collected the most so far: 
import os
import glob
import itertools
import collections
from dateutil.parser import parse
from debian.changelog import Changelog
data = collections.defaultdict(set)
for package in glob.glob('/apt/*'):
    if os.path.islink(package):
        continue # Consider source packages only
    try:
        changelog = Changelog(open('%s/debian/changelog' % package))
    except:
        continue # Ignore invalid changelogs
    for entry in changelog:
        try:
            data[entry.author].add(parse(entry.date).strftime('%z'))
        except ValueError:
            pass # Ignore invalid dates
fn = lambda x: len(x[1])
top = sorted(data.items(), key=fn, reverse=True)
for k, g in itertools.groupby(top, key=fn):
    print "\n%d timezone(s):" % k
    for author, timezones in sorted(g):
        print " * %s (%s)" % (
            author.encode('utf8', 'ignore'),
            ', '.join(sorted(timezones, reverse=True)),
        )

14 timezone(s):
 * Bdale Garbee <bdale@gag.com> (-0800, -0700, -0600, -0500, -0400, -0300,
   +1300, +1100, +1030, +0900, +0300, +0200, +0100, +0000)
12 timezone(s):
 * Joey Hess <joeyh@debian.org> (-1000, -0900, -0800, -0700, -0500, -0400,
   -0300, -0200, +0300, +0200, +0100, +0000)
11 timezone(s):
 * Paul Wise <pabs@debian.org> (-0400, -0300, +1300, +1100, +1000, +0930,
   +0900, +0800, +0200, +0100, +0000)
9 timezone(s):
 * Barak A. Pearlmutter <bap@debian.org> (-0700, -0600, -0500, -0400, +0500,
   +0300, +0200, +0100, +0000)
 * Martin Michlmayr <tbm@cyrius.com> (-1000, -0700, -0300, +1100, +1000,
   +0300, +0200, +0100, +0000)
 * Martin Pitt <mpitt@debian.org> (-0800, -0700, -0600, -0500, -0400, +0300,
   +0200, +0100, +0000)
 * Sam Hocevar (Debian packages) <sam+deb@zoy.org> (-0700, -0500, -0400, -0300,
   +0930, +0300, +0200, +0100, +0000)
Full output. However, something tells me we aren't going to see widespread gamification of Debian development.

18 August 2011

Rapha&#235;l Hertzog: People behind Debian: Peter Palfrader, Debian System Administrator

You might not know who Peter is because he s not very visible on Debian mailing lists. He s very active however and in particular on IRC. He was an admin of the OFTC IRC network at the time Debian switched from Freenode to OFTC. Nowadays he s a member of the Debian System Administration team who runs all the debian.org servers. If you went to a Debconf you probably met him since he s always looking for new signatures of his GPG key. He owns the best connected key in the PGP web of trust. He also wrote caff a popular GPG key signing tool. Raphael: Who are you? Peter: I m Peter Palfrader, also known as weasel. I m in my early 30s, born and raised in Innsbruck, Austria and am now living and working in Salzburg, Austria. In my copious free time, other than help running Debian s servers I also help maintaining the Tor project s infrastructure. Away from the computer I enjoy reading fiction (mostly English language Science Fiction and Fantasy), playing board games and going to the movies. Weather permitting, I also occasionally do some cycling. Raphael: How did you start contributing to Debian? Peter: I installed my first Debian the week slink came out. That was Debian 2.1 for the youngsters, in early 1999. The one thing I immediately liked about slink was that Debian s pppd supported RAS authentication which my university s dial-up system required. No way I d go back to SuSE 5.3 when I had working Internet with my Debian box. :) During that year I started getting involved in the German language Debian channel on IRCnet which got me in contact with some DDs. Christian Kurz (<shorty>) was working on Debian QA at the time and he asked my help in writing a couple of scripts. Some of that work, debcheck, still produces parts of the qa.d.o website, tho the relevance of that nowadays is probably negligible. While trying to learn more Perl earlier, I had written a program to produce syntax highlighted HTML for code snippets in various languages. I didn t really know what I was doing but it kinda worked, and probably still does since I still get mail from users every now and then. I figured that it would be really nice if people could just get my software together with Debian. According to code2html s Debian changelog the initial release of the package was done on a weekday at 2:30 in the morning early in 2000, and if my memory serves me correctly, shorty uploaded it shortly afterwards. I started packaging a couple of other piece of software and in the same year I sent my mail to the debian account managers to register my intent to become a DD. No new developers where being accepted at that time since the DAMs wanted to overhaul the entire process so I wasn t surprised to not get any immediate reply. Of course what the silence also meant was that the mail had been lost, but I only learned of that later when I took all my courage to ask DAM about the status of application a couple months later. Once that was sorted out I was assigned an AM, did the usual dance, and got my account late in November 2000. Raphael: Four years ago, the Debian System Administration team was a real bottleneck for the project and personal conflicts made it almost impossible to find solutions. You were eager to help and at some point you got dropped as a new member in that team. Can you share your story and how you managed the transition in the difficult climate at that time? Peter: Ah, that was quite the surprise for an awful lot of people, me included. Branden Robinson, who was our DPL for the 2005-2006 term, tried to get some new blood added to DSA who were at the time quite divided. He briefly talked to me on IRC some time in summer 2005, telling me I had come recommended for a role on the sysadmin team . In the course of these 15 minutes he outlined some of the issues he thought a new member of DSA would face and asked me if I thought I could help. My reply was cautiously positive, saying that I didn t want to step on anybody s toes but maybe I could be of some assistance. And that was the first and last of it, until some fine November day two years later I got an email from Phil Hands saying I ve just added you to the adm group, and added you to the debian-admin@d.o alias. and welcome on board . *blink* What!? My teammates at the time were James Troup (elmo), Phil Hands (fil), Martin Joey Schulze and Ryan Murray (neuro). The old team, while apparently not on good terms with one another, was however still around to do heavy lifting when required. I still remember when on my first or second day on the team two disks failed in the raid5 of ftp-master.debian.org aka ries. Neuro did the reinstall once new disks had arrived at Brown University. I m sure I d have been way out of my league had this job fallen to me. Fortunately my teammates were all willing and able to help me find whatever pieces of information existed that might help me learn how debian.org does its stuff. Unfortunately a lot of it only existed in various heads, or when lucky, in one of the huge mbox archives of the debian-admin alias or list. Anyway, soon I was able to get my hands dirty with upgrading from sarge to etch, which had been released about half a year earlier. Raphael: I know the DSA team has accomplished a lot over the last few years. Can you share some interesting figures? Peter: Indeed we have accomplished a lot. In my opinion the most important of these accomplishment is that we re actually once again a team nowadays. A team where people talk to one another and where nobody should be a SPoF. Since this year s debconf we are six people in the admin team: Tollef Fog Heen (Mithrandir) and Faidon Liambotis (paravoid) joined the existing members: Luca Filipozzi, Stephen Gran, Martin Zobel-Helas, and myself. Growing a core team, especially one where membership comes with uid0 on all machines, is not easy and that s why I m very glad we managed to actually do this step. I also think the infrastructure and our workflows have matured well over the last four years. We now have essential monitoring as a matter of course: Nagios not only checks whether all daemons that should be running are in fact running, but it also monitors hardware health of disks, fans, etc. where possible. We are alerted of outstanding security updates that need to be installed and of changes made to our systems that weren t then explicitly acked by one of us. We have set up a centralized configuration system, puppet, for some of our configuration that is the same, or at least similar, on all our machines. Most, if not all, pieces of software, scripts and helpers that we use on debian.org infrastructure is in publicly accessible git repositories. We have good communication with other teams in Debian that need our support, like the ftp folks or the buildd people. As for figures, I don t think there s anything spectacular. As of the time of our BoF at this year s DebConf, we take care of approximately 135 systems, about 100 of them being real iron, the other virtual machines (KVM). They are hosted at over 30 different locations, tho we are trying to cut down on that number, but that s a long and difficult process. We don t really collect a lot of other figures like web hits on www.debian.org or downloads from the ftp archive. The web team might do the former and the latter is pretty much impossible due to the distributed nature of our mirrors, as you well know. Raphael: The DSA team has a policy of eating its own dog food, i.e. you re trying to rely only on what s available in Debian. How does that work out and what are the remaining gaps? Peter: Mostly Debian, the OS, just meets our needs. Sure, the update frequency is a bit high, we probably wouldn t mind a longer release cycle. But on the other hand most software is recent enough. And when it s not, that s easy to fix with backports. If they aren t on backports.debian.org already, we ll just put them there (or ask somebody else to prepare a backport for us) and so everybody else benefits from that work too. Some things we need just don t, and probably won t, exist in Debian. These are mainly proprietary hardware health checks like HP s tools for their servers, or various vendors programs to query their raid controller. HP actually makes packages for their stuff which is very nice, but other things we just put into /usr/local, or if we really need it on a number of machines, package ourselves. The push to cripple our installers and kernels by removing firmware was quite annoying, since it made installing from the official media next to impossible in some cases. Support for working around these limitations has improved with squeeze so that s probably ok now. One of the other problems is that especially on embedded platforms most of the buildd work happens on some variation of development boards, usually due to increased memory and hard disk requirements than the intended market audience. This often implies that the kernel shipped with Debian won t be usable on our own debian.org machines. This makes keeping up with security and other kernel fixes way more error prone and time intensive. We keep annoying the right people in Debian to add kernel flavors that actually boot on our machines, and things are getting better, so maybe in the future this will no longer be a problem. Raphael: If you could spend all your time on Debian, what would you work on? Peter: One of the things that I think is a bit annoying for admins that maintain machines all over the globe is mirror selection. I shouldn t have to care where my packages come from, apt-get should just fetch them from a mirror, any mirror, that is close by, fast and recent. I don t need to know which one it was. We have deployed geodns for security.debian.org a while ago, and it seems to work quite well for the coarse granularity we desired for that setup, but geodns is an ugly hack (I think it is a layer violation), it might not scale to hundreds or thousands of mirrors, and it doesn t play well with DNSSEC. What I d really like to see is Debian support apt s mirror method that I think (and I apologize if I m wronging somebody) Michael Vogt implemented recently. The basic idea is that you simply add deb mirror://mirror.debian.org/ or something like that to your sources.list, and apt goes and asks that server for a list of mirrors it should use right now. The client code exists, but I don t know how well tested it is. What is missing is the server part. One that gives clients a mirror, or list of mirrors, that are close to them, current, and carry their architecture. It s probably not a huge amount of work, but at the same time it s also not entirely trivial. If I had more time on my hands this is something that I d try to do. Hopefully somebody will pick it up. Raphael: What motivates you to continue to contribute year after year? Peter: It s fun, mostly. Sure, there are things that need to be done regularly that are boring or become so after a while, but as a sysadmin you tend to do things once or twice and then seek to automate it. DSA s users, i.e. DDs, constantly want to play with new services or approaches to make Debian better and often they need our support or help in their endeavors. So that s a constant flow of interesting challenges. Another reason is that Debian is simply where some of my friends are. Working on Debian with them is interacting with friends. I not only use Debian at debian.org. I use it at work, I use it on my own machines, on the servers of the Tor project. When I was with OFTC Debian is what we put on our machines. Being a part of Debian is one way to ensure what Debian releases is actually usable to me, professionally and with other projects. Raphael: Is there someone in Debian that you admire for their contributions? Peter: That s a hard one. There are certainly people who I respect greatly for their technical or other contributions to Debian, but I don t want to single anybody out in particular. I think we all, everyone who ever contributed to Debian with code, support or a bug report, can be very proud of what we are producing one of the best operating systems out there.
Thank you to Peter for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.

15 August 2011

Steve McIntyre: In one word: AWESOME

Every year I worry that DebConf might not be as good as I hope, or not as good as previous years. Well, I've yet to be let down! The road trip down to Banja Luka was good fun, even if it took a little longer than planned. We ended up travelling down through Germany on the same Friday as much of the country finished work/school for their summer vacation, so there was a lot of traffic. Meh, we got there in the end. I made my usual mistake of planning some things to hack on during the DebConf week; by now I should know better... :-) I made a start on one small project, but then got so distracted by so many talks and side meetings with people that it's still waiting. As always, my own personal TODO list picked up huge amounts of extra stuff from those discussions. Catching up with, and socialising with, Debian friends from all over the world was also fun as always! Lots of highlights of the week for me, both for technical and social reasons: Banja Luka was a lovely place to visit, and a great host city for DebConf. Looking forwards to Managua now!

13 August 2011

Martin-&#201;ric Racine: Configuring GNOME themes for both gconf and dconf?

For a number of years, I've been making myself metapackages that pull favorite packages and pre-configure the desktop environment to use my favorite global default settings. This currently means configuring the gconf keys for the GTK theme, icon theme and background image for GDM. Until now, I implemented this by calling gconftool-2 directly in postinst as user gdm but Lintian recently started warning me that the correct way to do this would be by calling gconf-schemas or update-gconf-defaults instead. I was thus wondering, would anyone have any sample code that accomplishes this? To further complicate things, GNOME 3 has transitioned from gconf to dconf, which suggests that a slightly different method needs to be used to perform the same. Would anyone have any sample code to implement this? Thanks in advance to anyone who can help me with this!

1 August 2011

Christian Perrier: DD, 10 years

On July 30th, 2001, I received a mail from James Troup stating that I was a Debian developer, this 10 days after my Application Manager, namely Martin Michlmayr, sent his AM report. Summarizing these 10 years will be....interesting. Now that I'm on holidays, I'll try too cook something up. Sorry, folks, I completely forgot celebrating this last Saturday.

26 July 2011

Martin F. Krafft: Wisdom tooth left in Bosnia

For those who care or wonder: the reason why I hold a white icepack to my cheek here at Debconf11 in Banja Luka, Bosnia & Hercegovina, talk fairly little and try not to smile is because I had one of my wisdom teeth removed this morning by one of the local dentists. Some might cringe at the idea of submitting yourself to such a treatment in Bosnia, but I have to say that Doctor Sa a Dabi did a splendid job, even though we weren t really able to communicate a lot. Still, 45 minutes after I entered the office, I saw my tooth on the table and was able to leave again. My tooth had been building up an infection for several weeks, and it was starting to become unbearable. Therefore I decided to simply bite the bullet, after having seen the x-rays and judging that it wouldn t be too hard to remove. It wasn t, the pain is now minor, the swelling mostly under control, the drugs are beer-compatible, and you all should just enjoy while I cannot talk for tomorrow I ll be back!

24 July 2011

Martin Michlmayr: Upgrade to mainline U-Boot from Debian archive

When Marvell originally released the first plug computer, they created their own version of u-boot with support for their new devices. Unfortunately, this version of u-boot is fairly out of date nowadays compared to mainline u-boot and has several problems. Support for plug computers (such as SheevaPlug and GuruPlug) have been integrated into the mainline u-boot (also known as DENX u-boot) in the meantime and Clint Adams has packaged it for Debian. I finally found the time to test Clint's u-boot binary on my devices and have updated the SheevaPlug installation guide accordingly. If you're have installed Debian to a SheevaPlug according to my instructions, I suggest you upgrade. If you boot from a MMC/SD card, you should be aware that the mmcinit command has been renamed to mmc init in order to be consistent with the naming of other commands. You'll therefore have to update your bootcmd_mmc variable in u-boot like this: 
setenv bootcmd_mmc 'mmc init; ext2load mmc 0:1 0x00800000 /uImage; ext2load mmc 0:1 0x01100000 /uInitrd'
saveenv

21 July 2011

Rapha&#235;l Hertzog: People behind Debian: Martin Michlmayr, former Debian Project Leader

Martin Michlmayr is a Debian developer since 2000 and I share quite a few things with him, starting with his age and involvement in the quality assurance team. He managed to be elected Debian Project Leader in 2003 and 2004. He s no longer as active as he used to be but his input is always very valuable and he continues to do very interesting things in particular concerning the support of NAS devices. Read on for the details. Raphael: Who are you? Martin: I m Martin Michlmayr. I m 32, originally from Austria, and currently living in the UK. I ve contributed to various free software projects over the years but Debian is without doubt the one I m most passionate about. I joined Debian in 2000 when I was a student. I worked on Debian more or less full time for a few years while I was pretending to study. Later I started a PhD to do research about quality and management aspects of volunteer free software projects. I investigated the release process in several free software projects, in particular looking at time-based releases. After finishing my PhD in 2007, I joined Hewlett-Packard. I m part of HP s Open Source Program Office and work on various free software and open source activities, both internally and within the community. Raphael: How did you start contributing to Debian? Martin: I first used Debian in the days of 0.93R6, some time around the end of 1995. The 0.93R6 release was still based on a.out but I needed an ELF-based system for some application, so I moved to Slackware. I then quickly moved to Red Hat Linux where I stayed for several years. I rediscovered Debian in 2000 and quickly decided to join the project. I cannot recall how I rediscovered Debian but when I did, it was clear to me that Debian was the ideal project for me: I could identify with its philosophy, I liked the volunteer nature of the project, and I found the size and diversity of Debian interesting since a large project offers a lot of different challenges and opportunities. I remember how many new things there were to learn and back then the documentation and other resources for new contributors were nowhere as good as they are today. My application manager, Julian Gilbey, was a great help he was incredibly friendly and passionate about Debian. I also remember meeting up with Peter Palfrader (weasel) for key signing when we were both in the New Maintainer queue. I was incredibly lucky with my New Maintainer process and soon became an official Debian Developer. Because there was a shortage of application managers, my first major contribution in Debian was to become an application manager myself and help other people join the project. Debian is a large project with a long history and a rich culture, so new contributors should expect that it will take some time to become familiar with everything. Fortunately, there are many resources, such as documentation and the debian-mentors list, for new contributors. Another great way to become familiar with the way things are done in Debian is to subscribe to various Debian mailing lists and ideally to read some mailing list archives. It s also a great idea to attend the Debian Conference or other conferences since meeting people in real life is a great way to integrate. I remember attending Debian Conference 1 in Bordeaux where I gave my first public talk. Finally, new contributors should find an area where they can make a unique contribution. Most people in Debian maintain packages but there are so many other ways to contribute. For example, most of my contributions were not technical but were about coordination and other organizational activities. Raphael: What s your biggest achievement within Debian? Martin: I m particularly proud of a number of achievements: Raphael: Speaking about NAS devices: what exactly are you doing on this topic and how can people help? Martin: There are plenty of instructions on the Internet to install Linux distributions on NAS or various embedded devices by connecting a serial console and then typing in hundreds of commands. What I found is that such instructions significantly limit the user base because they are way too complicated for most users. There are just too many steps that can go wrong. So instead, in Debian, we provide a solution that just works: usually, you download a firmware image for your NAS device from Debian and when you upgrade you get the Debian installer. You connect to the installer via SSH and perform a normal installation. The installer knows about the device and will prepare everything for you automatically for example, it knows if the device has requirements for the partition layout and it will install the kernel where the device expects to find it; unfortunately, NAS devices are not like PCs, so the requirements are different for almost every device and therefore you need special code to support a new device. Finally, there are detailed installation guides and we provide help on our mailing lists. There are a number of technical areas for improvement. The installation could be made even easier, and it would be nice to support new platforms and devices. A bigger problem is that while we ve implemented a great solution for NAS devices, we haven t really extended this work to support other classes of devices. For example, tablets and mobile phones are getting incredibly popular and we don t have a compelling solution for such devices, mostly because of the lack of an appropriate GUI. Raphael: What are your plans for Debian Wheezy? Martin: I ve recently been asked by Stefano Zacchiroli, our current Debian Project Leader, to coordinate the care-taking of Debian finances. Debian, as a volunteer project, relies on donations and in-kind gifts (e.g. hardware) to maintain its infrastructure and to support various development efforts, such as funding sprints and other developer gatherings. Debian s money and other assets are held by affiliate organizations around the world. My responsibility will be to keep track of money and other assets (e.g. hardware and trademarks), work with the DPL to establish procedures related to the use of Debian s assets, and make sure that the procedures are followed. Finally, we want to publish public statements so our donors know how we use their donations to further improve Debian. I just started working on this and this will be my main activity in Debian in the coming months. Raphael: Speaking of money, I plan to run a fundraising to get the Debian book I wrote with Roland Mas translated (cf. http://debian-handbook.info). Is this something Debian should support? Martin: First of all, I should make it clear that I don t decide how Debian spends its money. This is up to the DPL to decide together with the project at a whole. I ll just make sure that procedures are followed and expenses tracked and reported properly. Having said that, in my opinion, it s unlikely that Debian as a project will fund this effort. It would be inconsistent with the position of the project not to fund work directly (only some related expenses, such as travel costs to allow Debian teams to organize face-to-face meetings). Whether Debian should support the fundraising effort by helping to promote it is another question and that s probably not as clear cut. It looks like a worthwhile effort, but on the other hand it would be unfair for authors of other Debian books for Debian to put its weight behind one and there are many other efforts that are worth promoting if you promote one, where do you stop? So while it sounds worthwhile, it s probably better for Debian to stay out of it. But somehow related to this, I sometimes worry about the fact that there are so few paid opportunities around Debian. If you contribute to the Linux kernel for a while, you have an excellent chance to get hired by someone and to work on the kernel full time. The kernel may be an extreme example but there are a lot of projects that have more paid opportunities than Debian, e.g. Mono, GNOME, OpenOffice/LibreOffice and KDE. Obviously, there are some Debian Developers who can spend some time on Debian as part of their job. I know that some Canonical employees contribute to Debian, that support companies like credativ improve Debian as part of their work, and that system administrators fix bugs or package new software as they deploy Debian. But I think this is a minority of contributors and even they don t work full time on Debian. Instead what I see is that a lot of people leave university, get a job and then no longer have time for Debian or people start a family and no longer have time. I can take myself as an example since I don t have nearly as much time as I did in the past when I was a student. I guess there are different ways to deal with this problem one would be to create more paid opportunities around Debian outside the project, another one might be to make it easier for new volunteers to join the project. I don t have the answers to these questions but it s something I wonder about, and I also wonder whether pure volunteer projects can still keep up with projects with a lot of full time contributors. Raphael: What motivates you to continue to contribute year after year? Martin: Debian is a great project with a great mission, goals and people. I contribute to make Debian a better solution and to promote the free software philosophy. Finally, the community around Debian provides a lot of motivation. It s amazing how much I ve learned about other cultures because of my involvement in Debian and how many friends I ve made over the years all around the world. Raphael: Do you have wishes for Debian Wheezy? Martin: Not really. I m pretty happy with the way things are going at the moment. We have made a lot of organizational changes in the last few years from which the project has greatly benefited. I m particularly pleased about the plans to adopt a time-based freeze. Raphael: Is there someone in Debian that you admire for their contributions? Martin: There are many people I admire greatly. I d like to mention Joey Hess because he s a great example to follow. He doesn t get involved in politics, is easy to work with and does great technical work. In fact, he has made not one but several contributions that have completely changed Debian (debconf, debhelper, and debian-installer). Furthermore, Debian has a lot of contributors who have done great work over the years but who are not very vocal about it. People like Colin Watson or Peter Palfrader. Debian has many unique contributors and the list of people I admire is much longer than the few people I just mentioned.
Thank you to Martin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. He raised some interesting questions. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.

15 July 2011

Dirk Eddelbuettel: MCMC and faster Gibbs Sampling using Rcpp

Sanjog Misra, who uses Rcpp for Monte Carlo Markov Chain (MCMC) analyses in quantitative marketing, kindly set me a short example of Rcpp use. The example is based on a blog post by Darren Wilkinson which itself discusses and compares the suitability of R, Python, Java or C for MCMC analysis, using the Gibbs sampler as a concrete example. Darren's post is worth checking out: he stresses the rather pragmatic aspects of how fast and/or easy it is to write the code, rather than just the mere runtime. As such, he is not too concerned with a speed advantage of Python over R which he sees at a factor of around 2.4, leaving him to continue to prototype in R. Similarly, with C 'only' being faster than Java by a factor of two, he prefers Java for the numerically more demanding parts. We do of course advocate the use of Rcpp to combine the best aspects of R and C++, respectively. This Gibbs sampler example provides a nice backdrop. So working with Darren's example, consider the same Gibbs sampler for a bivariate distribution (and apologies for the lack of latex typesetting on my blog) 
f(x,y) = k x^2 exp( -x y^2 - y^2 + 2y - 4x)
where the conditional distributions are 
f(x y) = (x^2)*exp(-x*(4+y*y))               ## a Gamma density kernel
f(y x) = exp(-0.5*2*(x+1)*(y^2 - 2*y/(x+1))  ## a Gaussian kernel
Sanjog then spotted and corrected a small error in the variance expression in Darren's derivation; this is now acknowledged on Darren's website. Full details are in the R script now committed in SVN. The R code for the Gibbs sample can therefore be written as follows below. Note that this uses thinning to minimize serial correlation in the conditional densities--which renders the computation more demanding as 'N times thin' draws have to be generated: 
## Here is the actual Gibbs Sampler
## This is Darren Wilkinsons R code (with the corrected variance)
## But we are returning only his columns 2 and 3 as the 1:N sequence
## is never used below
Rgibbs <- function(N,thin)  
    mat <- matrix(0,ncol=2,nrow=N)
    x <- 0
    y <- 0
    for (i in 1:N)  
        for (j in 1:thin)  
            x <- rgamma(1,3,y*y+4)
            y <- rnorm(1,1/(x+1),1/sqrt(2*(x+1)))
         
        mat[i,] <- c(x,y)
     
    mat
A second variant can be computed using the R bytecode compiler which appeared with the recent release of R 2.13.0 (and which we analysed in this blog post from April). This is as easy as 
## We can also try the R compiler on this R function
RCgibbs <- cmpfun(Rgibbs)
Thanks to Rcpp and the inline package, we can also write a C++ variant that can be built and launched from R with ease. The C++ code is assigned to an R text variable gibbscode. (And we used to typeset such code on the blog as a character string, ie in a faint red color---but have now switched to highlight it as if it were freestanding C++ code. It really is passed as a single string to R which then uses the cxxfunction() to compile, link and load a C++ function built around the code. See previous posts on the inline package for more.) 
## Now for the Rcpp version -- Notice how easy it is to code up!
gibbscode <- '

  using namespace Rcpp;   // inline does that for us already
  // n and thin are SEXPs which the Rcpp::as function maps to C++ vars
  int N   = as<int>(n);
  int thn = as<int>(thin);
  int i,j;
  NumericMatrix mat(N, 2);
  RNGScope scope;         // Initialize Random number generator
  // The rest of the code follows the R version
  double x=0, y=0;
  for (i=0; i<N; i++)  
    for (j=0; j<thn; j++)  
      x = ::Rf_rgamma(3.0,1.0/(y*y+4));
      y = ::Rf_rnorm(1.0/(x+1),1.0/sqrt(2*x+2));
     
    mat(i,0) = x;
    mat(i,1) = y;
   
  return mat;             // Return to R
'
# Compile and Load
RcppGibbs <- cxxfunction(signature(n="int", thin = "int"),
                         gibbscode, plugin="Rcpp")
It is noteworthy how the code logic is essentially identical between the basic R version, and the C++ version. Two nested loops control draws of x from a Gamma distribution, conditional on y, as well as draws of y from a Normal, conditional on x. Add a small amount of parameter passing to obtain the parameters N and thin, allocation of a results matrix and setup of the random number generator state to remain consistent with R, as well as a return of the matrix---and that is all. As Darren's code uses the GNU GSL in its C variant, I also became interested in seeing how a C/C++ hybrid variant using our RcppGSL package would fare. The code is below. 
gslgibbsincl <- '
  #include <gsl/gsl_rng.h>
  #include <gsl/gsl_randist.h>
  using namespace Rcpp;  // just to be explicit
'
gslgibbscode <- '
  int N = as<int>(ns);
  int thin = as<int>(thns);
  int i, j;
  gsl_rng *r = gsl_rng_alloc(gsl_rng_mt19937);
  double x=0, y=0;
  NumericMatrix mat(N, 2);
  for (i=0; i<N; i++)  
    for (j=0; j<thin; j++)  
      x = gsl_ran_gamma(r,3.0,1.0/(y*y+4));
      y = 1.0/(x+1)+gsl_ran_gaussian(r,1.0/sqrt(2*x+2));
     
    mat(i,0) = x;
    mat(i,1) = y;
   
  gsl_rng_free(r);
  return mat;           // Return to R
'
## Compile and Load
GSLGibbs <- cxxfunction(signature(ns="int", thns = "int"),
                        body=gslgibbscode, includes=gslgibbsincl,
                        plugin="RcppGSL")
This code is similar to the version using just Rcpp, but we need to allocate space for the GSL random generator object (and later release that memory allocation), and we do of course call the GSL functions. This also necessitates declaring the function using the two header files listed as arguments for the include argument of cxxfunction(). So how is the performance? The script (now committed in Rcpp's SVN repo as examples/RcppGibbs/RcppGibbs.R, and also part of the next release) creates the output below when running in non-interactive mode (whereas interactive mode creates a few charts too): 
edd@max:~/svn/rcpp/pkg/Rcpp/inst/examples/RcppGibbs$ r RcppGibbs.R
Replication # 1 complete 
Replication # 2 complete 
Replication # 3 complete 
Replication # 4 complete 
                    N=1000 N=5000 N=10000 N=20000
Elasped Time (R)     0.124  2.650  10.511  41.773
Elasped Time (RC)    0.081  1.916   7.594  30.280
Elapsed Time (Rcpp)  0.003  0.076   0.306   1.221
Elapsed Time (Rgsl)  0.002  0.049   0.196   0.784
SpeedUp Rcomp.       1.530  1.380   1.380   1.380
SpeedUp Rcpp        41.330 34.870  34.350  34.210
SpeedUp GSL         62.000 54.080  53.630  53.280
               test replications elapsed  relative user.self sys.self
4  GSLGibbs(N, thn)           10   7.845  1.000000      7.84        0
3 RcppGibbs(N, thn)           10  12.218  1.557425     12.22        0
2   RCgibbs(N, thn)           10 312.296 39.808286    311.98        0
1    Rgibbs(N, thn)           10 420.953 53.658764    420.59        0
The first block is a hand-computed comparison using four sets of parameters; the second block uses the excellent rbenchmark package to compare ten runs at twenty-thousand draws. We see a nominal increase in performance due to the bytecode compiler, saving roughly 38 percent which seems appropriate given that the R code mostly controls the loops; actual work in undertaken in the already compiled RNG functions. The Rcpp variant is about 34 times faster than the pure R code. Sanjog reported higher increases on his OS X machines (and Darren's post echos a similar order of magnitude). However, on my i7 running Linux I always obtained an improvement in the mid- to high thirties. That is certainly already a rather pleasant result. What surprised and stunned me at first was that the GSL solution scores an improvement of around 53 times (close to the factor of 60 reported by Darren). A closer look at the code (shown above) makes it clear that there are very few compute-intensive operations outside of the RNG draws. I validated this further with a second study timing just one million draws each from a Gaussian and Gamma using R via Rcpp, and using the GSL. It turns out that the R code is about 2.5 times slower for random draws from the Gamma distribution than the GSL. Inspection of the source code---in files src/nmath/rgamma.c for R and randist/gamma.c for the GSL shows why. R uses a much more complex (and presumably more precise) algorithm. I may follow up with Martin Maechler to see if we can illustrative the numerical benefits of this more expensive approach---this blog entry is getting long enough already. So to sum up: Gibbs sampling is a somewhat resource-heavy Monte Carlo Markov Chain method for investigating multivariate distributions. R excels at quick and simple explorations, albeit at somewhat slower execution speed. The Rcpp package can help by providing easy means to accelerate simulations by a significant factor. The example discussed here is now in SVN repository for Rcpp and will be part of the next release.

Updated 2011-07-16: Darren has just updated his comparison; fixed two typos here too.

Next.

Previous.