Raphaël Hertzog: People behind Debian: Peter Palfrader, Debian System Administrator
You might not know who Peter is because he s not very visible on Debian mailing lists. He s very active however and in particular on IRC. He was an admin of the OFTC IRC network at the time Debian switched from Freenode to OFTC. Nowadays he s a member of the Debian System Administration team who runs all the debian.org servers.
If you went to a Debconf you probably met him since he s always looking for new signatures of his GPG key. He owns the best connected key in the PGP web of trust. He also wrote caff a popular GPG key signing tool.
Raphael: Who are you?
Peter: I m Peter Palfrader, also known as weasel. I m in my early 30s, born
and raised in Innsbruck, Austria and am now living and working in
Salzburg, Austria. In my copious free time, other than help running
Debian s servers I also help maintaining the Tor project s infrastructure.
Away from the computer I enjoy reading fiction (mostly English language
Science Fiction and Fantasy), playing board games and going to the movies.
Weather permitting, I also occasionally do some cycling.
Raphael: How did you start contributing to Debian?
Peter: I installed my first Debian the week slink came out. That was Debian
2.1 for the youngsters, in early 1999. The one thing I immediately
liked about slink was that Debian s pppd supported RAS
authentication which my university s dial-up system required. No way
I d go back to SuSE 5.3 when I had working Internet with my Debian box. 
During that year I started getting involved in the German language
Debian channel on IRCnet which got me in contact with some DDs.
Christian Kurz (<shorty>) was working on Debian QA at the time
and he asked my help in writing a couple of scripts. Some of that
work, debcheck, still produces parts of the qa.d.o website, tho the relevance of
that nowadays is probably negligible.
While trying to learn more Perl earlier, I had written a program to
produce syntax highlighted HTML for code snippets in various languages.
I didn t really know what I was doing but it kinda worked, and probably
still does since I still get mail from users every now and then.
I figured that it would be really nice if people could just get my
software together with Debian. According to code2html s
Debian changelog the initial release of the package was done on a
weekday at 2:30 in the morning early in 2000, and if my memory serves me
correctly, shorty uploaded it shortly afterwards.
I started packaging a couple of other piece of software and in the same
year I sent my mail to the debian account managers to register my intent
to become a DD. No new developers where being accepted at that time
since the DAMs wanted to overhaul the entire process so I wasn t
surprised to not get any immediate reply. Of course what the silence
also meant was that the mail had been lost, but I only learned of that
later when I took all my courage to ask DAM about the status of
application a couple months later. Once that was sorted out I was
assigned an AM, did the usual dance, and got my account late in November
2000.
Raphael: Four years ago, the Debian System Administration team was a real bottleneck
for the project and personal conflicts made it almost impossible to
find solutions. You were eager to help and at some point you got
dropped as a new member in that team. Can you share your story and how
you managed the transition in the difficult climate at that time?
Peter: Ah, that was quite the surprise for an awful lot of people, me included.
Branden Robinson, who was our DPL for the 2005-2006 term, tried to get
some new blood added to DSA who were at the time quite divided. He
briefly talked to me on IRC some time in summer 2005, telling me I had
come recommended for a role on the sysadmin team . In the course of
these 15 minutes he outlined some of the issues he thought a new member
of DSA would face and asked me if I thought I could help. My reply was
cautiously positive, saying that I didn t want to step on anybody s toes
but maybe I could be of some assistance.
And that was the first and last of it, until some fine November day two
years later I got an email from Phil Hands saying I ve just added
you to the adm group, and added you to the debian-admin@d.o
alias. and welcome on board . *blink* What!?
My teammates at the time were James Troup (elmo), Phil Hands (fil),
Martin Joey Schulze and Ryan Murray (neuro).
The old team, while apparently not on good terms with one another, was
however still around to do heavy lifting when required. I still remember
when on my first or second day on the team two disks failed in the raid5
of ftp-master.debian.org aka ries. Neuro did the reinstall once new
disks had arrived at Brown University. I m sure I d have been way out
of my league had this job fallen to me.
Fortunately my teammates were all willing and able to help me find
whatever pieces of information existed that might help me learn how
debian.org does its stuff. Unfortunately a lot of it only existed in
various heads, or when lucky, in one of the huge mbox archives of the
debian-admin alias or list. Anyway, soon I was able to get my hands
dirty with upgrading from sarge to etch, which had been released about
half a year earlier.
Raphael: I know the DSA team has accomplished a lot over the last few years.
Can you share some interesting figures?
Peter: Indeed we have accomplished a lot. In my opinion the most important
of these accomplishment is that we re actually once again a team
nowadays. A team where people talk to one another and where nobody
should be a SPoF.
Since this year s debconf we are six people in the admin team: Tollef
Fog Heen (Mithrandir) and Faidon Liambotis (paravoid) joined the
existing members: Luca Filipozzi, Stephen Gran, Martin Zobel-Helas, and
myself. Growing a core team, especially one where membership comes with
uid0 on all machines, is not easy and that s why I m very glad we
managed to actually do this step.
I also think the infrastructure and our workflows have matured well over
the last four years.
We now have essential monitoring as a matter of course: Nagios not only
checks whether all daemons that should be running are in fact running,
but it also monitors hardware health of disks, fans, etc. where
possible. We are alerted of outstanding security updates that need to
be installed and of changes made to our systems that weren t then
explicitly acked by one of us.
We have set up a centralized configuration system, puppet, for some of
our configuration that is the same, or at least similar, on all our
machines.
Most, if not all, pieces of software, scripts and helpers that we use on
debian.org infrastructure is in publicly accessible git repositories.
We have good communication with other teams in Debian that need our
support, like the ftp folks or the buildd people.
As for figures, I don t think there s anything spectacular. As of the
time of our BoF at this year s DebConf, we take care of approximately
135 systems, about 100 of them being real iron, the other virtual
machines (KVM). They are hosted at over 30 different locations, tho we
are trying to cut down on that number, but that s a long and difficult
process.
We don t really collect a lot of other figures like web hits on
www.debian.org or downloads from the ftp archive. The web team might do
the former and the latter is pretty much impossible due to the
distributed nature of our mirrors, as you well know.
Raphael: The DSA team has a policy of eating its own dog food, i.e. you re trying to rely only on what s available in Debian. How does that work out and what are the remaining gaps?
Peter: Mostly Debian, the OS, just meets our needs. Sure, the update frequency
is a bit high, we probably wouldn t mind a longer release cycle. But on
the other hand most software is recent enough. And when it s not,
that s easy to fix with backports. If they aren t on backports.debian.org
already, we ll just put them there (or ask somebody else to prepare a
backport for us) and so everybody else benefits from that work too.
Some things we need just don t, and probably won t, exist in Debian.
These are mainly proprietary hardware health checks like HP s tools for
their servers, or various vendors programs to query their raid
controller. HP actually makes packages for their stuff which is very
nice, but other things we just put into /usr/local, or if we really need
it on a number of machines, package ourselves.
The push to cripple our installers and kernels by removing firmware was
quite annoying, since it made installing from the official media next to
impossible in some cases. Support for working around these limitations
has improved with squeeze so that s probably ok now.
One of the other problems is that especially on embedded platforms most
of the buildd work happens on some variation of development boards,
usually due to increased memory and hard disk requirements than the
intended market audience. This often implies that the kernel shipped
with Debian won t be usable on our own debian.org machines. This makes
keeping up with security and other kernel fixes way more error prone and
time intensive. We keep annoying the right people in Debian to add
kernel flavors that actually boot on our machines, and things are
getting better, so maybe in the future this will no longer be a problem.
Raphael: If you could spend all your time on Debian, what would you work on?
Peter: One of the things that I think is a bit annoying for admins that
maintain machines all over the globe is mirror selection. I shouldn t
have to care where my packages come from, apt-get should just fetch them
from a mirror, any mirror, that is close by, fast and recent. I don t
need to know which one it was.
We have deployed geodns for security.debian.org a while ago, and it
seems to work quite well for the coarse granularity we desired for that
setup, but geodns is an ugly hack (I think it is a layer violation), it
might not scale to hundreds or thousands of mirrors, and it doesn t play
well with DNSSEC.
What I d really like to see is Debian support apt s mirror method that
I think (and I apologize if I m wronging somebody) Michael Vogt implemented
recently. The basic idea is that you simply add deb
mirror://mirror.debian.org/ or something like that to your
sources.list, and apt goes and asks that server for a list of mirrors it
should use right now.
The client code exists, but I don t know how well tested it is. What is
missing is the server part. One that gives clients a mirror, or list of
mirrors, that are close to them, current, and carry their architecture.
It s probably not a huge amount of work, but at the same time it s also
not entirely trivial. If I had more time on my hands this is something
that I d try to do. Hopefully somebody will pick it up.
Raphael: What motivates you to continue to contribute year after year?
Peter: It s fun, mostly. Sure, there are things that need to be done regularly
that are boring or become so after a while, but as a sysadmin you tend
to do things once or twice and then seek to automate it.
DSA s users, i.e. DDs, constantly want to play with new services or
approaches to make Debian better and often they need our support or help
in their endeavors. So that s a constant flow of interesting
challenges.
Another reason is that Debian is simply where some of my friends are.
Working on Debian with them is interacting with friends.
I not only use Debian at debian.org. I use it at work, I use it on my
own machines, on the servers of the Tor project. When I was with OFTC
Debian is what we put on our machines. Being a part of Debian is one
way to ensure what Debian releases is actually usable to me,
professionally and with other projects.
Raphael: Is there someone in Debian that you admire for their contributions?
Peter: That s a hard one. There are certainly people who I respect greatly for
their technical or other contributions to Debian, but I don t want to
single anybody out in particular. I think we all, everyone who ever
contributed to Debian with code, support or a bug report, can be very
proud of what we are producing one of the best operating systems out
there.
Thank you to Peter for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.
No comment Liked this article? Click here. My blog is Flattr-enabled.