Calypso CalDAV/CardDAV/WebDAV for Android and Evolution Ever since I bought my first Palm Pilot in 1997, I ve relied upon a pocket-able device to carry a copy of my calendar and contacts, and for that same database to be present on my laptop. I went through a long list of Palm-compatible devices, including both the Palm Treo and Palm Centro telephones. I even wrote a number of my own Palm applications. Years ago, it was pretty obvious that I d have to find a new phone, but I was stuck looking for something that could provide the same hot-sync functionality. SyncML on the Series 40 I bought a Series 40 Nokia phone in Shanghai that promised SyncML support. Given that I had seen numerous SyncML implementations for Linux, it seemed like I should be able to get something to work. SyncML on the Series 40 is a disaster the phone couldn t actually store all of my contacts, and it couldn t hold half of the data fields I used. So, that phone landed in the box and back to the Centro I went. SyncML on the N900 Nokia kindly sent me an N900 at some point, so I gave SyncML another try. Given that the N900 runs evolution-data-server, and that I ve had evolution-data-server running on my laptop, it seemed like I should be in business. Well, almost. It took several days of hacking to fix bugs in the evolution SyncML back end, then another several days fussing with opensync. I ended up abandoning direct synchronization as unworkable opensync would sit in an infinite loop, or worse, trash my database completely. I finally found syncml-ds-tool , which is a debugging tool that comes with opensync. This tool simply synchronizes a set of disk files, one per contact or calendar entry, with the phone. That worked for well over a year. And then, a few months ago, Bluetooth on the laptop stopped connecting with the phone s SyncML server. I d get ECONNREFUSED every time I tried to use it. So much for the N900. DUN still worked, mostly, although it too would get ECONNREFUSED at times, but retrying seemed to make it work. However, While the N900 SyncML solution worked, I discovered another thing I wanted contacts and calendar entries stored in individual files and revision controlled with git. This makes it reasonable to delete stale calendar entries and know that they re never really gone, just left behind in an older version of the calendar. And, if you mess up, you can recover by poking at the database with git directly. I switched back to the venerable Palm Centro; it turns out that calendar and contacts are more important to me than being able to surf the web on my phone. Alas, my Centro went swimming in August and has passed on to the great electronics recycling house in the sky. I pulled the SIM out and switched back to the N900. I got my contacts imported on the N900 by copying files over the net work; not a long-term strategy, but at least I had phone numbers again. There was no hope for my calendar. I started looking for a solution in earnest. How about Android? At this point in my story, I m sure you re asking why I didn t just use one of the numerous Android phones that came through my hands. The answer is simple my calendar and contacts are probably some of my most personal data, and I m not willing to store them outside of my direct control, for reasons similar to those which are driving the development of the FreedomBox. When Android first came out, it could only talk to Google services, which didn t meet my hard requirement for personal data storage. One of my co-workers had his Google account suspended for violating the terms-of-service; he asked what he had done, but they wouldn t say. He asked if he could get his data back, and they said no . They invited him to create a new account, but it would not ever get any of the old data. A few days later, he got a nice apologetic email letting him know that they d made a mistake and that he hadn t, in fact, violated any of the terms-of-service, and that his old account was restored with all of his data intact. Wasn t that nice of Google? WebOS? I got a WebOS phone over the summer and discovered that while it had multiple contact/calendar back ends, none of them used standard protocols and so you only had the choice between multiple corporate data centers, which isn t actually a choice at all. Furthermore, the WebOS phone refused route PAN packets over the phone network, even though I have a data plan which allows this. It s not that it couldn t support it, it s that it refused. A couple of weeks after the WebOS phone arrived, HP canceled all of their WebOS hardware products, which made me less interested in trying to solve this problem. Android recovers About the same time the WebOS phone arrived, I discovered that Google had published enough information about the calendar/contacts internals for Marten Gajda to write CalDAV-Sync and CardDAV-Sync. And then, Andrew McMillan wrote aCal, which is a complete replacement for the built-in calendar and contacts applications and supports CalDAV and CardDAV. With two different standards-compliant solutions available, it seemed like it might be time to try Android again. I d love for CardDAV-Sync and CalDAV-Sync to become free software like aCal is. Andrew makes money from aCal by offering it for sale via the Android Market, while still publishing the sources for those who want to build their own copy. CalDAV/CardDAV on Linux I think the most widely known CalDAV server for Linux is probably DAViCal, a huge pile of PHP and SQL sitting on top of Apache. I m sure it s suitable for running on a server and being accessed over the internet, but I m not interested in that, nor am I interested in having my laptop run Apache and PostgreSQL. I found a tiny little CalDAV server, Radicale, which seemed like a lot better fit. It s written in Python and uses the usual Python HTTP server infrastructure, which provides SSL and authentication support along with some fairly convenient APIs for parsing and generating HTML. Before long, I discovered that Radicale was actually too simple for my needs. It stores the whole calendar in a single file, re-parsing it whenever a request is made, so a calendar with just hundreds of entries caused the server to slow down enough that evolution would time-out when talking to it. Also, Radicale doesn t actually parse the calendar entries completely, it has some ad-hoc code that finds various pieces of data, but without dealing with the whole syntax. I started hacking at Radicale to see how far I could get. I changed the storage code to store one event per file, then added hooks to use git for change management. Then, I found a full vcalendar/vcard parsing library in python, vobject, which I used to replace the ad-hoc parsing code. Finally, I added support for VCARD entries as well, allowing the system to store both calendar and contact information. Introducing Calypso, a CardDAV/CalDAV server With this much divergence from the original project, I ve figured I d best rename things to avoid confusion, so I decided to call it calypso , after a brief trip through the dictionary looking for names starting with ca . Calypso works with evolution, iceowl and the Android CalDAV/CardDAV plugins. It does not yet work with aCal; for some reason aCal cannot find any calendars on the server. Calypso also supports importing calendar changes from the command line, allowing you to integrate support into a text-based email application like notmuch or mutt. Calypso is available via git from git://keithp.com/git/calypso and is distributed under the GPL (v3 or later). I still consider it a work derived from Radicale, and so the code retains all of the Radicale copyrights along with my own. Using Calypso Initial setup Calypso runs as a regular user, all data are stored in ~/.config/calypso. To initialize calypso:

$ mkdir ~/.config/calypso ~/.config/calypso/calendars
$ cat > ~/.config/calypso/config << EOF
[server]
ssl=true
certificate=/home/keithp/.config/calypso/ssl/server.crt
key=/home/keithp/.config/calypso/ssl/server.key
[acl]
;type=htpasswd
type=fake
filename=/home/keithp/.config/calypso/passwd

Running calypso Then run calypso:

$ python ./calypso.py

No, I haven t figured out how to install it Creating new calendars To add a new database:

$ mkdir -p ~/.config/calypso/calendars/private/my_calendar
$ cd ~/.config/calypso/calendars/private/my_calendar
$ git init
$ git commit --allow-empty -m'initialize new calendar'

The new calendar should now be visible as https://localhost:5233/private/my_calendar You can add files to the directory at any time; calypso will check the directory mtime at each operation and update its internal state from that on disk automatically when the directory changes. Importing files Given a set of files with VCALENDAR or VCARD entries, you can import them with:

$ calypso --import private/my_calendar <filenames...>

This will update any changed entries and add any new ones. ToDo list for calypso

1. Document the config file contents.
2. Make it installable
3. Figure out what aCal wants
4. Support calendar creation

More Android info

• Synker. This provides a desktop widget which starts the sync process running on a list of accounts. This makes it easy to manually synchronize with the laptop when you are connected
• Contact Editor. This lets you fully edit contacts synchronized over CardDAV. The built-in contact editor doesn t let you change anything other than the name for some reason.

I m running cyanogenmod on my Nexus S as that provides PAN support. With PAN, I can create a network link between laptop and phone which doesn t depend on any local WiFi infrastructure and which gives both phone and laptop static IP addresses, allowing me to configure the sync URLs statically on the phone. I d use mDNS, but Android doesn t bother to support that.

Here's the mail archiving script I was talking about: http://github.com/jmtd/detachment. Sorry about the name-pun. There's still a LOT of work to do, but it's useful enough to deploy, with modifications, now. If you set up a way of pushing stuff in a local directory to a remote webserver (ideally behind an SSL certificate and with access control configured) and you have a resolvable hostname for that site, then you can get at your files from most mail/web client combinations, including smartphones. Here's some stats from running it on a sample mailbox: Before

$ du -sh ncl/archive/2010
116M    ncl/archive/2010

After

$ du -sh ncl/archive/2010 bucket
41M     ncl/archive/2010
51M     bucket

Or, approximately 20% smaller. (That's the on-disk usage for the mail and the attachments, now not base64-encoded, and saved to disk). The above folder was a local Maildir copy of a folder on an Exchange mail system, synchronised via offlineimap. Here are the sizes reported by Exchange for the same folder:

• Before: 83991KB (82M)
• After: 25435KB (25M)

Or, approximately 70% smaller. (That's the mails without the attachments counted, which are not in the Exchange server any more). Lots of caveats:

• If you are using offlineimap, you will need to do more than just run this script, or offlineimap may not notice that the messages have changed. Stripping out the U=\d+,FMD5=[0-9a-f] 32 component of filenames seemed to do the trick.
• If you are using Exchange, a copy of the original messages will be retained in your Deleted Items folder. Beware if you are close to your mail quota.
• If you are using Exchange via IMAP, then syncing the results of such an operation might be very slow. The above operation took well over an hour to sync, and I'm only a few hops away from the Exchange server via 100M links.

Google Maps is now mapping the indoors. I saw an ad for this while passing through MSP yesterday (given how much time I spend on the Internet, it's strange and a little embarrassing to learn about new things on the Internet from airport billboards), since one of their initial targets is the infamous Mall of America.

Detailed floor plans automatically appear when you re viewing the map and zoomed in on a building where indoor map data is available. The familiar blue dot icon indicates your location within several meters, and when you move up or down a level in a building with multiple floors, the interface will automatically update to display which floor you re on. All this is achieved by using an approach similar to that of My Location for outdoor spaces, but fine tuned for indoors.

Thoughts about this:

• Okay, having a map of Ikea or an airport on my phone might be kind of nice.
• Shouldn't they have launched this years ago when people still went to malls?
• Shouldn't they have launched this at least earlier in the year, in time for people to know about it and have it before the one time each year when people who don't go to malls still maybe go to the mall?
• How does it know where you are, when cell and GPS so often don't work in these places? "fine tuned for indoors" is a very mysterious phrase. Based on nearby access points?
• Is your location also being transmitted if you use this service? In other words, will Google now know which stores you shop at, how many times you use the restroom, and whether you were actually shopping for gifts for other people?
• Relatedly, will this be free software? Or will it rely on proprietaryness in order to do things users would prefer it didn't do? (Answer: It's proprietary software, part of the existing proprietary Google Maps mobile application.)
• Will they sell data to the stores and malls?
• Will they start indicating various promotions on the maps?
• Will these promotions be targeted based on your past meandering behavior?
• Are they mapping less-commercial spaces, like museums?
• Are they making the map data available for others to use? Especially for truly public spaces?
• How do we get the functional part of this into OpenStreetMap?
• Is Google secretly working on computer-piloted mobility scooters?
• Where am I?

Title from John Ashbery

This year is in its last throes and a few things, that don't really warrant their own blog post, have piled up here. To get them out of the door before 2012 starts, I'd like to go over them in the next few lines. First: there seem to be a group of rather cheap people who want to have the Humble Indie Bundle #4, but don't like to give at least 0.01 USD. Really guys, you seem to have enough money for a fast-enough computer and a (broadband) internet connection (otherwise downloading the circa 2.8 GB of games won't be a lot of fun), but you don't have the 0.01 USD to spare? If you don't like closed-source games (a sentiment I can truly understand, in fact, that was something I really pondered before buying) then just don't play them. But searching for a torrent for the HIB#4 while it is still being sold to just avoid paying the minimal fee, I can't understand. Now, some might wonder how I know this. The easy answer is, I blogged about buying the HIB#4 and mentioned, that I downloaded the games through a torrent (you can choose between direct downloads and torrents on your "login page") and a lot of people started hitting the blog entry with a referrer from a search engine, showing they'd searched for "humble indie bundle 4 torrent". Please make a decision: don't play the games or if you wish to, then pay something. The people behind those games make it easy and, in comparison to many other game studios, a fair offer. That should be worth something. For those of you who loved Josephine, there is a new awesome piece of art, Creative Commons licensed, by An-U (German homepage). The title (and the drawing) are derived from the equally awesome Katzenjammer song "On the Devil's Back" (in case you're still looking for a present, you might want to consider one of their CDs or a ticket to their next concert near you). Then, I wanted to thank all those people, who donated something over the course of 2011! Thanks a lot, especially for showing me, that you liked my services/work enough to even consider a donation! It means a lot to me, to know this. At year's end everybody is doing statistics and while I won't bore you with long in-depth analysis, I thought I throw out, what the top posts/things where (so far). Maybe you find yourself represented in there or maybe you find something new. In no particular order:

• My unofficial wine-unstable packages generated a lot of traffic and back links.
• The mini-tips series had many hits.
• My two posts (1, 2) about the Linux graphics stack garnered a lot of attention.
• The note about removing Sun's Java implementation was the first post to be mentioned in a Debian Project News issue.
• Right after my wine-unstable packages, my post about OSM and OpenLayers generated the most e-mails.
• The YouTube playlist fetcher is also pretty popular, though the Greasemonkey script collection has been picking up speed recently.

I hope your 2011 was a good year and I wish you all the best for 2012! I'm pretty sure Debian will continue to grow, but there is also a lot of work ahead, maybe you can join the effort and make Debian that much more awesome!

In 1977 Tal Brody said after winning the Euroleague Basketball cup with Maccabi Tel Aviv as sentence which became quite famous in Israel:

We are on the map! And we are staying on the map not only in sports, but in everything.

I found out through Invar Hovav that there s a regression in the Ubuntu installation doesn t believe this is true anymore, as according to lp#905754, you can t select Israel or its major cities, and instead get suggestions for Gaza and Hebron. This is a regression from past installers when you could at least choose Tel Aviv, Jerusalem and Gaza. It used to be hard to pin point each location, but was still possible. This might be part of a bigger problem of selecting close/adjacent cities, but the default behavior in the past was different. From the technical point of view, the Gaza and Hebron TZ files are weird (tzdata version 2011n-1). Both don t have info for 2012 daylight saving time (usually start in late March or early April). Hebron switches twice a year to DST, hopefully nothing critical uses this time zone.

$ zdump -v Asia/Hebron grep 2011
Asia/Hebron Fri Apr 1 10:00:59 2011 UTC = Fri Apr 1 12:00:59 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Fri Apr 1 10:01:00 2011 UTC = Fri Apr 1 13:01:00 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Sun Jul 31 20:59:59 2011 UTC = Sun Jul 31 23:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Sun Jul 31 21:00:00 2011 UTC = Sun Jul 31 23:00:00 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Mon Aug 29 21:59:59 2011 UTC = Mon Aug 29 23:59:59 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Mon Aug 29 22:00:00 2011 UTC = Tue Aug 30 01:00:00 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Thu Sep 29 23:59:59 2011 UTC = Fri Sep 30 02:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Fri Sep 30 00:00:00 2011 UTC = Fri Sep 30 02:00:00 2011 EET isdst=0 gmtoff=7200

And Gaza only have a 3 months daylight saving time, while Israel has 6 and European eastern time has 7 months (all are UTC+2 region).

$ zdump -v Asia/Gaza grep 2011
Asia/Gaza Sat Apr 2 10:00:59 2011 UTC = Sat Apr 2 12:00:59 2011 EET isdst=0 gmtoff=7200
Asia/Gaza Sat Apr 2 10:01:00 2011 UTC = Sat Apr 2 13:01:00 2011 EEST isdst=1 gmtoff=10800
Asia/Gaza Sun Jul 31 20:59:59 2011 UTC = Sun Jul 31 23:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Gaza Sun Jul 31 21:00:00 2011 UTC = Sun Jul 31 23:00:00 2011 EET isdst=0 gmtoff=7200

p.s.
This isn t a political post, don t make it into one.
Filed under: Debian GNU/Linux, Ubuntu

My goal with the 3.0 (quilt) source format has always been to standardize the patch management in Debian source packages. This message seems to have been well understood. dbs and dpatch have been deprecated by their respective maintainers. I made numerous efforts to make this source format useful in as many use cases as possible (but some improvements are still possible) and I have added hints to encourage maintainers to switch. Thanks to this, the adoption rate of this new source format has been very good and it s now the most widely used source package format in Debian only two years after its introduction in Debian unstable. With 9829 source package using 3.0 (quilt) , it surpassed the number of source package still using 1.0 (7368). (Those numbers have been taken from http://upsilon.cc/~zack/stuff/dpkg-v3/ on december 13th 2011.) The number of source packages using 3.0 (quilt) doubled this year.

(Click on the picture to see it full size) Of the 7368 packages using the old format, 6816 packages trigger the missing-debian-source-format lintian tag. This means that only 552 source packages have explicitly opted to keep using the old format and that the bulk of the remaining packages are rarely updated packages that have not been switched yet.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

I just discovered that bug report over at Ubuntu.
Short summary:
They have a script in upstart which is not meant to be run manually and if you do it will erase your whole file system. Additionally it seems that the fact that you shall not run that script is not communicated anywhere.

That alone isn't the most spectacular about it. Bugs happen. Whats spectacular about it is how a Canonical employee and member of the TechBoard (for people who don't know it: The people who decide about the technical direction Ubuntu takes) handles that bug. One quote of him to reflect it all:

Sorry, the only response here is "Don't Do That Then"

So what we have here is a classical case of bad programming. The problem in question is that the script expects a certain environment variable to be set. Fair enough. However it does not check if its set at all and instead of failing or using a sensible default it simply sticks to undefined behaviour. What we have here is a classical programming mistake every beginner tends to do. People who start programming often forget (or don't know) that every external value we rely on must be considered untrustworthy. Therefore a good practice is to check those values.In this case someone decided that this is useless because they suffer from the wrong assumption that nobody ever calls it manually and the other wrong assumption that caller of the scriptwill always set the environment variable correctly. This is a double-fail.
Now the developer in question does not accept that (someone else indicated why the behaviour of the script is dangerous), he simply says that the bug is invalid. Thats really a pity.

The route cache is a Linux kernel component enabling route lookups to be faster by caching the results in some table and checking it before issuing a regular lookup in the route tables. When using Linux as a router, the inefficiency of the route cache can hinder the performances of your box. The documentation on this component is scarce and it is difficult to find up-to-date bits on how the route cache works and how to tune it. The book Understanding Linux Network Internals from O Reilly is an exception and contains valuable information on how the route cache works. Even if the book is targeted at 2.6.12, the part on the route cache is still quite accurate. Unfortunately, it fails to provide appropriate tips on how to monitor and tune the route cache. I hope to provide here a concise view of the route cache, as it is implemented in Linux 3.1. It is protocol-dependent¹ and I will cover only the IPv4 version here.

Overview of the route cache subsystem To handle an incoming or outgoing IP datagram, the kernel needs to issue a lookup in the route tables. While it seems to be quite trivial, several questions need to be answered:

• Does the source address and the destination address appear to be valid?
• Is the source address a martian address²?
• Is the destination address mine or should I forward the packet?
• Which route table should I use?
• Does the destination address match this route? And this one?
• Can I currently contact the gateway that I should use?

Those checks can be a bit time consuming, even for small route tables. To avoid them for each packets, Linux maintains a route cache which is queried before doing a regular lookup and updated after each one. You can dump it with ip -s route show cache:

$ ip -s route show cache
198.51.100.7 from 203.0.113.2 via 192.0.2.1 dev eth1
  cache used 7 age 2sec ipid 0x1fce rtt 131ms rttvar 45ms cwnd 10
198.51.100.17 from 203.0.113.15 via 192.0.2.1 dev eth1
  cache used 3 age 0sec ipid 0xc3bd
local 192.0.2.18 from 203.0.113.15 dev lo src 192.0.2.18
  cache <local> used 154 age 1sec iif eth0

Here are two examples:

• If Linux receives a packet from 198.51.100.17 to 203.0.113.15, it will find this flow in the route cache. Therefore, it already knows that it should forward the packet to 192.0.2.1. No checks needed.
• If it receives a packet from 198.51.100.16 to 203.0.113.15, there is no appropriate entry in the route cache and therefore, the system will have to look at the route tables. It is likely to use the exact same entry than if the source was 198.51.100.17 but maybe there is some policy routing requesting the use of a special route table or 198.51.100.16 is a local address and the packet will therefore be classified as martian.

The schema below shows how this cache is implemented³. It uses a separate hash table (BSD systems keep the cache in the routing table). Each bucket is a chained list of route cache entries. Once an entry has been added to the route cache, there are several ways to remove it. Most entries are removed by the garbage collector which will scan the route cache and remove invalid and older entries. It will be triggered when the route cache is full or at regular interval, once a certain threshold has been met.

Available knobs There are several values you can tune. Most of them are available in sysctl.

• rhash_entries is the size of the hash table⁴. If you don t specify it on the kernel command line, it is computed dynamically based on the memory available on your system. You can view its value by looking at something like IP route cache hash table entries: 262144 (order: 9, 2097152 bytes) in the kernel logs.
• net.ipv4.route.max_size is the maximum number of entries in the route cache. Except under exceptional circumstances, this value is never exceeded.
• net.ipv4.route.gc_elasticity is the target average length of a chain in the route cache. The garbage collector will work harder if this value is exceeded. This means that if you multiply this value by the value of rhash_entries, you will get the target average number of entries in the route cache.
• net.ipv4.route.gc_min_interval_ms is the minimum delay between two runs of the garbage collector, except when the cache is full. The default value should be fine.
• net.ipv4.route.gc_thresh is a threshold triggering the garbage collector every net.ipv4.route.gc_min_interval_ms milliseconds.
• net.ipv4.route.gc_timeout is the base value to determine if an entry is old enough to be removed or not. Whatever its value, the garbage collector will attempt to remove the same number of entries. However, this value could potentially influence its efficiency. See below for more details on this.

You may find documentation about those obsolete sysctl values:

• net.ipv4.route.secret_interval has been removed in Linux 2.6.35; it was used to trigger an asynchronous flush at fixed interval to avoid to fill the cache.
• net.ipv4.route.gc_interval has been removed in Linux 2.6.38. It is still present until Linux 3.2 but has no effect. It was used to trigger an asynchronous cleanup of the route cache (but in a different way of what the garbage collector does). The garbage collector is now considered efficient enough for the job.

Statistics & monitoring Linux maintains some statistics about the use of the route cache. You can find them in /proc/net/stat/rt_cache. The command lnstat can print them nicely for you:

$ lnstat -s1 -i1 -c-1 -f rt_cache
rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache 
 entries   in_hit in_slow_ in_slow_ in_no_ro   in_brd in_marti in_marti  out_hit out_slow out_slow gc_total gc_ignor gc_goal_ gc_dst_o in_hlist out_hlis 
                       tot       mc      ute            an_dst   an_src              _tot      _mc                ed     miss  verflow  _search t_search 
 3096848    42309      686        0        0        0        0        0        3        0        0      674      672        0        0    27644        8 
 3096984    41405      636        0        0        0        0        0        3        0        0      623      621        0        0    28189        8 
 3097160    42483      700        0        0        0        0        0        5        0        0      694      692        0        0    29506       12 

Except for the first column, lnstat outputs values in units per second. Let s review some of those values:

• rt_cache_entries is the number of entries in the route cache. You should compare it with net.ipv4.route.max_size and ensure that the cache is never full to avoid triggering the garbage collector too often.
• rt_cache_in_hit and rt_cache_out_hit are the number of regular lookups avoided because the result was found in the cache for incoming and outgoing packets, respectively. When the system is a router, most lookups only happen on the incoming side. You should compare this value with rt_cache_in_slow_tot and rt_cache_in_out_slow_tot which are the number of lookups in the route tables. On this system, the efficiency of the route cache is more than 98% which is quite good.
• rt_cache_gc_total is how often the garbage collector was requested to be triggered while rt_cache_gc_ignored corresponds to how often it was finally not run because it has already been triggered shortly beforehand (less than net.ipv4.route.gc_min_interval_ms milliseconds). The difference between those two values should be very small to ensure that the garbage collector does not run more a handful of times per second.
• rt_cache_gc_goal_miss is how often the garbage collector was not able to fulfill its goal. This value should rarely be different of 0.
• rt_cache_gc_dst_overflow is how often the route cache is bigger than the allowed maximum size. This should never happen except when you try to shrink the cache.
• rt_cache_in_hlist_search and rt_cache_out_hlist_search is how often a lookup in the cache had to look at the next entry in the chain for the computed bucket: each time Linux has to follow the next pointer in a cache entry, it increments one of those counters. It is a clue on the average length of chains in the route cache hash table. Compare those values with the number of cache lookups (hit and miss).

As an illustration, here is a plot of the various statistics exposed above for a router whose route cache receives about 1000 routes par second: rhash_entries is 1,048,576, as is net.ipv4.route.gc_thresh. Therefore, the garbage collector was requested to be run when the number of entries goes above this level. Because the cache is not full, it is only really run twice per seconds (the value of net.ipv4.gc_interval_ms is 500). net.ipv4.gc_elasticity is set to 3. This explains why the garbage collector is aggressive when the number of entries reached 3,145,728. As you can see, the efficiency is near 100% all the time. The percentage of collisions is rt_cache_in_hlist_search ratio to the sum of rt_cache_in_hit and rt_cache_in_slow_tot.

Tuning Do you need to modify any of those values? You have two questions to ask yourself:

1. How much efficiency do you want to get from the route cache?
2. How much memory do you want to dedicate to the route cache?

As a rule of thumb, two millions entries eat about 500 MB of memory on a 64bit system. You should be able to compute the average memory usage and the maximum memory usage from the values of net.ipv4.route.max_size, rhash_entries and net.ipv4.route.gc_elasticity. For example, if the route cache hash table has 262,144 buckets, the maximum allowed number of entries in the cache is 4,194,304 and net.ipv4.route.gc_elasticity is set to 8, the memory usage will be 500 MB on average and 1 GB max. If this is too much, you will need to lower some values. Look at the previous section to compute the current efficiency of your route cache. It should be above 90%. If you are dissatisfied with that, you could increase the cache size. If you want to double the number of entries, double net.ipv4.route.max_size, net.ipv4.route.gc_thresh and rhash_entries but keep net.ipv4.route.gc_elasticity to 8. For the garbage collector to be efficient, the route cache should not be filled too fast. The garbage collector should be able to cope with this situation but this may impact performance because it needs to walk several times the route cache to find entries to expire. Watching gc_goal_miss may give you a hint about this: if this value starts to be different of 0, lower net.ipv4.gc_timeout.

In-depth look into the garbage collector Those different pieces of advice may have puzzled you. We need to understand how the garbage collector works to better cope with them. The garbage collector is triggered when a new entry needs to be added to the cache and the number of entries is superior to net.ipv4.route.gc_thresh. It is implemented in rt_garbage_collect() function. It will do nothing if it has been called less than net.ipv4.route.gc_interval_ms milliseconds and the cache is not full (more than net.ipv4.route.max_size entries).

Setting a goal The garbage collector will first assess the situation. It will look how the number of entries currently in the cache compares to the product of rhash_entries and net.ipv4.gc_elasticity:

1. Above this limit, it will try to remove at least rhash_entries entries.
2. Otherwise, it will try to remove no more than half the difference.

If you look at the previous plot, you can easily see the difference when the garbage collector is not aggressive (above 1,048,576 entries but below 3,145,728) and when it is (above 3,145,728 entries). If we assume the garbage collector is able to meet its goal, we can easily simulate its algorithm: The first plot shows what would happen if about 2000 routes are added per second with rhash_entries equal to 262,144 and net.ipv4.route.gc_elasticity set to 8. When net.ipv4.route.gc_threshold is met, the garbage collector has almost no effect. However, when the number of entries reaches 2,097,152, the garbage collector sets its goal to 262,144. From this point, the number of entries oscillates around 2 millions entries. On the second plot, rhash_entries is now equal to 1,048,576 but net.ipv4.route.gc_elasticity has been set to 2. Therefore, the aggressive part of the garbage collectors kicks at the same threshold than for the previous plot. However, its goal is now four times larger and the oscillations have a larger amplitude. The fact that the aggressive part of the garbage collector kicks less often is nullified by the fact that it needs to remove more entries each time. Because of the slight impact on cache efficiency, it seems better to keep net.ipv4.route.gc_elasticity around 8, or 4 if we want to keep shorter chains (but there seems to be no improvement to do so). The other plots show what happens when there is a sudden surge in the number of routes added or when there is a pause.

Meeting the goal Now that we understand how rhash_entries, net.ipv4.route.gc_elasticity and net.ipv4.route.gc_threshold interact, let s look at net.ipv4.route.gc_timeout. Once the garbage collector has set its goal and if it is positive, it needs to choose which entries to remove from the cache. It walks the hash table from the position of its last run and remove entries until its goal is met. If the entry is not current anymore (for example, the network interface associated to it has changed its IP configuration), it is removed. Otherwise, the system looks at the age of the entry and its position in the chain. If the entry is the first in the chain, it is kept only if its age is below net.ipv4.route.gc_timeout. If it is second, it is kept only if its age is below half of net.ipv4.route.gc_timeout. If it is third, the threshold is a quarter of net.ipv4.route.gc_timeout, and so on. The garbage collector will favor short chains. If after a full run of the hash table, the garbage collector was not able to meet its goal, it will start again but will behave more aggressively, as if net.ipv4.route.gc_timeout is set to half of its value. It will do as many passes as necessary until its goal is met or until there is no way to remove any entry (or until it has spent too much time). Once the garbage collector has switched to this more aggressive behavior, it will keep being aggressive for a few cycles (a bit less for each cycle). With a very large value of net.ipv4.route.gc_timeout, the garbage collector will have a hard time to find entries to expire. It will need to do several passes until it is able to expire some entries. On the other hand, if you choose a very small value, the garbage collector may remove entries that were just added, even if there are older entries further in the hash table. For a more in-depth coverage of how the route cache works, look at chapter 33 of Understanding Linux Network Internals. Just be aware that multipath route caching has been removed (and was never really used) and asynchronous cleanup (rt_periodic_timer) does not exist anymore.

Conclusion When tuning the route cache, rhash_entries, net.ipv4.route.gc_elasticity, net.ipv4.route.max_size and net.ipv4.route.gc_threshold are related and should not be modified independently. net.ipv4.route.gc_thresh should be below the product of net.ipv4.route.gc_elasticity and rhash_entries while net.ipv4.route.max_size should be above this value. Linux exposes several interesting metrics related to the route cache. Monitoring them allows one to watch the efficiency of the route cache and may uncover some anomalies (garbage collector running too often, difficulty to remove routes from the cache, ). The route cache subsystem still evolves and some old behaviors have been dismissed. The best documentation is, unfortunately, still the code and other documentations tend to become obsolete. The IPv6 route cache is quite different of the one presented here. Don t blindly apply the same tuning for IPv6.

---

1. Linux provides a protocol-independent destination cache subsystem (DST). This component is not a generic cache layer and only enables loose coupling with external subsystems.
2. Martian addresses are addresses that cannot be used as a source address, either because they are reserved for special-use (like a multicast address) or because of the use of reverse path filtering which checks if a packet received on one interface would be answered on the same interface, as defined in RFC 3704. This feature is enabled by setting rp_filter in Linux.
3. For more details, you may want to look at include/net/dst.h, include/net/route.h and net/ipv4/route.c.
4. In fact, the size of the hash table is always a power of two. If specified, rhash_entries is rounded to the next power of two and, internally, is stored as rt_hash_mask + 1.

Ben Hutchings is a rather unassuming guy but hiding behind his hat, there s a real kernel hacker who backports new drivers for the kernel in Debian stable so that our flagship release supports very recent hardware. Read on to learn more about Ben and the kernel team s projects for Debian Wheezy! Raphael: Who are you? Ben: I m a professional programmer, living in Cambridge, England with my long-suffering wife Nattie. In Debian, I mostly work on the Linux kernel and related packages. Raphael: How did you start contributing to Debian? Ben: I started using Debian in 1998 and at some point I subscribed to Debian Weekly News. So in 2003 I heard about the planned Debian 10th birthday party in Cambridge, and thought I would like to go to that. Somehow I persuaded Nattie that we should go, even though it was on the day of our wedding anniversary! We both enjoyed it; we made new friends and met some old ones (small world). From then on we have
both been socially involved in Debian UK. In 2004 there was a bug-squashing party in Cambridge, and we attended that as well. That s where I really started contributing fixing bugs and learning about Debian packaging. Then in 2005 I made my first package (sgt-puzzles), attended DebConf, and was persuaded to enter the New Maintainer process. NM involved a lot of waiting, but by the time I was given questions and tasks to do I had learned enough to get through quite quickly. In April 2006 I was approved as a Debian Developer. Meanwhile, I looked at the videos from DebConf 5 and thought that it would be useful to distribute them on a DVD. That led me to start writing video software and to get involved in the video team for the next year s DebConf. Raphael: You have been one the main driver behind the removal of non-free firmwares from the kernel. Explain us what you did and what s the status nowadays? Ben: That s giving me a bit more credit than I deserve. For a long time the easy way for drivers to load firmware programs was to include them as a blob in their static data, but more recently the kernel has included a simple method for drivers to request a named blob at run-time. These requests are normally handled by udev by reading from files on disk, although there is a build-time option to include blobs in the kernel. Several upstream and distribution developers worked to convert the older drivers to use this method. I converted the last few of these drivers that Debian included in its binary packages. In the upstream Linux source, those blobs have not actually been removed; they have been moved to a firmware subdirectory. The long-term plan is to remove this while still allowing the inclusion of blobs at build-time from the separate linux-firmware repository. For now, the Debian source package excludes this subdirectory from the upstream tarball, so it is all free software. There are still a few drivers that have not been converted, and in Debian we just exclude the firmware from them (so they cannot be built). And from time to time a driver will be added to the staging section of Linux that includes firmware in the old way. But it s understood in the kernel community that it s one of the bugs that will have to be fixed before the driver can move out of staging . Raphael: Do you believe that Debian has done enough to make it easy for users to install the non-free firmwares that they need? Ben: The installer, the Linux binary packages and initramfs-tools will warn about specific files that may be needed but are missing. Users who have enabled the non-free section should then be able to find the necessary package with apt-cache search, because each of the
binaries built from the firmware-nonfree source package includes driver and file names within its description. For the installer, there is a single tarball that provides everything. We could make this easier, but I think we have gone about as far as we can while following the Debian Social Contract and Debian policy. Raphael: At some point in the past, the Debian kernel team was not working very well. Did the situation improve? Ben: Back in 2008 when I started working on the Linux kernel package to sort out the firmware issues, I think there were some problems of communication and coordination, and quite possibly some members were burned-out. Since then, many of the most active kernel team members have been able to meet face-to-face to discuss future plans at LPC 2009 in Portland and the 2010 mini-DebConf in Paris. We generally seem to have productive discussions on the debian-kernel mailing list and elsewhere, and I think the team is working quite well. Several new contributors have joined after me. I would say our biggest problem today is that we just don t have enough time to do all we want to. Certainly, almost all my Debian time is now taken up with integrating upstream kernel releases and handling some fraction of the incoming bug reports. Occasionally I can take the time to work on actual features or the other packages I m neglecting!

Our biggest problem today is that we just don t have enough time to do all we want to.

Raphael: It is widely known that Linux is maintained in a git repository. But the Debian kernel team is using Subversion. I believe a switch is planned. Why was not git used from the start? Ben: The linux-2.6 source package dates from the time when Linus made his first release using git. I wasn t part of the team back then so I don t know for sure why it was imported to Subversion. However, at that time hardly anyone knew how to use git, no-one had experience hosting public git repositories, and Alioth certainly didn t offer that option. Today there are no real blockers: everyone on the kernel team is familiar with using git; Alioth is ready to host it; we don t have per-architecture patches that would require large numbers of branches. But it still takes time to plan such a conversion for what is a relatively complex source package (actually a small set of related source packages). Raphael: What are your plans for Debian Wheezy? Ben: Something I ve already done, in conjunction with the installer team, is to start generating udebs from the linux-2.6 source package. The kernel and modules have to be repacked into lots of little udebs to avoid using too much memory during installation. The configuration for this used to be in a bunch of separate source packages; these could get out of step with the kernel build configuration and this would only be noticed some time later. Now we can update them both at the same time, they are effectively cross-checked on every upload, and the installer can always be built from the latest kernel version in testing or unstable. I think that we should be encouraging PC users to install the 64-bit build (amd64), but many users will still use 32-bit (i386) for backward compatibility or out of habit. On i386, we ve slightly reduced the variety of kernel flavours by getting rid of 686 and making 686-pae the default (previously this was called 686-bigmem ). This means that the NX security feature will be used on all systems that support it. It should also mean that the first i386 CD can have suitable kernel packages for all systems. I have been trying to work on providing a full choice of Linux Security Modules (LSMs). Despite their name, they cannot be built as kernel modules, so every enabled LSM is a waste of memory on the systems that don t use it. This is a significant concern for smaller Debian systems. My intent is to allow all unused LSMs to be freed at boot time so that we can happily enable all of them. I recently proposed to drop support for older x86 systems, starting with 486-class processors in wheezy. In general, this would allow the use of more compiler optimisations throughout userland and the kernel. However it seems that there isn t that much to be gained unless we also drop 586-class processors, and there are still quite a few of those in use. So I think this will have to wait. Uwe Kleine-K nig has been working to include real-time support (also known as PREEMPT_RT). This can provide low and very predictable I/O latency, which is useful for live audio synthesis, for example. It still requires a number of patches and a build configuration change, resulting in a separate binary package. We re currently only building that for 64-bit PCs. (You may notice this is missing for Linux 3.1, because the real-time developers skipped this release.) Raphael: What s the biggest problem of Debian? Ben: I think we try too hard to accommodate every possible option, without regard for the cost to developers and users in general. As an example, we now have sysvinit, file-rc, upstart and systemd all in testing. Daemon maintainers can t rely on any advanced features of upstart or systemd because we refuse to choose between them. And the decision to support the FreeBSD kernel means that we cannot choose upstart or systemd as the only option. So all daemon maintainers will have to maintain those baroque init scripts for the indefinite future. We really should be able to decide as a distribution that when one option is technically good and popular then it can be made the only option. But no-one really has the authority to do that, so we muddle along with the pretence that all the options are equally valid and functional, while none of them is supported as well as they should be.

We try too hard to accommodate every possible option, without regard for the cost to developers and users in general.

We also try to build every package on every architecture, in general. I m quite sure there are many (package, architecture) combinations that have no users, ever. But if at some point that combination FTBFS, developers will waste time investigating and fixing that time that could have been spent working on bugs and features that users actually care about. Yes, sure, portability is good but you can t prove portability just by making a package compile on every architecture. This also applies to the selection of drivers for the kernel, by the way. Raphael: Is there someone in Debian that you admire for their contributions? Well, there are many people, but I will pick out just a few: Steve McIntyre, for his work as DPL to improve communication with the various Debian derivatives and to bring fresh blood into various core teams. Also for being a generous host for countless Debian social and bug-squashing events. Stefano Zacchiroli, for improving further on communications with both downstream and upstream projects, and for regularly exercising his power to lead discussions to the benefit of the project. Julien Cristau, for maintaining good humour while not only fighting against the tide of graphics driver regressions in X and the Linux kernel but also working on release management. Jonathan Nieder, for taking on the unglamorous and frustrating task of kernel bug triage as a non-maintainer and developing it to a fine art.

---

Thank you to Ben for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

4 comments Liked this article? Click here. My blog is Flattr-enabled.

Mako's IronBlogger is a great idea. I often find myself postponing blog posts for a very long time, simply out of laziness. IronBlogger provides a nice community incentive to counter (my) laziness and blogging more often. As a related challenge, we have to face the fact that different subsets of our communities use different media to stay informed: mailing lists, blog (aggregators), social media, IRC, etc. Disparities in how they stay informed are a pity and can be countered using multiple medias at a time. Although I haven't blogged very often as of lately, I managed to keep the Debian (Developer) community informed of what happens in "DPL land" on a monthly basis, by the means of bits from the DPL mails sent to d-d-a. While the target of bits mails perfectly fits d-d-a, there is no reason to exclude a broader public from them. After all, who knows, maybe we'll find the next DPL victim^W candidate among Planet readers! Bonus point: blogging this also helped me realize that my mails are not as markdown-clean as I thought they were. I still have no IronBlogger squad, though. (And sharing beers with folks in the Boston area is not terribly handy for me ). Anyone interested in setting up a BloggeurDeFer in the Paris area? (SCNR)

---

Dear Project Members,
another month has passed, it's time to bother you again about what has happened in DPL land in November (this time, with even less delay than the last one, ah!). Call for Help: press/publicity team I'd like to highlight the call for help by the press / publicity teams. They are "hiring" and have sent out a call for new members a couple of weeks ago. The work they do is amazing and is very important for Debian, as important as maintaining packages or fixing RC bugs during a freeze. It is only by letting the world know what Debian is and what we do, that we can keep the Project thriving. And letting the world know is exactly what the publicity and press teams do. If you're into writing, blogging, or simply have a crush for social media, please read the call and "apply"! Interviews November has apparently been the "let's interview the DPL" month. I've spent quite some time giving interviews to interested journalists about various topics. For both my embarrassment and transparency on what I've said on behalf of Debian, here are the relevant links:

• audio interviewwith Karen Sandler for the Free as in Freedom oggcast by herself and Bradley M. Kuhn
• text interview with Raphael Hertzog for the People behind Debian series
• text interview for the Linux Certification Institute Italy (LICI) (note: the original of this one is in Italian; the English version is a translation by LICI)
• video interview with Amber Granger during last UDS

Assets

• the Video Team sprint approved in October has happened at beginning of November; report is still pending but the organizers are on it
• Christian Perrier reported about his trip to India mini-DebConf, even that he attended on behalf of the Project. I find amazing both the outcome of the event and Christian's report, a very good example of how to inform the Project when acting on behalf of it
• the hardware replacement for ftp,lists .d.o I've mentioned last month has been ordered, has arrived, and DSA have already put their hands on it for the setup. Many thanks to DSA and in particular to Martin Zobel-Helas for the logistics
• as an administratrivia, I've asked SPI to setup for Debian a "gift" credit card, i.e. a credit card with a limited, refillable budget. It is needed for using (with a safeguard) credits offered to the Project by public cloud providers (e.g. AWS' donation), as well as to use various kinds of services that we need from time to time

Legal advice (work in progress)

• I've sought and got from a former SPI lawyer a first legal advice on the relationships among trademark law and the procedures of a Free Software distro like Debian when distributing software subject to trademarks. I'm still waiting to seek a second advice from the current SPI lawyers (whom I've been keeping busy with other stuff). In the meantime I've pending a summary of what I've got thus far in the trademark-vs-DFSG thread. Stay tuned there!
• I've been working with SPI lawyers on a proposal of software patent policy that I think we should advertise to make explicit the position of the Debian Project on software patents. I see it as a continuation/completion of the FAQ we have already published on the subject. I'll let you know as soon as we reach a stable draft, that should be pretty soon.

Relationships with others

• I've been approached by members of the board of the OpenSource Initiative (OSI), about the soon to be open OSI affiliation program. They'd like to know if Debian is interested in being affiliated to OSI. I think we should and I'll eventually start a discussion on this topic. In the meantime, I welcome feedback from anyone who is interested in the topic.
• Thanks to the interest of Andrew Pollock, LaMont Jones, Florian Weimer and the Security Team, we are discussing with ISC to have Debian --- as a project, rather than as individuals who just happen to maintain ISC software in Debian --- become member of all forums relevant for software we distribute (BIND, NTP, DHCP). They are doing all the coordination work, but I've anticipated I'll be happy to pledge for Debian membership.
• I've taken the first step to reach out to companies that have a strategic interest in Debian. According to discussion I've had with representative of such companies, they often face similar challenges in offering Debian support and in contributing back to Debian. I think we should offer a forum for them to discuss and find synergies, as well as explain our needs to them. An official mailing list might be such a place. Once we have a suitable forum ready, I'll propose to the press team to send out a corresponding call for companies. In the meantime, if you happen to know, or even work for, companies that might be interested, please ask them to get in touch with me.

Miscellanea

• I've had a chance to discuss with the editors of the SPDX specification about its relationship with DEP-5. Thanks to the help of Dominique Dumont work is already in progress to have a mutual converter among the two formats. Also, although with some bumps, work on finalizing DEP-5 has finally restarted.
• an interesting thread of a while ago is in need of some work to produce a useful outcome, i.e. a set of security guidelines for DDs. If you're willing to help, have a look at the thread
• some more mediation-fu has been going on during November; have a look at my daily logs below if you care about the details

Thanks for reading thus far,
and happy hacking.

---

PS as usual, the boring day-to-day activity log is available at master:/srv/leader/news/bits-from-the-DPL.*

Perhaps the immortal bard was making a more noble and deep reference to the stages of ones life but I feel no compunction lifting the line for my purpose.

Long time has passed since last entry in this blog. Moderation panel is full of spam comments. It s still unclear if I will ever will continue it. However, given what is happening in my country these days, I *must* write at least one entry, just to put a number there. 632305222316434 Everyone who disagrees with criminal behaviour of Russian president, government and all those rascals, must do that.

It s been busy lately, and I haven t had the time to blog. With the change in job, various travel, and settling into a new routine, I ve not done as much writing of late. But life marches on, and before memories grow too fleeting, I think I should share a few. We recently changed the arrangements for the boys. Instead of them each having their own room in which they sleep and sometimes play, we purchased a bunk bed. Oliver graduated from his crib to the lower bunk, and Jacob has the higher bunk. This has, predictably, created a few opportunities for behavior issues. Overall, it s going well, and they appreciate their new, more open, play room. Both boys sleep with their stuffed animals. Jacob calls his my friends . He still likes his butterfly, which he has had since he was an infant. He sometimes talks about how much he loves his friends, and how they like to get hugs, and how they are happy. Jacob continues to enjoy reading. He has a toy low-res camera and he even recorded a video of himself reading Brown Bear, Brown Bear. Oliver s vocabulary is coming alive and is fun to watch. Jacob has taken to trying to teach Oliver how to say things. One day, Jacob saw a number like 451 on the side of a train, producing a conversation like this: Oliver, can you say four hundred and fifty-one? Four dred iffy on! You got it!!! YAY! Today as I was walking past Oliver s train track on the floor, he grabbed me by the hand, had me sit down, and kept holding on to make sure I d stay right where he wanted me as he pointed and talked all about his trains. Aww. One cold and windy Saturday morning, the boys were getting restless. What to do, we thought? I decided to bring out one of the manual typewriters from my collection. Oliver loved watching it do things as he pressed buttons. Jacob enjoyed spelling mom . New problem: boys fighting over how long each one s turn at the typewriter is. This has gone on for a month now. Ahh, winter. A few weeks ago, Jacob informed us that he built an antenna out of blocks. He was REALLY proud of it, and even, incredibly, insisted I take his picture with it! I m pleased to have a 5-year-old that calls this structure an antenna instead of a skyscraper or tower or some such thing ;-) We took a train trip to Portland, OR, recently. That s about 2.5 days on the train each way. It went pretty well we had quite a bit of excitement though it got a little long for the boys at times. One evening, Jacob excitedly noted that the sky was almost really dark blue, just like my song! Wow. That was a song he made up in New York in summer 2010. Jacob enjoyed collecting leaves as we walked around in Portland. He would then stash his pile of leaves outside the door of whatever building we d enter, then hope to find them still there when we got back out. It usually worked out OK for him.

At work we use .procmailrc files generated by CGIpaf to let non-technical users create forwards, out-of-office mails, etc. and any combination thereof. This also has the advantage that we can filter out double bounces and spam (which also prevents us from being listed in spammer blacklists). Unfortunately autofs (seems independent if autofs4 or autofs5 is used) seems to be unreliable if there are bursts of mount or umount requests, resulting either in File or directory not found error message while trying to access the home directory of a user, or Directory not empty error messages if the automounter tries to remove the mount point after unmounting. In that case a not mounted directory owned by root is left over. In the end both cases lead to procmail behaving as if that user does not have a .procmailrc which looks like sporadically lost mails to those who forward all mails. (The mails then can be found in the local default INBOX for that user.) Additionally there are similar issues when the NFS servers are not available. The most effective countermeasure we found so far was adding tests to the global /etc/procmailrc to check if the user s home directory exists and belongs to the correct user:

# -----------------
# Global procmailrc
# -----------------
# For debugging, turn off if everything works well
VERBOSE=1
LOGFILE=/var/log/procmail.log
# This only works with bourne shells, $SHELL defaults to the user's
# login shell. And by experience dash seems not work, so we use bash.
OLDSHELL=$SHELL
SHELL=/bin/bash
# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $LOGNAME is not set for some reason. (Just to be sure our paths
# later on are not senseless.
:0
* ? test -z "$LOGNAME"
 
    LOG="Expected variable LOGNAME not set. "
    EXITCODE=75
    :0
    /dev/null
 
# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $HOME is not readable. ~$LOGNAME does not seem to work, so this uses
# a hard wired /home/.
:0
* ? test ! -r /home/$LOGNAME
 
    LOG="Home of user $LOGNAME not readable: /home/$LOGNAME "
    EXITCODE=75
    :0
    /dev/null
 
# temporary failure (see EX_TEMPFAIL in /usr/include/sysexits.h) if
# $HOME has wrong owner. ~$LOGNAME does not seem to work, so this uses
# a hard wired /home/.
:0
* ? test ! -O /home/$LOGNAME
 
    LOG="Home of user $LOGNAME has wrong owner: /home/$LOGNAME "
    EXITCODE=75
    :0
    /dev/null
 
[ ]

If you want to store a copy of these mails for debugging purposes on every delivery attempt, replace /dev/null with some Maildir or mbox only accessible for root. One small but important part was to explicitly declare bash as shell for executing the tests, otherwise mails for users with tcsh or zsh as login shell filled up the mail queue and never get delivered (if the SHELL variable never gets fixed). Only drawback so far: This leads to more lagging e-mail on e-mail bursts also for those users who have no .procmailrc because procmail can t check if there s really no .procmailrc. Extensive procmail documentation can be found online at the Procmail Documentation Project as well as in the man pages procmail(1), procmailrc(5) and procmailex(5).

My current running Xorg server, built with -O0 -g3 -ggdb, having been through a few suspend cycles, hosting a full GNOME Shell session with several billion tabs in a long-running Chromium instance, a couple of terminals, and a freshly-opened gedit, is currently sitting at 20MB resident memory usage, of which 11MB is shared. The freshly-opened gedit instance has 25MB resident, of which 17MB is shared. Bloat is in the eye of the beholder, I guess.

(Those numbers are RES/SHR respectively in top. Correct me if I'm wrong about what they mean.)

Could you not trust this man?Max and Fraser are very interested in this year's election. I guess political awareness starts at the first election after you turn 10, or so. They're both very curious about how mum and dad are voting, and while their mother will not tell (though we all have our suspicions :-) I don't see any reason not to discuss my political leanings with them.I've been what is commonly termed a 'floating' voter for most of my voting life. I imagine the politicians themselves might term people like me 'floaters', with an eye to the scatalogical implications. After all, their livelihoods are on the line!So, after reading this blog post I thought I might as well also share my personal voting plans with the world.

It s been one year since the first People behind Debian interview. For this special occasion, I wanted a special guest and I m happy that our Debian Project Leader (DPL) Stefano Zacchiroli accepted my invitation. He has a difficult role in the community, but he s doing a really great job of it. He s a great mediator in difficult situations, but he s also opinionated and can push a discussion towards a conclusion. Read on to learn how he became a Debian developer and later DPL, what he s excited about in the next Debian release, and much more. Raphael: Who are you? Stefano: I m Stefano Zacchiroli, but I prefer to be called Zack, both on the Internet and in real life. I m 32, Italian, emigrated to France about 4 years ago. I live in Paris, and I find it to be one of the most gorgeous and exciting cities in the world. As my day job I m a Computer Science researcher and teacher at University Paris Diderot and IRILL. In my copious free time I contribute to Debian, and I m firmly convinced that doing so is an effective way to help the cause of Free Software. Besides, I find it to be a lot of fun! Raphael: How did you start contributing to Debian? Stefano: Flash back to 1999, when I was a 2nd year student in Computer Science at the University of Bologna. Back then in Italy it was uncommon for young geeks to get exposed to Free Software: Internet was way less pervasive than today and most computer magazines didn t pay much attention to GNU/Linux. Luckily for me, the professor in charge of the student lab was a Free Software enthusiast and all students machines there were running Debian. Not only that, but there was also a student program that allowed volunteers to become sysadmins after having shown their skills and convinced the director they were trustworthy. Becoming one of those volunteer Debian admins quickly became one of my top objectives for the year, and that is were I ve learned using Debian. The year after that, I got in touch with a research group that was to become the happy bunch of hackers with whom I would have done both my master and PhD theses. They were designing a new proof assistant. Most of the development was in OCaml and happened on Debian. OCaml was available in Debian, but many of the libraries we needed were not. So I approached the Debian OCaml Team offering to help. Before I realize what was going on I was (co-)maintainer of tens of OCaml-related packages. At some point I got told I think you should apply as a Debian Developer . So I did and in a couple of months I went through the New Member (NM) process, that was back then in its infancy. I still remember my happiness while reading the account created mail, the day after my 22nd birthday. I know the NM process went through some bad publicity in the past, but I m happy to see that nowadays the process can be as swift as it has been for me 10 years ago. Raphael: It s your second year as Debian Project Leader (DPL). Are you feeling more productive in the role? Do you fear to burn out? Stefano: I m feeling way more productive, no doubts. The task of the Debian Project Leader is not necessarily difficult, but it is a complex and scarcely documented one. It is also profoundly different from any other task that Debian people usually work on, so that experience doesn t help much in getting started. Before becoming effective as DPL one needs to get to know many people and mechanisms he is not familiar with. More importantly, one needs to set up a personal work-flow that allows to keep up with day-to-day DPL tasks (which are aplenty) as well as with urgencies (that tend to pop-up in the leader@debian.org INBOX at the least convenient time). Finally, one also needs to do proper traffic shaping and always retain enough motivation bandwidth to keep the Project informed about what is going on in DPL-land. Finding the right balance among all these ingredients can take some time. Once one is past it, everything goes way more smoothly. The above is why I m constantly encouraging people interested in running for DPL in the future to reach out to me and work on some tasks of the current DPL s TODO list. I swear it is not just a cheap attempt at slavery!. It is rather an attempt at DPL mentoring that could be beneficial: both to give future candidates more awareness of the task, and to reduce the potential downtime when handing over from one DPL to the next. Regarding burn out, I don t feel prone to its risk these days. If I look back, I can say that my contributions as DPL have been pretty constant in volume over time; my enthusiasm for the task, if anything, is on the rise. The effectiveness of my contributions as DPL are, on the other hand, not mine to judge. Raphael: If you had to single out two achievements where you were involved as DPL, what would they be? Stefano: I d go for the following two, in no particular order:

• Dialogue with derivatives. When I became DPL ~1.5 years ago the situation on that front was pretty dire. In the specific case of Ubuntu, by far the most successful and customized of all Debian derivatives, I remember being scared of raising the topic of collaboration with them on mailing lists. More generally, we had no specific initiatives to foster technical collaboration with and among derivatives. A huge potential of (forwarded) contributions to Debian was being wasted. Today things look much better, as I ve documented in recent talks at DebConf11 and UDS-P. The amount of forwarded patches we receive from downstream is at its maximum and many people who apply to become Debian Developers come from derivatives. Conflict situations still exist, for good reasons that we still have to either fix or figure out entirely. But I m positive we re on the right track.

  The amount of forwarded patches we receive from downstream is at its maximum

  This is by far not an achievement of mine alone. In particular, many of the activity of the Derivatives Front Desk have been organized by other enthusiastic volunteers. But I ve done my part, especially in breaking the ice and in proposing a vision of Free Software distribution where all distros play a role and are welcome to join the game, as long as they give back and give credit to their respective upstreams.
• Process membership. I m proud of having promoted the general resolution (GR) that has clarified (and advertised to the world) that Debian welcomes all kind of contributions, and that they all equally matter to become proper members of the Project. I notice only now while writing this that the GR title, traditionally chosen by the secretary, was Debian Project Members . That choice harmonically closes the circle with the recent renaming of the NM process to New Member process. Today, we have several project members (AKA Debian Developers ) that are active citizen of Debian with voting rights, even though they take care of tasks other than packaging. Anyone can become a Debian citizen, as long as they are ready to abide by Debian s values, have a track record of verifiable contributions to Debian, and are committed to keep them coming in the future.

  We have several project members that [ ] take care of tasks other than packaging.

  Once gain, this is by far not an achievement of mine alone, very little project-wide achievements are. DAM has helped a lot and support from the project as a whole has been immense.

OK, let me cheat and add a third one I m also proud of having been able to report to the Project my whereabouts as DPL, thoroughly and periodically, since the very beginning is first term. People annoyed by my reporting logorrhea now have all my sympathies. Raphael: Among the possible new features of Debian Wheezy, which one gets you excited most? Stefano: It s multi-arch, no doubt. Even though it is not a directly user visible change, it s a very far reaching one. It is also one of those changes that make me feel that moment of truth of coders, when you realize you are finally doing the right thing and ditching piles of ugly hacks.

It s multi-arch [ ] you realize you are finally doing the right thing and ditching piles of ugly hacks.

Raphael: If you were not DPL and could spend all your time on Debian, what project would you do? Stefano: I would sit down and do software development for Debian. It s impressive how many important and beneficial changes for Debian could be delivered by specific software improvements in various parts of our infrastructure. We tend to attract many packagers, but not so many people willing to maintain Debian infrastructure softwares like dak, britney, debbugs, the PTS, etc. Their maintenance burden then falls on the shoulders of the respective teams which are generally very busy with other important tasks. As a project, we seem to be more appealing to packagers than to software developers. That is a pity given the amount of exciting coding tasks that are everywhere in Debian. Part of the reason we are not appealing to developers is that we are not particularly good at collecting coding tasks in a place where interested developers could easily pick them up. It also takes quite a bit of inside knowledge to spot infrastructure bugs and understand how to fix them. I long for some spare hacking time to check if I m still good enough of a coder to hunt down longstanding bugs in our infrastructure, which have ended up being my pet peeves. I d also love to dive again into RCBW. It s less committing than package maintenance, more diverse and challenging, and also an immensely useful activity to get Debian releases done. Raphael: Martin Michlmayr is worried that there is so few paid opportunities around Debian. Do you agree with his sentiment, and if yes do you have ideas on how to improve this situation? Stefano: The idealistic me wishes Debian to be a community made only of volunteers that devote their free time to the Project. Oh, and that me also wishes Debian to be competitive with similar projects, no matter how many full-time employees others have! That is coherent with a view of society where everyone has a day job, but also engages in volunteering activities ensuring that public interest is pursued by people motivated by interests other than profit. But I do realize that for Free Software to succeed companies, employees, and salaries should all have a role. I admire projects that strike a good balance between volunteer and paid work. The Linux kernel is emblematic in that respect: many developers are paid by companies that have a commercial or strategic interest in Linux. Nevertheless volunteers contributions are aplenty and the Linux community gives a convincing impression that choices are driven by the community itself (or by its benevolent dictator) without money-driven impositions.

I do realize that for Free Software to succeed companies, employees, and salaries should all have a role.

Such an ecosystem does not exist around Debian. We do have a partner program that allows for it to happen, but we have very few partners with an interest in doing distribution development work. Like Martin, I m worried by this state of affairs, because it de facto means we lag behind in terms of available people power. In a community of volunteers, that might frustrate people and that is not good. To improve over the status quo the first step is to federate together small and medium companies that have a strategic interest in Debian and listen to their needs. I m already in touch with representatives of such companies that, in many cases, already employ Debian Developers to do some distribution work in Debian. We will be soon sending out a call to reach out to more such companies, but since we are discussing this, why waiting? If some of our readers here are representative of such companies, I encourage them to get in touch with me about this. Raphael: You know that the fundraising campaign for the Debian Administrator s Handbook is on good track but the liberation of the book is not yet assured. What do you think of this project? Stefano: I m happy about the project, to the point that I ve accepted writing a testimonial for it . I m sad about the scarce availability of up to date and high quality (DFSG-)Free books about Debian and I welcome any initiative that might help closing that gap.

I m sad about the scarce availability of up to date and high quality (DFSG-)Free books about Debian.

Free Culture is a great offspring of Free Software and I m convinced we need to stand up against double standards in the two camps. Letting aside software-specific licensing details, the basic freedoms to be defended are the same. They are those freedoms that ensure that a reader is in full control of his book, pretty much as they ensure that a computer user is in full control of the software that runs on it. I m therefore proud that Debian has long resolved that the Debian Free Software Guidelines (DFSG) apply not only to software but also to books and other pieces of documentation. But the status quo implies that not only we have very few up to date, high quality books about Debian. It also implies that, at present, we have no such book that we can distribute in the Debian archive, showing off the Free Software (and Free Culture!) values we stand for.
Crowdfunding is considered to be a good mate for Free Culture, where the services model that applies to Free Software is more difficult to exploit. I so wish any luck to yours and Roland s initiative. A different matter is whether Debian, as a project, should endorse the initiative and actively campaign for it. As you know, I think it should not. While we do advertise general project donations, we don t do mission-specific fundraising campaign for Debian itself. Coherently with that, I don t think we should relay crowdfunding campaigns for 3rd parties, even when the result would be beneficial to Debian. Raphael: Is there someone in Debian that you admire for their contributions? Stefano: There are two classes of people that I particularly admire in Debian:

• Those with an uncanny ability to guide discussions towards constructive conclusions. We are lucky to have many in Debian and I admire all of them. Having to single out one I d name Russ Allbery, in honor of whom I hereby propose the periodic Russ Allbery s Distinguished Flametamer Award .
• People stepping up for responsibility roles, especially when the responsibility put them in tough spots. Release teams, ftp-masters, DSA, DAM, as well as past and present members of teams with similarly hot seats have all my admiration.

---

Thank you to Zack for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

7 comments Liked this article? Click here. My blog is Flattr-enabled.

Yesterday I wrote about an alleged attack on the Windows 8 secure boot implementation. As I later clarified, it turns out that the story was, to put it charitably, entirely wrong. The attack is a boot kit targeted towards BIOS-based boots. It lives in the MBR. It'll never be executed on any UEFI systems, let alone secure boot ones. In fact, this is precisely the kind of attack that secure boot is intended to protect against. So, context.

The MBR contains code that's executed by the BIOS at boot time. This code is unverifiable - it's permitted to have arbitrary functionality. There's only 440 bytes, but that's enough to jump to somewhere else and read code from elsewhere. There's no way for the BIOS to know that this code is malicious. And one thing this code can obviously do is load the normal boot code and modify it to behave differently. Any self-validation code in the loader can be patched out at this point. The modified loader will then load the kernel, and potentially also modify it. At this point, you've lost. Any attempts to validate the code can be redirected to the original code and so everything will look fine, up until the point where the user runs a specific application and suddenly your kernel is sending all your keystrokes over UDP to someone in Nigeria.

These attacks exist now. They're in the wild. In a normal UEFI world you'd do the same thing by just replacing the UEFI bootloader. But with secure boot you'll be able to validate that the bootloader is appropriately signed and if someone's modified it you'll drop into some remediation mode that recovers your files, from install media if necessary.

Obviously, this protection is based on all the components of secure boot (ie, everything that runs before ExitBootServices() is called) being perfect. As I said, if any of them accept untrusted input and misinterpret it in such a way that they can be tricked into running arbitrary code, you'll still have problems. But when discussing the pros and cons of secure boot, it's important to make sure that we're talking about reality rather than making provably false assertions.

comments

I probably don t have to present Mark Shuttleworth he was already a Debian developer when he became millionaire after having sold Thawte to Verisign in 1999. Then in 2002 he became the first African (and first Debian developer) in space. 2 years later, he found another grandiose project to pursue: bring the Microsoft monopoly to an end with a new alternative operating system named Ubuntu (see bug #1). I have met Mark during Debconf 6 in Oaxtepec (Mexico), we were both trying to find ways to enhance the collaboration between Debian and Ubuntu. The least I can say is that Mark is opinionated but any leader usually is, and in particular the self-appointed ones! Read on to discover his view on the Ubuntu-Debian relationship and much more. Raphael: Who are you? Mark: At heart I m an explorer, inventor and strategist. Change in technology, society and business is what fascinates me, and I devote almost all of my time and wealth to the catalysis of change in a direction that I hope improves society and the environment. I m 38, studied information systems and finance at the University of Cape Town. My hearts home is Cape Town, and I ve lived there and in Star City and in London, now I live in the Isle of Man with my girlfriend Claire and 14 precocious ducks. I joined Debian in around 1995 because I was helping to setup web servers for as many groups as possible, and I thought Debian s approach to packaging was very sensible but there was no package for Apache. In those days, the NM process was a little easier Raphael: What was your initial motivation when you decided to create Ubuntu 7 years ago? Mark: Ubuntu is designed to fulfill a dream of change; a belief that the potential of free software was to have a profound impact on the economics of software as well as its technology. It s obvious that the technology world is enormously influenced by Linux, GNU and the free software ecosystem, but the economics of software are still essentially unchanged. Before Ubuntu, we have a two-tier world of Linux: there s the community world (Debian, Fedora, Arch, Gentoo) where you support yourself, and the restricted, commercial world of RHEL and SLES/SLED. While the community distributions are wonderful in many regards, they don t and can t meet the needs of the whole of society; one can t find them pre-installed, one can t get certified and build a career around them, one can t expect a school to deploy at scale a platform which is not blessed by a wide range of institutions. And the community distributions cannot create the institutions that would fix that. Ubuntu brings those two worlds together, into one whole, with a commercial-grade release (inheriting the goodness of Debian) that is freely available but also backed by an institution. The key to that dream is economics, and as always, a change in economics; it was clear to me that the flow of money around personal software would change from licensing ( buying Windows ) to services ( paying for your Ubuntu ONE storage ). If that change was coming, then there might be room for a truly free, free software distribution, with an institution that could make all the commitments needed to match the commercial Linux world. And that would be the achievement of a lifetime. So I decided to dedicate a chunk of my lifetime to the attempt, and found a number of wonderful people who shared that vision to help with the attempt. It made sense to me to include Debian in that vision; I knew it well as both a user and insider, and believed that it would always be the most rigorous of the community distributions. I share Debian s values and those values are compatible with those we set for Ubuntu.

Debian would always be the most rigorous of the community distributions.

Debian on its own, as an institution, could not be a partner for industry or enterprise. The bits are brilliant, but the design of an institution for independence implies making it difficult to be decisive counterparty, or contractual provider. It would be essentially impossible to achieve the goals of pre-installation, certification and support for third-party hardware and software inside an institution that is designed for neutrality, impartiality and independence. However, two complementary institutions could cover both sides of this coin. So Ubuntu is the second half of a complete Debian-Ubuntu ecosystem. Debian s strengths complement Ubuntu s, Ubuntu can achieve things that Debian cannot (not because its members are not capable, but because the institution has chosen other priorities) and conversely, Debian delivers things which Ubuntu cannot, not because its members are not capable, but because it chooses other priorities as an institution. Many people are starting to understand this: Ubuntu is Debian s arrow, Debian is Ubuntu s bow. Neither instrument is particularly useful on its own, except in a museum of anthropology

Ubuntu is Debian s arrow, Debian is Ubuntu s bow.

So the worst and most frustrating attitude comes from those who think Debian and Ubuntu compete. If you care about Debian, and want it to compete on every level with Ubuntu, you are going to be rather miserable; you will want Debian to lose some of its best qualities and change some of its most important practices. However, if you see the Ubuntu-Debian ecosystem as a coherent whole, you will celebrate the strengths and accomplishments of both, and more importantly, work to make Debian a better Debian and Ubuntu a better Ubuntu, as opposed to wishing Ubuntu was more like Debian and vice versa. Raphael: The Ubuntu-Debian relationship was rather hectic at the start, it took several years to mature . If you had to start over, would you do some things differently? Mark: Yes, there are lessons learned, but none of them are fundamental. Some of the tension was based on human factors that cannot really be altered: some of the harshest DD critics of Canonical and Ubuntu are folk who applied for but were not selected for positions at Canonical. I can t change that, and wouldn t change that, and would understand the consequences are, emotionally, what they are. Nevertheless, it would have been good to be wiser about the way people would react to some approaches. We famously went to DebConf 5 in Porto Allegre and hacked in a room at the conference. It had an open door, and many people popped a head in, but I think the not-a-cabal collection of people in there was intimidating and the story became one of exclusion. If we d wanted to be exclusive, we would have gone somewhere else! So I would have worked harder to make that clear at the time if I d known how many times that story would be used to paint Canonical in a bad light. As for engagement with Debian, I think the situation is one of highs and lows. As a high, it is generally possible to collaborate with any given maintainer in Debian on a problem in which there is mutual interest. There are exceptions, but those exceptions are as problematic within Debian as between Debian and outsiders. As a low, it is impossible to collaborate with Debian as an institution, because of the design of the institution.

It is generally possible to collaborate with any given maintainer [ ] [but] it is impossible to collaborate with Debian as an institution.

In order to collaborate, two parties must make and keep commitments. So while one Debian developer and one Ubuntu developer can make personal commitments to each other, Debian cannot make commitments to Ubuntu, because there is no person or body that can make such commitments on behalf of the institution, on any sort of agile basis. A GR is not agile . I don t say this as a critique of Debian; remember, I think Debian has made some very important choices, one of those is the complete independence of its developers, which means they are under no obligation to follow a decision made by anyone else. It s also important to understand the difference between collaboration and teamwork. When two people have exactly the same goal and produce the same output, that s just teamwork. When two people have different goals and produce different product, but still find ways to improve one anothers product, that s collaboration. So in order to have great collaboration between Ubuntu and Debian, we need to start with mutual recognition of the value and importance of the differences in our approach. When someone criticises Ubuntu because it exists, or because it does not do things the same way as Debian, or because it does not structure every process with the primary goal of improving Debian, it s sad. The differences between us are valuable: Ubuntu can take Debian places Debian cannot go, and Debian s debianness brings a whole raft of goodness for Ubuntu. Raphael: What s the biggest problem of Debian? Mark: Internal tension about the vision and goals of Debian make it difficult to create a harmonious environment, which is compounded by an unwillingness to censure destructive behaviour. Does Debian measure its success by the number of installs? The number of maintainers? The number of flamewars? The number of packages? The number of messages to mailing lists? The quality of Debian Policy? The quality of packages? The freshness of packages? The length and quality of maintenance of releases? The frequency or infrequency of releases? The breadth of derivatives? Many of these metrics are in direct tension with one another; as a consequence, the fact that different DD s prioritise all of these (and other goals) differently makes for interesting debate. The sort of debate that goes on and on because there is no way to choose between the goals when everyone has different ones. You know the sort of debate I mean Raphael: Do you think that the Debian community improved in the last 7 years? If yes, do you think that the coopetition with Ubuntu partly explains it? Mark: Yes, I think some of the areas that concern me have improved. Much of this is to do with time giving people the opportunity to consider a thought from different perspectives, perhaps with the benefit of maturity. Time also allows ideas to flow and and of course introduces new people into the mix. There are plenty of DD s now who became DD s after Ubuntu existed, so it s not as if this new supernova has suddenly gone off in their galactic neighbourhood. And many of them became DD s because of Ubuntu. So at least from the perspective of the Ubuntu-Debian relationship, things are much healthier. We could do much better. Now that we are on track for four consecutive Ubuntu LTS releases, on a two-year cadence, it s clear we could collaborate beautifully if we shared a freeze date. Canonical offered to help with Squeeze on that basis, but institutional commitment phobia reared its head and scotched it. And with the proposal to put Debian s first planned freeze exactly in the middle of Ubuntu s LTS cycle, our alignment in interests will be at a minimum, not a maximum. Pure <facepalm />. Raphael: What would you suggest to people (like me) who do not feel like joining Canonical and would like to be paid to work on improving Debian? Mark: We share the problem; I would like to be paid to work on improving Ubuntu, but that s also a long term dream Raphael: What about using the earnings of the dormant Ubuntu Foundation to fund some Debian projects? Mark: The Foundation is there in the event of Canonical s failure to ensure that commitments, like LTS maintenance, are met. It will hopefully be dormant for good Raphael: The crowdfunding campaign for the Debian Administrator s Handbook is still going on and I briefly envisioned the possibility to create the Ubuntu Administrator s Handbook. What do you think of this project? Mark: Crowdfunding is a great match for free software and open content, so I hope this works out very well for you. I also think you d find a bigger market for an Ubuntu book, not because Ubuntu is any more important than Debian but because it is likely to appeal to people who are more inclined to buy or download a book than to dive into the source. Again, this is about understanding the difference in audiences, not judging the projects or the products. Raphael: Is there someone in Debian that you admire for their contributions? Mark: Zack is the best DPL since 1995; it s an impossible job which he handles with grace and distinction. I hope praise from me doesn t tarnish his reputation in the project!

---

Thank you to Mark for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

32 comments Liked this article? Click here. My blog is Flattr-enabled.