Forbes has an interesting article about crowd-sourcing by criminals and law enforcement [1]. Ulissescastr0 made a Youtube video showing how to install SE Linux on Debian/Etch [2]. Probably no-one is using Etch nowadays so this video is outdated, but it s a good way of teaching people. It would be good if someone made a similar video showing how to do SE Linux things on Squeeze. I discovered the above SE Linux video through Explow which provides a neat interface to multiple searches and information sources [3]. I don t think I will be using Explow much in future as I could get the same result through Google video search. They also have a news portal but there are other sites for that. But it does seem that Explow would be useful for newbies. Eric Michael Johnson wrote an interesting article about the inherent bias in Psychological research based in the US [4]. People who live in urban environments think differently in some ways to people who live in different environments or who have different lifestyles. Therefore generalising from university students in the US to the entire human race is likely to get incorrect results. This is something to consider the next time you are tempted to generalise to the wider population from your own friends, colleagues, etc. The Daily Kos has a scary article about the TSA having a woman detained for reciting part of the US constitution [5]. The US will remain on my list of countries to avoid for the forseeable future. Vorlon has written an informative article about the use of hardening options when building Debian packages [6]. It s now even easier to do this, so every package that simultaneously deals with data of differing levels of integrity or sensitivity should be built this way. Bunker Roy gave an interesting TED talk about his Barefoot College that teaches useful skills to people in rural parts of India who don t have a traditional school education [7]. His talk really shows up some of the arrogance in the people who run traditional education. Justin Hall-Tipping gave an interesting TED talk about ways of solving the world energy problems [8]. He started with explaining the problems and why they need to be urgently solved and then described in detail some of the research that his group has done to solve the problems. This includes flexible photo-voltaic cells, infra-red vision to save on lighting, and a way of using carbon nano-tubes to control the thermal properties of windows.

• [1] http://www.forbes.com/sites/oreillymedia/2011/10/03/the-rise-of-crime-sourcing/
• [2] http://www.youtube.com/watch?v=FLPx7HLLteI
• [3] http://explow.com
• [4] http://blogs.plos.org/blog/2010/09/23/reflections-on-the-weird-evolution-of-human-psychology/
• [5] http://www.dailykos.com/story/2011/10/18/1027775/-TSA-Arrests-Me-for-Using-the-Fourth-Amendment-as-a-Weapon-(Tales-from-the-Edge-of-a-Revolution-2)?via=siderec
• [6] http://web.dodds.net/~vorlon/wiki/blog/Debian:_not_stale_just_hardened/
• [7] http://www.ted.com/talks/bunker_roy.html
• [8] http://www.ted.com/talks/justin_hall_tipping_freeing_energy_from_the_grid.html

Once the private key of some HTTPS web site is compromised, an attacker is able to build a man-in-the-middle attack to intercept and decrypt any communication with the web site. The first step against such an attack is the revocation of the associated certificate through a CRL or a protocol like OCSP. Unfortunately, the attacker could also have recorded past communications protected by this private key and therefore decrypt them. Forward secrecy allows today information to be kept secret even if the private key is compromised in the future. Achieving this property is usually costly and therefore, most web servers do not enable it on purpose. Google recently announced support of forward secrecy on their HTTPS sites. Adam Langley wrote a post with more details on what was achieved to increase efficiency of such a mechanism: with a few fellow people, he wrote an efficient implementation of some elliptic curve cryptography for OpenSSL.

Without forward secrecy To understand the problem when forward secrecy is absent, let s look at the classic TLS handshake when using a cipher suite like AES128-SHA. During this handshake, the server will present its certificate and both the client and the server will agree on a master secret. This secret is built from a 48byte premaster secret generated and encrypted by the client with the public key of the server. It is then sent in a Client Key Exchange message to the server during the third step of the TLS handshake. The master secret is derived from this premaster secret and random values sent in clear-text with Client Hello and Server Hello messages. This scheme is secure as long as only the server is able to decrypt the premaster secret (with its private key) sent by the client. Let s suppose that an attacker records all exchanges between the server and clients during a year. Two years later, the server is decommissioned and sent for recycling. The attacker is able to recover the hard drive with the private key. She can now decrypt any session she recorded: the encrypted premaster secret sent by a client is decrypted with the private key and the master secret is derived from it. The attacker can now recover passwords and other sensitive information that can still be valuable today. The main problem lies in the fact that the private key is used for two purposes: authentication of the server and encryption of a shared secret. Authentication only matters while the communication is established, but encryption is expected to last for years.

Diffie-Hellman with discrete logarithm One way to solve the problem is to keep using the private key for authentication but uses an independent mechanism to agree on a shared secret. Hopefully, there exists a well-known protocol for this: the Diffie-Hellman key exchange. It is a method of exchanging keys without any prior knowledge. Here is how it works in TLS:

1. The server needs to generate once (for example, with openssl dhparam command):
   • p , a large prime number,
   • g , a primitive root modulo p (for every integer a coprime to p , there exists an integer k such that g^k=a\mod p ).
2. The server picks a random integer a and compute g^a \mod p . After sending its regular Certificate message, it will also send a Server Key Exchange message (not included in the handshake depicted above) containing, unencrypted but signed with its private key for authentication purpose:
   • random value from the Client Hello message,
   • random value from the Server Hello message,
   • p , g ,
   • g^a\mod p=A .
3. The client checks that the signature is correct. It also picks a random integer b and sends g^b \mod p=B in a Client Key Exchange message. It will also compute A^b\mod p=g^ ab \mod p which is the premaster secret from which the master secret is derived.
4. The server will receive B and compute B^a\mod p=g^ ab \mod p which is the same premaster secret known by the client.

Again, the private key is only used for authentication purpose. An eavesdropper will only know p , g , g^a\mod p and g^b\mod p . Computing g^ ab \mod p from those values is the discrete logarithm problem for which there is no known efficient solution. Because the Diffie-Hellman exchange described above always uses new random values a and b , it is called Ephemeral Diffie-Hellman (EDH or DHE). Cipher suites like DHE-RSA-AES128-SHA use this protocol to achieve perfect forward secrecy¹. To achieve a good level of security, parameters of the same size as the key are usually used (the security provided by the discrete logarithm problem is about the same as the security provided by factorisation of two large prime numbers) and therefore, the exponentiation operations are pretty slow as we can see in the benchmark below:

Diffie-Hellman with elliptic curves Fortunately, there exists another way to achieve a Diffie-Hellman key exchange with the help of elliptic curve cryptography which is based on the algebraic structure of elliptic curves over finite fields. To get some background on this, be sure to check first Wikipedia article on elliptic curves. Elliptic curve cryptography allows one to achieve the same level of security than RSA with smaller keys. For example, a 224bit elliptic curve is believed to be as secure as a 2048bit RSA key.

Some theory Diffie-Hellman key exchange described above can easily be translated to elliptic curves. Instead of defining p and g , you get some elliptic curve, y^2=x^3+\alpha x+\beta , a prime p and a base point G . All those parameters are public. In fact, while they can be generated by the server, this is a difficult operation and they are usually chosen among a set of published ones. The use of elliptic curves is an extension of TLS described in RFC 4492. Unlike with the classic Diffie-Hellman key exchange, the client and the server need to agree on the various paremeters. Most of this agreement is done inside Client Hello and Server Hello messages. While it is possible to define some arbitrary parameters, web browsers will only support a handful of predefined curves, usually NIST P-256, P-384 and P-521. From here, the key exchange with elliptic curves is pretty similar to the classic Diffie-Hellman one:

1. The server picks a random integer a and compute aG which will be sent, unencrypted but signed with its private key for authentication purpose, in a Server Key Exchange message.
2. The client checks that the signature is correct. It also picks a random integer b and sends bG in a Client Key Exchange message. It will also compute b\cdot aG=abG which is the premaster secret from which the master secret is derived.
3. The server will receive bG and compute a\cdot bG=abG which is the same premaster secret known by the client.

An eavesdropper will only see aG and bG and won t be able to compute efficiently abG . Using ECDHE-RSA-AES128-SHA cipher suite (with P-256 for example) is already a huge speed improvement over DHE-RSA-AES128-SHA thanks to the reduced size of the various parameters involved. Web browsers only support a handful of well-defined elliptic curves, chosen to ease an efficient implementation. Bodo M ller, Emilia K sper and Adam Langley have provided 64bit optimized versions of NIST P-224, P-256 and P-521 for OpenSSL. To get even more details on the matter, you can read the end of the introduction on elliptic curves from Adam Langley, then a short paper from Emilia K sper which presents a 64bit optimized implementation of the NIST elliptic curve NIST P-224.

In practice First, keep in mind that elliptic curve cryptography is not supported by all browsers. Recent versions of Firefox and Chrome should handle NIST P-256, P-384 and P-521 but for most versions Internet Explorer, you are currently out of luck. Therefore, you need to keep accepting other cipher suites. You need a recent version of OpenSSL. Support for ECDHE cipher suites has been added in OpenSSL 1.0.0. Check with openssl ciphers ECDH that your version supports them. If you want to use the 64bit optimized version, you need to run a snapshot of OpenSSL 1.0.1, configured with enable-ec_nistp_64_gcc_128 option. A recent GCC is also required in this case. Next, you need to choose the appropriate cipher suites. If forward secrecy is an option for you, you can opt for ECDHE-RSA-AES128-SHA:AES128-SHA:RC4-SHA cipher suites which should be compatible with most browsers. If you really need forward secrecy, you may opt for ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:EDH-DSS-DES-CBC3-SHA instead. Then, you need to ensure the order of cipher suites is respected. On nginx, this is done with ssl_prefer_server_ciphers on. On Apache, this is SSLHonorCipherOrder on. UPDATED: You need to check ECDHE support for your web server. For nginx, the support has been added in 1.0.6 and 1.1.0. The curve selected defaults to NIST P-256. You can specify another one with ssl_ecdh_curve directive. For Apache, it has been added in 2.3.3 and does not exist in the current stable branch. Adding support for ECDHE is quite easy. You can check how I added it in stud. This issue also exists for DHE cipher suites, in which case you also might have to specify DH parameters to use (generated with openssl dhparam) using some special directive or by appending the parameters to the certificate. Check Immerda Techblog article for more background about this point. The implementation of TLS session tickets may be incompatible with forward secrecy, depending on how they are implemented. When they are protected by a random key generated at the start of the server, the same key could be used for months. Some implementations² may derive the key from the private key. In this case, forward secrecy is broken. If forward secrecy is a requirement for you, you need to either disable tickets or ensure that key rotation happens often. Check with openssl s_client -tls1 -cipher ECDH -connect 127.0.0.1:443 that everything works as expected.

Some benchmarks With the help of the micro-benchmark tool that I developed for my previous article, we can compare the efficiency of cipher suites providing forward secrecy: I have used a snapshot of OpenSSL 1.0.1 (2011/11/25). The optimized version of ECDHE is the one you get by using enable-ec_nistp_64_gcc_128 option when configuring OpenSSL. Let s focus on the server part. Enabling DHE-RSA-AES128-SHA cipher suite hinders the performance of TLS handshakes by a factor of 3. Using ECDHE-RSA-AES128-SHA instead only adds an overhead of 27%. However, if we use the 64bit optimized version, the cost is only 15%. The overhead is only per full TLS handshake. If 3 out of 4 of your handshakes are resumed, you need to adjust the numbers. Your mileage may vary but the computational cost for enabling perfect forward secrecy with an ECDHE cipher suite seems a small sacrifice for better security.

---

1. Perfect forward secrecy is an enhanced version of forward secrecy. It assumes each exchanged key are independent and therefore a compromised key cannot be used to compromise another one.
2. For example, this is the case of the implementation I have proposed for stud to enable sharing of tickets between multiple hosts.

This is both a release announcement for the next installment of The MirBSD Korn Shell, mksh R40b, and a follow-up to Sune s article about small tools of various degrees of usefulness. I hope I don t need to say too much about the first part; mksh(1) is packaged in a gazillion of operating environments (dear Planet readers, that of course includes Debian, which occasionally gets a development snapshot; I ll wait uploading R40c until that two month fixed gcc bug will finally find its way into the packages for armel and armhf. Ah, we re getting Arch Linux (after years) to include mksh now. (Probably because they couldn t stand the teasing that Arch Hurd included it one day after having been told about its existence, wondering why it built without needing patches on Hurd ) MSYS is a supposedly supported target now, people are working on WinAPI and DJGPP in their spare time, and Cygwin and Debian packagers have deprecated pdksh in favour of mksh (thanks!). So, everything looking well on that front. I ve started a collection of shell snippets some time ago, where most of those small things of mine ends up. Even stuff I write at work we re an Open Source company and can generally publish under (currently) AGPLv3 or (if extending existing code) that code s licence. I chose git as SCM in that FusionForge instance so that people would hopefully use it and contribute to it without fear, as it s hosted on my current money source s servers. (Can just clone it.) Feel free to register and ask for membership, to extend it (only if your shell-fu is up to the task, KNOPPIX-style scripts would be a bad style(9) example as the primary goal of the project is to give good examples to people who learn shell coding by looking at other peoples code). Maybe you like my editor, too? At OpenRheinRuhr, the Atari people sure liked it as it uses WordStar like key combinations, standardised across a lot of platforms and vendors (DR DOS Editor, Turbo Pascal, Borland C++ for Windows, ) ObPromise: a posting to raise the level of ferrophility on the Planet aggregators this wlog reaches (got pix)

It s been one year since the first People behind Debian interview. For this special occasion, I wanted a special guest and I m happy that our Debian Project Leader (DPL) Stefano Zacchiroli accepted my invitation. He has a difficult role in the community, but he s doing a really great job of it. He s a great mediator in difficult situations, but he s also opinionated and can push a discussion towards a conclusion. Read on to learn how he became a Debian developer and later DPL, what he s excited about in the next Debian release, and much more. Raphael: Who are you? Stefano: I m Stefano Zacchiroli, but I prefer to be called Zack, both on the Internet and in real life. I m 32, Italian, emigrated to France about 4 years ago. I live in Paris, and I find it to be one of the most gorgeous and exciting cities in the world. As my day job I m a Computer Science researcher and teacher at University Paris Diderot and IRILL. In my copious free time I contribute to Debian, and I m firmly convinced that doing so is an effective way to help the cause of Free Software. Besides, I find it to be a lot of fun! Raphael: How did you start contributing to Debian? Stefano: Flash back to 1999, when I was a 2nd year student in Computer Science at the University of Bologna. Back then in Italy it was uncommon for young geeks to get exposed to Free Software: Internet was way less pervasive than today and most computer magazines didn t pay much attention to GNU/Linux. Luckily for me, the professor in charge of the student lab was a Free Software enthusiast and all students machines there were running Debian. Not only that, but there was also a student program that allowed volunteers to become sysadmins after having shown their skills and convinced the director they were trustworthy. Becoming one of those volunteer Debian admins quickly became one of my top objectives for the year, and that is were I ve learned using Debian. The year after that, I got in touch with a research group that was to become the happy bunch of hackers with whom I would have done both my master and PhD theses. They were designing a new proof assistant. Most of the development was in OCaml and happened on Debian. OCaml was available in Debian, but many of the libraries we needed were not. So I approached the Debian OCaml Team offering to help. Before I realize what was going on I was (co-)maintainer of tens of OCaml-related packages. At some point I got told I think you should apply as a Debian Developer . So I did and in a couple of months I went through the New Member (NM) process, that was back then in its infancy. I still remember my happiness while reading the account created mail, the day after my 22nd birthday. I know the NM process went through some bad publicity in the past, but I m happy to see that nowadays the process can be as swift as it has been for me 10 years ago. Raphael: It s your second year as Debian Project Leader (DPL). Are you feeling more productive in the role? Do you fear to burn out? Stefano: I m feeling way more productive, no doubts. The task of the Debian Project Leader is not necessarily difficult, but it is a complex and scarcely documented one. It is also profoundly different from any other task that Debian people usually work on, so that experience doesn t help much in getting started. Before becoming effective as DPL one needs to get to know many people and mechanisms he is not familiar with. More importantly, one needs to set up a personal work-flow that allows to keep up with day-to-day DPL tasks (which are aplenty) as well as with urgencies (that tend to pop-up in the leader@debian.org INBOX at the least convenient time). Finally, one also needs to do proper traffic shaping and always retain enough motivation bandwidth to keep the Project informed about what is going on in DPL-land. Finding the right balance among all these ingredients can take some time. Once one is past it, everything goes way more smoothly. The above is why I m constantly encouraging people interested in running for DPL in the future to reach out to me and work on some tasks of the current DPL s TODO list. I swear it is not just a cheap attempt at slavery!. It is rather an attempt at DPL mentoring that could be beneficial: both to give future candidates more awareness of the task, and to reduce the potential downtime when handing over from one DPL to the next. Regarding burn out, I don t feel prone to its risk these days. If I look back, I can say that my contributions as DPL have been pretty constant in volume over time; my enthusiasm for the task, if anything, is on the rise. The effectiveness of my contributions as DPL are, on the other hand, not mine to judge. Raphael: If you had to single out two achievements where you were involved as DPL, what would they be? Stefano: I d go for the following two, in no particular order:

• Dialogue with derivatives. When I became DPL ~1.5 years ago the situation on that front was pretty dire. In the specific case of Ubuntu, by far the most successful and customized of all Debian derivatives, I remember being scared of raising the topic of collaboration with them on mailing lists. More generally, we had no specific initiatives to foster technical collaboration with and among derivatives. A huge potential of (forwarded) contributions to Debian was being wasted. Today things look much better, as I ve documented in recent talks at DebConf11 and UDS-P. The amount of forwarded patches we receive from downstream is at its maximum and many people who apply to become Debian Developers come from derivatives. Conflict situations still exist, for good reasons that we still have to either fix or figure out entirely. But I m positive we re on the right track.

  The amount of forwarded patches we receive from downstream is at its maximum

  This is by far not an achievement of mine alone. In particular, many of the activity of the Derivatives Front Desk have been organized by other enthusiastic volunteers. But I ve done my part, especially in breaking the ice and in proposing a vision of Free Software distribution where all distros play a role and are welcome to join the game, as long as they give back and give credit to their respective upstreams.
• Process membership. I m proud of having promoted the general resolution (GR) that has clarified (and advertised to the world) that Debian welcomes all kind of contributions, and that they all equally matter to become proper members of the Project. I notice only now while writing this that the GR title, traditionally chosen by the secretary, was Debian Project Members . That choice harmonically closes the circle with the recent renaming of the NM process to New Member process. Today, we have several project members (AKA Debian Developers ) that are active citizen of Debian with voting rights, even though they take care of tasks other than packaging. Anyone can become a Debian citizen, as long as they are ready to abide by Debian s values, have a track record of verifiable contributions to Debian, and are committed to keep them coming in the future.

  We have several project members that [ ] take care of tasks other than packaging.

  Once gain, this is by far not an achievement of mine alone, very little project-wide achievements are. DAM has helped a lot and support from the project as a whole has been immense.

OK, let me cheat and add a third one I m also proud of having been able to report to the Project my whereabouts as DPL, thoroughly and periodically, since the very beginning is first term. People annoyed by my reporting logorrhea now have all my sympathies. Raphael: Among the possible new features of Debian Wheezy, which one gets you excited most? Stefano: It s multi-arch, no doubt. Even though it is not a directly user visible change, it s a very far reaching one. It is also one of those changes that make me feel that moment of truth of coders, when you realize you are finally doing the right thing and ditching piles of ugly hacks.

It s multi-arch [ ] you realize you are finally doing the right thing and ditching piles of ugly hacks.

Raphael: If you were not DPL and could spend all your time on Debian, what project would you do? Stefano: I would sit down and do software development for Debian. It s impressive how many important and beneficial changes for Debian could be delivered by specific software improvements in various parts of our infrastructure. We tend to attract many packagers, but not so many people willing to maintain Debian infrastructure softwares like dak, britney, debbugs, the PTS, etc. Their maintenance burden then falls on the shoulders of the respective teams which are generally very busy with other important tasks. As a project, we seem to be more appealing to packagers than to software developers. That is a pity given the amount of exciting coding tasks that are everywhere in Debian. Part of the reason we are not appealing to developers is that we are not particularly good at collecting coding tasks in a place where interested developers could easily pick them up. It also takes quite a bit of inside knowledge to spot infrastructure bugs and understand how to fix them. I long for some spare hacking time to check if I m still good enough of a coder to hunt down longstanding bugs in our infrastructure, which have ended up being my pet peeves. I d also love to dive again into RCBW. It s less committing than package maintenance, more diverse and challenging, and also an immensely useful activity to get Debian releases done. Raphael: Martin Michlmayr is worried that there is so few paid opportunities around Debian. Do you agree with his sentiment, and if yes do you have ideas on how to improve this situation? Stefano: The idealistic me wishes Debian to be a community made only of volunteers that devote their free time to the Project. Oh, and that me also wishes Debian to be competitive with similar projects, no matter how many full-time employees others have! That is coherent with a view of society where everyone has a day job, but also engages in volunteering activities ensuring that public interest is pursued by people motivated by interests other than profit. But I do realize that for Free Software to succeed companies, employees, and salaries should all have a role. I admire projects that strike a good balance between volunteer and paid work. The Linux kernel is emblematic in that respect: many developers are paid by companies that have a commercial or strategic interest in Linux. Nevertheless volunteers contributions are aplenty and the Linux community gives a convincing impression that choices are driven by the community itself (or by its benevolent dictator) without money-driven impositions.

I do realize that for Free Software to succeed companies, employees, and salaries should all have a role.

Such an ecosystem does not exist around Debian. We do have a partner program that allows for it to happen, but we have very few partners with an interest in doing distribution development work. Like Martin, I m worried by this state of affairs, because it de facto means we lag behind in terms of available people power. In a community of volunteers, that might frustrate people and that is not good. To improve over the status quo the first step is to federate together small and medium companies that have a strategic interest in Debian and listen to their needs. I m already in touch with representatives of such companies that, in many cases, already employ Debian Developers to do some distribution work in Debian. We will be soon sending out a call to reach out to more such companies, but since we are discussing this, why waiting? If some of our readers here are representative of such companies, I encourage them to get in touch with me about this. Raphael: You know that the fundraising campaign for the Debian Administrator s Handbook is on good track but the liberation of the book is not yet assured. What do you think of this project? Stefano: I m happy about the project, to the point that I ve accepted writing a testimonial for it . I m sad about the scarce availability of up to date and high quality (DFSG-)Free books about Debian and I welcome any initiative that might help closing that gap.

I m sad about the scarce availability of up to date and high quality (DFSG-)Free books about Debian.

Free Culture is a great offspring of Free Software and I m convinced we need to stand up against double standards in the two camps. Letting aside software-specific licensing details, the basic freedoms to be defended are the same. They are those freedoms that ensure that a reader is in full control of his book, pretty much as they ensure that a computer user is in full control of the software that runs on it. I m therefore proud that Debian has long resolved that the Debian Free Software Guidelines (DFSG) apply not only to software but also to books and other pieces of documentation. But the status quo implies that not only we have very few up to date, high quality books about Debian. It also implies that, at present, we have no such book that we can distribute in the Debian archive, showing off the Free Software (and Free Culture!) values we stand for.
Crowdfunding is considered to be a good mate for Free Culture, where the services model that applies to Free Software is more difficult to exploit. I so wish any luck to yours and Roland s initiative. A different matter is whether Debian, as a project, should endorse the initiative and actively campaign for it. As you know, I think it should not. While we do advertise general project donations, we don t do mission-specific fundraising campaign for Debian itself. Coherently with that, I don t think we should relay crowdfunding campaigns for 3rd parties, even when the result would be beneficial to Debian. Raphael: Is there someone in Debian that you admire for their contributions? Stefano: There are two classes of people that I particularly admire in Debian:

• Those with an uncanny ability to guide discussions towards constructive conclusions. We are lucky to have many in Debian and I admire all of them. Having to single out one I d name Russ Allbery, in honor of whom I hereby propose the periodic Russ Allbery s Distinguished Flametamer Award .
• People stepping up for responsibility roles, especially when the responsibility put them in tough spots. Release teams, ftp-masters, DSA, DAM, as well as past and present members of teams with similarly hot seats have all my admiration.

---

Thank you to Zack for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

7 comments Liked this article? Click here. My blog is Flattr-enabled.

I probably don t have to present Mark Shuttleworth he was already a Debian developer when he became millionaire after having sold Thawte to Verisign in 1999. Then in 2002 he became the first African (and first Debian developer) in space. 2 years later, he found another grandiose project to pursue: bring the Microsoft monopoly to an end with a new alternative operating system named Ubuntu (see bug #1). I have met Mark during Debconf 6 in Oaxtepec (Mexico), we were both trying to find ways to enhance the collaboration between Debian and Ubuntu. The least I can say is that Mark is opinionated but any leader usually is, and in particular the self-appointed ones! Read on to discover his view on the Ubuntu-Debian relationship and much more. Raphael: Who are you? Mark: At heart I m an explorer, inventor and strategist. Change in technology, society and business is what fascinates me, and I devote almost all of my time and wealth to the catalysis of change in a direction that I hope improves society and the environment. I m 38, studied information systems and finance at the University of Cape Town. My hearts home is Cape Town, and I ve lived there and in Star City and in London, now I live in the Isle of Man with my girlfriend Claire and 14 precocious ducks. I joined Debian in around 1995 because I was helping to setup web servers for as many groups as possible, and I thought Debian s approach to packaging was very sensible but there was no package for Apache. In those days, the NM process was a little easier Raphael: What was your initial motivation when you decided to create Ubuntu 7 years ago? Mark: Ubuntu is designed to fulfill a dream of change; a belief that the potential of free software was to have a profound impact on the economics of software as well as its technology. It s obvious that the technology world is enormously influenced by Linux, GNU and the free software ecosystem, but the economics of software are still essentially unchanged. Before Ubuntu, we have a two-tier world of Linux: there s the community world (Debian, Fedora, Arch, Gentoo) where you support yourself, and the restricted, commercial world of RHEL and SLES/SLED. While the community distributions are wonderful in many regards, they don t and can t meet the needs of the whole of society; one can t find them pre-installed, one can t get certified and build a career around them, one can t expect a school to deploy at scale a platform which is not blessed by a wide range of institutions. And the community distributions cannot create the institutions that would fix that. Ubuntu brings those two worlds together, into one whole, with a commercial-grade release (inheriting the goodness of Debian) that is freely available but also backed by an institution. The key to that dream is economics, and as always, a change in economics; it was clear to me that the flow of money around personal software would change from licensing ( buying Windows ) to services ( paying for your Ubuntu ONE storage ). If that change was coming, then there might be room for a truly free, free software distribution, with an institution that could make all the commitments needed to match the commercial Linux world. And that would be the achievement of a lifetime. So I decided to dedicate a chunk of my lifetime to the attempt, and found a number of wonderful people who shared that vision to help with the attempt. It made sense to me to include Debian in that vision; I knew it well as both a user and insider, and believed that it would always be the most rigorous of the community distributions. I share Debian s values and those values are compatible with those we set for Ubuntu.

Debian would always be the most rigorous of the community distributions.

Debian on its own, as an institution, could not be a partner for industry or enterprise. The bits are brilliant, but the design of an institution for independence implies making it difficult to be decisive counterparty, or contractual provider. It would be essentially impossible to achieve the goals of pre-installation, certification and support for third-party hardware and software inside an institution that is designed for neutrality, impartiality and independence. However, two complementary institutions could cover both sides of this coin. So Ubuntu is the second half of a complete Debian-Ubuntu ecosystem. Debian s strengths complement Ubuntu s, Ubuntu can achieve things that Debian cannot (not because its members are not capable, but because the institution has chosen other priorities) and conversely, Debian delivers things which Ubuntu cannot, not because its members are not capable, but because it chooses other priorities as an institution. Many people are starting to understand this: Ubuntu is Debian s arrow, Debian is Ubuntu s bow. Neither instrument is particularly useful on its own, except in a museum of anthropology

Ubuntu is Debian s arrow, Debian is Ubuntu s bow.

So the worst and most frustrating attitude comes from those who think Debian and Ubuntu compete. If you care about Debian, and want it to compete on every level with Ubuntu, you are going to be rather miserable; you will want Debian to lose some of its best qualities and change some of its most important practices. However, if you see the Ubuntu-Debian ecosystem as a coherent whole, you will celebrate the strengths and accomplishments of both, and more importantly, work to make Debian a better Debian and Ubuntu a better Ubuntu, as opposed to wishing Ubuntu was more like Debian and vice versa. Raphael: The Ubuntu-Debian relationship was rather hectic at the start, it took several years to mature . If you had to start over, would you do some things differently? Mark: Yes, there are lessons learned, but none of them are fundamental. Some of the tension was based on human factors that cannot really be altered: some of the harshest DD critics of Canonical and Ubuntu are folk who applied for but were not selected for positions at Canonical. I can t change that, and wouldn t change that, and would understand the consequences are, emotionally, what they are. Nevertheless, it would have been good to be wiser about the way people would react to some approaches. We famously went to DebConf 5 in Porto Allegre and hacked in a room at the conference. It had an open door, and many people popped a head in, but I think the not-a-cabal collection of people in there was intimidating and the story became one of exclusion. If we d wanted to be exclusive, we would have gone somewhere else! So I would have worked harder to make that clear at the time if I d known how many times that story would be used to paint Canonical in a bad light. As for engagement with Debian, I think the situation is one of highs and lows. As a high, it is generally possible to collaborate with any given maintainer in Debian on a problem in which there is mutual interest. There are exceptions, but those exceptions are as problematic within Debian as between Debian and outsiders. As a low, it is impossible to collaborate with Debian as an institution, because of the design of the institution.

It is generally possible to collaborate with any given maintainer [ ] [but] it is impossible to collaborate with Debian as an institution.

In order to collaborate, two parties must make and keep commitments. So while one Debian developer and one Ubuntu developer can make personal commitments to each other, Debian cannot make commitments to Ubuntu, because there is no person or body that can make such commitments on behalf of the institution, on any sort of agile basis. A GR is not agile . I don t say this as a critique of Debian; remember, I think Debian has made some very important choices, one of those is the complete independence of its developers, which means they are under no obligation to follow a decision made by anyone else. It s also important to understand the difference between collaboration and teamwork. When two people have exactly the same goal and produce the same output, that s just teamwork. When two people have different goals and produce different product, but still find ways to improve one anothers product, that s collaboration. So in order to have great collaboration between Ubuntu and Debian, we need to start with mutual recognition of the value and importance of the differences in our approach. When someone criticises Ubuntu because it exists, or because it does not do things the same way as Debian, or because it does not structure every process with the primary goal of improving Debian, it s sad. The differences between us are valuable: Ubuntu can take Debian places Debian cannot go, and Debian s debianness brings a whole raft of goodness for Ubuntu. Raphael: What s the biggest problem of Debian? Mark: Internal tension about the vision and goals of Debian make it difficult to create a harmonious environment, which is compounded by an unwillingness to censure destructive behaviour. Does Debian measure its success by the number of installs? The number of maintainers? The number of flamewars? The number of packages? The number of messages to mailing lists? The quality of Debian Policy? The quality of packages? The freshness of packages? The length and quality of maintenance of releases? The frequency or infrequency of releases? The breadth of derivatives? Many of these metrics are in direct tension with one another; as a consequence, the fact that different DD s prioritise all of these (and other goals) differently makes for interesting debate. The sort of debate that goes on and on because there is no way to choose between the goals when everyone has different ones. You know the sort of debate I mean Raphael: Do you think that the Debian community improved in the last 7 years? If yes, do you think that the coopetition with Ubuntu partly explains it? Mark: Yes, I think some of the areas that concern me have improved. Much of this is to do with time giving people the opportunity to consider a thought from different perspectives, perhaps with the benefit of maturity. Time also allows ideas to flow and and of course introduces new people into the mix. There are plenty of DD s now who became DD s after Ubuntu existed, so it s not as if this new supernova has suddenly gone off in their galactic neighbourhood. And many of them became DD s because of Ubuntu. So at least from the perspective of the Ubuntu-Debian relationship, things are much healthier. We could do much better. Now that we are on track for four consecutive Ubuntu LTS releases, on a two-year cadence, it s clear we could collaborate beautifully if we shared a freeze date. Canonical offered to help with Squeeze on that basis, but institutional commitment phobia reared its head and scotched it. And with the proposal to put Debian s first planned freeze exactly in the middle of Ubuntu s LTS cycle, our alignment in interests will be at a minimum, not a maximum. Pure <facepalm />. Raphael: What would you suggest to people (like me) who do not feel like joining Canonical and would like to be paid to work on improving Debian? Mark: We share the problem; I would like to be paid to work on improving Ubuntu, but that s also a long term dream Raphael: What about using the earnings of the dormant Ubuntu Foundation to fund some Debian projects? Mark: The Foundation is there in the event of Canonical s failure to ensure that commitments, like LTS maintenance, are met. It will hopefully be dormant for good Raphael: The crowdfunding campaign for the Debian Administrator s Handbook is still going on and I briefly envisioned the possibility to create the Ubuntu Administrator s Handbook. What do you think of this project? Mark: Crowdfunding is a great match for free software and open content, so I hope this works out very well for you. I also think you d find a bigger market for an Ubuntu book, not because Ubuntu is any more important than Debian but because it is likely to appeal to people who are more inclined to buy or download a book than to dive into the source. Again, this is about understanding the difference in audiences, not judging the projects or the products. Raphael: Is there someone in Debian that you admire for their contributions? Mark: Zack is the best DPL since 1995; it s an impossible job which he handles with grace and distinction. I hope praise from me doesn t tarnish his reputation in the project!

---

Thank you to Mark for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook .

32 comments Liked this article? Click here. My blog is Flattr-enabled.

After recently reading a number of SSL/TLS-related articles, I decided to experiment and look for the ideal OpenSSL configuration for Apache (using mod_ssl since I haven't tried mod_gnutls yet) and nginx.

By "ideal" I mean that this configuration needs to be compatible with most user agents likely to interact with my website as well as being fast and secure.

Here is what I came up with for Apache:

SSLProtocol TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!kEDH

and for nginx:

ssl_protocols  TLSv1;
ssl_ciphers  RC4-SHA:HIGH:!kEDH;
ssl_prefer_server_ciphers   on;

Cipher and protocol selectionIn terms of choosing a cipher to use, this configuration does three things:

• disables all weak ciphers and protocols
• disables very slow ciphers that use ephemeral Diffie-Hellman exchanges
• gives priority to the RC4 cipher to minimize CPU usage and defend against the BEAST attack

Testing toolsThe main tool I used while testing various configurations was the SSL labs online tool. The CipherFox extension for Firefox was also quite useful to quickly identify the selected cipher.

Of course, you'll want to make sure that your configuration works in common browsers, but you should also test with tools like wget, curl and httping. Many of the online monitoring services are based on these.

Other considerationsTo increase the performance and security of your connections, you should ensure that the following features are enabled:

• SSL session caching with a session store shared between all of your web servers
• HSTS headers to let browsers know that they should always visit your site over HTTPS

Two days ago, I finally brought my system up-to-date with regard to the GNOME packages that I use. I was very reluctant because I am using a heavily customized gnome-panel and I waited until all the applets that I use have been ported to gtk3 and the libgnome-applet library. I am the Debian maintainer of one of these applets, link-notification-daemon, and I have made the changes and tested them in a chroot before, but now that I am actually using it, I found and fixed more visual glitches. I think by now it is as usable as it was before on GNOME 2. I am still looking for someone to step up as the new upstream maintainer for it! I had some trouble with gnome-session and notification-daemon, thought I found the cause and reported it, but a day later I could not reproduce it anymore. Weird. Other problems, such as the busy cursor not disappearing after the session start, or evolution going into a freeze also have stopped occurring by themselves.
Other problems I found and fixed, for example the battery status applet was not visible until a major state change, or panel applets were not loaded due to a race condition, or tooltips not working (which called for a fix in gtk+ itself). I also added to the xmonad package the files required to allow the user to select GNOME with Xmonad in GDM, and found out how to have Galeon appear in the list of possible default browsers.
I don t really have to say much about GNOME3 itself. I disliked the default font, Cantarell, mostly because of the extra spacing that reduced the number of folders and mail I can see in evolution. (Also see what nekohayo has to say about that.) I also was under the impression that the font is much lighter, and I started to adjust the position of my laptop monitor to increase the contrast. So I switch back to DejaVu Sans.
I also disabled the sleep-on-lid-close feature I often run backups, or download stuff, or compile stuff, or calculate stuff, so I want better control over this. So, I guess, now that everything is almost as it was in GNOME2, I can say I like it...

Flattr this post

Since there have been a few blog posts about GNOME 3 already, I m sure you don t mind me adding one. I m one of those who don t do any work, just lurk around and wait for stuff to be fixed or working in the first place. I also don t like compiling GNOME myself. I barely touched the already packaged stuff in the repository of our Debian GNOME packaging team. But I run unstable plus stuff from experimental. I ll thus have a bit of a different experience with GNOME 3 than current testing users who at least in parts don t seem to like what they re getting. That being said I love it. No, I don t mind running network manager on my systems, I even have it on my netbook (Atom processor). I used wicd for a long time because network manager really got nothing done back in the days (half a year ago or what :)). Nowadays, since version 0.9, it works fine for me. No fiddling around with config files, perfect integration in the desktop, and LAN and WLAN Just Work [tm]. No, I don t like if some weird dependencies dictate what tools to use. I usually used pidgin for messaging, for instance. Now I m using empathy, simply because it s best integrated in GNOME. And no, I won t switch from mutt to evolution. :-P And there are more such cases. Also, these weird accessibility tools that are stuck in the gnome-shell top panel , I don t need them. Still there. (There s an extension to remove it, by the way.) Also, I must say, GNOME 3.2 is a lot better than 3.0 already. And I think 3.4 might do the final trick for me. Fact is, with GNOME 3 I have a much cleaner desktop, there is more room for windows (which is nice on a netbook display), there are web apps which I already use a lot, and it has a professional, not too playful look (i.e. a few effects, clear and consistent theming, no bubbling windows). And the new notification stuff is just awesome, especially with the empathy integration. Oh, and too many mouse click to access an application? 1) Put your most used apps in the favorites bar (or whatever it is called); it s just one click then. 2) Press your meta-key (that way our name for the windows key, or was that the ESC key? Whatever, press your windows key), type in the first two or three letters of your application, and hit enter; you don t need a mouse at all. There. It had to be said. :)

There are various presentations that state the goodness of PAV on Linux. Most revolve around using multipath-tools to assemble a volume if you don't have HyperPAV. But it turns out that the DASD device driver does multipathing for them internally in current kernels (which includes the squeeze kernel).

So all you need to do is setting those alias devices online. When you do that the kernel will log that it detected a new device, but you'll find that it won't create any dasd* device nodes for them, nor will it list partitions. lsdasd will only show you "alias" without mentioning the base volume, but you can fetch that information easily from the uid sysfs entry.

I think that it will come as no surprise to my fellow software engineers if I note that I almost never write new software any more. I maintain, I augment, I refactor, I debug but very, very rarely do I start something new.

Finally! Last Friday, after two years worth of work, I finally got the first box of books for the Construcci n Colaborativa del Conocimiento (Collaborative Knowledge Construction) project I worked on as a coordinator together with Alejandro Miranda (pooka), and together with a large group of 11 authors: Translating over from the back cover text (and this is just a quick translation from me It reads better in Spanish ;-) ):

What defines us as humans is our ability, on one side, to
create knowledge, and on the other, to share or communicate it with our neighbors. Both features have worked together over tens of thousands of years, and, working together, have led the knowledge to transcend the individual, avoiding the need to rediscovery or reinvention of is already known. Sharing knowledge is what has taken our species to the dominant role it occupies today. But knowledge creation and sharing has seen a deep transformation in recent decades, thanks to the quick evolution of telecommunications, specially the massification of Internet and cellular telephony. We are transiting towards the so desired and at the same time so feared knowledge society. In this book, eleven authors from very different disciplinary backgrounds and geographic origins ellaborate on how a hyper-connected world has modified the basic rules of interaction in areas as diverse as artistic creation, social organizations, computer code development, education or the productive sector. This book is the result of a year worth of work for in the "Collaborative Construction of Knowledge" seminar, during which we
used the same new forms of knowledge production we have studied. The videos of the sessions, electronic participations and the full contents of this book are available under a permisive license at
http://seminario.edusol.info/seco3/

We will soon have the book ready in IIEc's e-store (which is mostly meant for national requests). I am also uploading the book to the lulu.com self-publishing service, and we are working on a epub-like edition. Right now it is still not available, but it should be there in some days. I will keep you posted. Meanwhile, the full contents can be read online at http://seminario.edusol.info/seco3

Even though some three months have passed since DebConf11 has successfully ended, I still wanted to give you just a glimpse on how some parts were played through the eyes of a lunatic (read: organizer). Of course, blog post can t come close near of explaining anything but at least it should give some insight and hopefully some pointers to the future DebConf organizers. If you don t feel like reading this much text you can listen to last episode of This Week In Debian podcast and after it just head to Beginning of the end part. DebConf11 First of all I d like to apologize if I offended or hurt anyone in this whole process as it wasn t my intention and during DebConf organization this is nothing irregular, for you to get hurt or you hurting somebody else. One thing that got stuck with me this whole time is when Mart n Ferrari approached me after I did bid proposal of Bosnia/Herzegovina for DebConf11 back on DebConf9 and told me you have no idea what you just got yourself into I played cool and said something along the line of coure I do and boy did I lie. During DebConf organization you re bound to make some of your decisions instantaneously in which you ll lose something, the most you can do is assess what s the thing you can cut the cord on, even though if that thing or a person for that matter might have meant something to you. You ll lose things, relationships, friendships, contacts at the times sense of humor and maybe even common logic. Some of it comes back to you, some the very next day and for some it may take longer. Some of it was lost irreversibly, but even in that case it wasn t lost and could even be traced to some other cause. One thing that I m thankful for is that I didn t lose, not lose but even get distanced from those close to me, and that I didn t lose guy who was with me in this from very first day, my right hand, my partner in crime Velimir ( aroundthfur ). It s not even losing I m talking about here, because even by losing you re gaining something and this is part of your everyday life, it s just that your life frame kinda gets stuck on fast forward button. So once the ride s over you may feel bit dizzy asking yourself what just happened. During this time I was also contacted by Google recruiter regrading software engineer internship but to make a long story short I didn t make it due to my heavy involvement and my lack of time (read: fail). But I don t find this as losing either, first time I was contacted by them, next time they ll be contacted by me maybe even for different position this time (project manager? . Even this DebConf wasn t our first bid we lost DebConf7 bid to Edinburgh, but it is perseverance that got us here. The moral I m trying to tell here is if you have a goal never back down from it. What I found to be our biggest problem and what I personally found as most painstaking in all of this was having and reporting to so many different sides and different groups while trying to keep them in same communication channel. Even my right hand (Velimir) was apart from me (Vienna), then place where two of us are coming from is not the place where DebConf is getting organized in. As in the end we ended up having all the parties separated and pulling to their own side, from Government of Republika Srpska to NGO to Debian to local team and to those who remained as undeclared . Option of cutting the cord in this case was missing as these were the components that DebCon11 was made of. Of course this meant you ll be missing workforce and there were days you d wake up driving >= 200 km/h (literally) with one hand on the wheel, other on the phone and your other right hand (Velimir) holding your RedBull. But you wouldn t do this just because you had to and there was no one else to do it, you d do it because you re responsible for the sake of the project and won t be just sitting there because by some unwritten hierarchy rules you re supposed to sit and supervise things. I d advise you do the same thing when you see there s no one else to do something, even if it means making coffee for one of the teams. I m not trying to scare you away from organizing your own DebConf, on the contrary after all I ve been through in past two years if someone asked me would I do it if I knew it was like this, without a doubt my answer would still be fsck yea . All the previous DebConf organizers told me about all of this, but why didn t I give up? For the same reason you won t give up and for the same reason why we believe in the same goal. If anything this is one of the stories you ll want to tell your kids about so When I was your age I used to sleep naked in the snow would turn to When I was your age I organized DebConf11 in Maybe this blog post wasn t even necessary and some things weren t supposed to be said, I guess sometimes I just can t shut up For those who were on DebConf11 I hope you enjoyed and had a great time, otherwise I m saying all of this in vein. As organizer all you see is problems and you miss most of the confrence, but by what I know there are 11 candidate bids for DebConf13 and for me hearing that tells me the results of our efforts. DebConf11 after effects I could write a whole novel about this, but to keep it as short as possible, for last two years as a side project I was working on an idea of Government or some of its institutions migrating to Linux. At first I was somewhat loud about it, then after Microsoft heard about it and after they tried stopping the idea by trying to scare me by trying to interfere with my private life; as that didn t work its lobbyist came even near of obstructing the whole conference within the Government. For the sake of the conference, I convinced the Government that by supporting DebConf it doesn t mean they need to move to Linux and publicly stopped talking about it. I also convinced them that our only goal was to have successful conference and promote alternative options and open ideas. I wasn t lying as I saw this as new opportunity of them concluding on their own why they should or shouldn t not move, the better conference was the more chances of success we had. That s why I tried pushing as many representatives from various companies as in this case we would use reverse psychology where basically no one or few know what Linux or Debian for that matter is, but everybody knows who Google is, so if you have participants from i.e: Google or Austrian E-Health care system talking about how they are using your technology is better way to explain what s it all about really. Eventually Microsoft even had their first ever conference in Bosnia/Herzegovina and you wanna take a wild guess where it was held? In the end we had a great conference, after the conference we were the ones that were approached by some big local companies interested in future co-operation and in the end a meeting with Mark Shuttleworth and President along with the core of Government was scheduled. Topic? Migration to Linux. For me personally this meeting went better then I could possibly even picture it, many topics were discussed and basically it was up to us/me to make a draft of the project plan and submit request for proposal to the Government. There was still some lobbying but it seemed as it all disappeared, runway was clear and open for the lift off. Necessary contacts with potential future partners were made and before any meetings were scheduled I announced I ll be taking a short break. I missed this years vacation, for matter of a fact I missed going to a vacation for last two years. I decided going to Dubrovnik for few days, just to recollect my thoughts, recompile my kernel and switch to lower gear as I was stuck in 6th gear for way too long. Beginning of the end Being on this vacation , even if it was only for few days this gave me an opportunity to look at it all from a different prospective, different angle and that s when it came to me. I realized that even before I was finished with one project without even realizing I was already involved with another one, one that would take much more time and one that might leave me wondering ~5 years later with question what if? After all if get involved with something I m not going to back out, it s either failure is not an option or I won t start doing it at all. Even if it brought me everything I wanted, how would you know if that s what you wanted if you haven t thought about it first? Having DebConf for the first time in this region already showed some (positive) results, but for seeing the actual results it may take a bit longer. We sowed the seeds now the best thing we can do is leave them as it is and see if anything grows out. Of course I could lean over them and protect them from any natural or any other type of hazards, this would also mean when I m not around it wouldn t know how to protect itself on its own. Besides all, interfering with natures way of doing things is rarely a good option. I consider myself to still be relatively young and threatening my well being over safe keeping some seeds and watching over if they grow into something or not doesn t really seem that attractive to me. Maybe one day I ll decide to leave the land, maybe I won t even get to see if anything grows out after all. Maybe some day someone will find this fruit, recognize its worth and claim it as it s own, maybe over time everybody forgets everything and even that this land once was a fertile ground. Maybe, maybe not. I m still a student, I was supposed to graduate last year but I took less classes last year so I wouldn t snap under all the pressure. That s why this year I have less classes and am supposed to graduate in May, that s less then a year until then and I just want to devote some time to myself, figure out what I want with me life, learn the things I want to learn, do the things I want to do. I want to turn back to working on Eclipse, I have some good plans for Eclipse and Android developers, lately I ve been slacking off way too much but I think I have a good excuse In the end, all I know is that I still want to change the world, whether it s doing that inside of Google or in my own company, or some third place; or I ll just be laughing at this statement only time will show. I m hoping that now with all of this out of me I can finally finish the DebConf11 chapter of my life.

It s been a while since I ve written about Moonshot. A lot has gone on; we ve been too busy doing to be busy blogging. However there s something that s happened recently that s so cool I had to take a moment to discuss it. Padl Software, the same people (well person) who brought us LDAP support to replace NIS and the first Active Directory clone, has now produced a GSS-EAP Security Service Provider. That s software that implements the Moonshot protocol and plugs it into the standard Windows security infrastructure. This is neat because it allows you to use GSS-EAP with unmodified Windows applications like Internet Explorer and Outlook/Exchange. Obviously, this will be great for Moonshot. However, I think the positive affects are more far-reaching than that. Luke has demonstrated that we can evolve the Windows security infrastructure without waiting for Microsoft to lead the way. For those of us working in the enterprise security space, that s huge. We can innovate and bring our innovation to Windows. In terms of getting acceptance in important user communities, getting funding for work, and making a practical difference, that s a big deal. This code is still in the early stages. Padl has not decided how the code will be made available. We don t know if it will be under an open-source license yet. Luke, naturally wants to get paid for his work. However if this code does get released under an open-source license, it will be very valuable. That will give all of us who are looking for a starting point for security innovations a starting point for bringing our innovations to Windows. Some in the open-source community will argue that we shouldn t work on improving Windows: if the open-source platforms have features Windows does not, then it may drive people to open-source. Especially for enterprise infrastructure, it tends not to work that way. You need broad cross-platform support to drive new technology. However, it does mean that we can take control of the evolution of our infrastructure; even for Windows there is no requirement that a single vendor controls what is possible.

A month ago, I published an article on the compared performances of stunnel, nginx and stud as SSL terminators. The conclusion was to use stud on a 64-bit system, with session caching and AES. stunnel was unable to scale properly and nginx exhibited important latency issues. I got constructive comments on many aspects. Therefore, here is the second round. The protagonists are the same but both the conditions and the conclusions have changed a bit.

Benchmarks The hardware platform remains unchanged:

• HP DL 380 G7
• 6 GB RAM
• 2 L5630 @ 2.13GHz (8 cores), hyperthreading disabled
• 2 Intel 82576 NIC
• Spirent Avalanche 2900 appliance

We focus on the following environment:

• Ubuntu Lucid, featuring a patched version of OpenSSL 0.9.8k with AES-NI builtin and enabled by default
• 2.6.39 kernel (250 HZ)
• 64-bit
• 1 core for HAProxy, 2 cores for network cards¹, 4 cores for SSL, 1 core for the system
• system limit of 100,000 file descriptors per process

On the SSL side, the main components are:

• 2048-bit certificates
• AES128-SHA cipher suite
• 8 HTTP/1.0 requests per client
• HTTP response payload of 1024 bytes

If you are interested in different conditions (32-bit, number of cores, 1024-bit certificates, proportion of session reuses), have a look at the previous article whose results on the matter are still valid. I only consider 2048-bit certificates because most certificates issued today are 2048-bit. In SP800-57, NIST advises that 1024-bit RSA keys will no longer be viable after 2010 and advises moving to 2048-bit RSA keys:

If information is initially signed in 2009 and needs to remain secure for a maximum of ten years (i.e., from 2009 to 2019), a 1024-bit RSA key would not provide sufficient protection between 2011 and 2019 and, therefore, it is not recommended that 1024-bit RSA be used in this case.

And the protagonists are:

• stud, @e207dbc5a3, with session cache enabled for 20,000 sessions, listen backlog of 1000 connections,
• stunnel 4.44, configured with --disable-libwrap --with-threads=pthread (configuration),
• nginx 1.1.4, configured with --with-http_ssl_module (configuration).

UPDATED: Configuration of nginx has been updated to switch to off both multi_accept and accept_mutex thanks to the feedback of Maxim Dounin on latency issues that were present in the previous article (and still present in the early version of this article). HAProxy (configuration) is used for stunnel and stud to add load-balancing. For each test, we want to maximize the number of HTTP transactions per second (TPS) while minimizing latency and unsucessful transactions. The maximum number of allowed failed transactions is set at 1 transaction out of 1000. The maximum allowed average latency is 100 ms².

Control case Below is the first comparison between stud, nginx and stunnel on plain Ubuntu Lucid. Like in previous benchs, stunnel is not able to scale, with a maximum of 400 TPS. However, early latency issues of nginx has been solved, thanks to the change in configuration. stud and nginx gets similar performance but, as you can see on the bottom plot (whose scale is logarithmic) stud starts with an initial latency of 40 ms. This is pretty annoying. I have tried some micro optimization to solve this issue but without luck. If latency matters for you, stick with stunnel or nginx (or find where the problem lies in stud). UPDATED: After reporting this issue, a fix has been commited. The culprit was Nagle s algorithm which improves bandwidth at the expense of latency. With the fix, performances are mostly the same but there is no latency penalty any more.

OpenSSL 1.0.0e When I published the first round of benchmarks, Micha Trojnara was dubious about the bad results of stunnel. He told me on Twitter:

I solved it! You compiled stunnel against a 4 years old OpenSSL with MT-unsafe SSL_accept. Upgrade OpenSSL and rebuild stunnel. This bug was fixed in OpenSSL 1.0.0b. I implemented a (slow) workaround, so stunnel doesn t crash with earlier versions of OpenSSL.

Therefore, I backported Ubuntu Oneiric OpenSSL 1.0.0e to Ubuntu Lucid³ and, as you can see yourself on the plot below, there is a huge performance boost for stunnel: about 400%! stud features a slight improvement too (about 8%) but still starts with a latency of 40 ms while nginx does not improve.

Intel Accelerator Engine Most Linux distributions are still using OpenSSL 0.9.8. For example, Debian Squeeze ships with OpenSSL 0.9.8o, Ubuntu Lucid with 0.9.8k and RHEL 5.7 with 0.9.8e. Therefore, they are not eligible to recent performance improvements. To solve this, Intel released Intel Accelerator Engine, which also brings, among other things, AES-NI support:

This is essentially a collection of assembler modules from development OpenSSL branch, packed together as a standalone engine. Standalone means that it can be compiled outside OpenSSL source tree without patching the latter. Idea is to provide a way to utilize new code in already released OpenSSL versions constrained by support and release policies limiting their evolvement to genuine bug fixes.

Alas, in my tests, there was no improvement compared to OpenSSL 1.0.0e (already patched with AES-NI support). stunnel comes with out-of-the-box support for additional engines. Just add something like this to your stunnel.conf:

engine=dynamic
engineCtrl=SO_PATH:/WOO/intel-accel-1.4/libintel-accel.so
engineCtrl=ID:intel-accel
engineCtrl=LIST_ADD:1
engineCtrl=LOAD
engineCtrl=INIT

For stud, you should grab support for engine selection from GitHub and for nginx, just specify the engine with ssl_engine.

Intel Performance Primitives In his great article Accelerated SSL , Simon Kuhn shows huge performance improvements by using Intel s IPP library, a library aimed at leveraging power of multi-core processors and including some cryptographic primitives and a patch to make use of them in OpenSSL. Unfortunately, this library is not free, not even as in free beer . The patch provided by Intel only applies on OpenSSL 0.9.8j. Simon updated it for OpenSSL 1.0.0d. It also applies with a slight modification to OpenSSL 1.0.0e. Look at Simon s article for instructions on how to compile OpenSSL with IPP. Simon measured 220% improvement on RSA 1024-bit private key operations by using this library. Unfortunately, we don t get such a performance burst with stud which exhibits a 15% improvement (but remember that private key operations only happen once every 8 transactions thanks to session reuse). Worst, with stunnel, performances are reduced by one third and nginx just segfaults. I think such a library is risky to use in production.

Hyperthreading During the previous tests, only half the capacity of each core were used. For stud, adding more workers per core only slightly improves this situation (less than 10% with 4 workers per core). Therefore, I thought using hyperthreading could rectify the situation here. In this case, stud, stunnel and nginx are allocated 8 virtual cores instead of 4. stud only got a small improvement of 5%, nginx a moderate improvement of 11% while stunnel got a significant improvement of 26%.

Conclusion Here is a quick recap of the maximum TPS reached on each tests (while keeping average response time below 100 ms). All tests were run on 4 physical cores with 2048-bit certificates, AES128-SHA cipher suite. Each client makes 8 HTTP requests and reuse the first SSL session.

Context	nginx 1.1.4	stunnel 4.44	stud @e207dbc5a3
OpenSSL 0.9.8k	1489 TPS	400 TPS	1492 TPS
OpenSSL 1.0.0e	1501 TPS	1561 TPS	1609 TPS
OpenSSL 1.0.0e + IPP	-	1101 TPS	1857 TPS
OpenSSL 1.0.0e + HT	1718 TPS	1973 TPS	1694 TPS

I also would like to stress that below 1500 TPS, stunnel adds an overhead of a few milliseconds while stud adds 40 milliseconds. With OpenSSL 1.0.0e and hyperthreading enabled, stunnel has both good performance and low latency on low loads to moderate loads. You may just want to limit the number of allowed connections to avoid collapsing on high loads. While in the previous round, stud was the champion and stunnel lagged behind, the situation is now reversed : stunnel is on top! UPDATED: As said earlier, the latency issue with stud has been solved. Performances remain mostly the same.

---

1. HAProxy and network cards don t need dedicated core for this kind of load. However, if you also want to handle non SSL traffic, those dedicated cores can get busy.
2. The real condition is more complex. Each point is considered valid if the corresponding average latency is less than 100 ms or if the average on the next 8 points is less than 100 ms. Temporary latency bursts are therefore allowed. We are not so forgiveful with failed transactions.
3. This is pretty straighforward: I just dropped multiarch support. I chose to use Ubuntu version of OpenSSL instead of upstream because there are some interesting patches, like automatic support of AES-NI.

It's become kind of a clich for me to claim that the reason I'm happy working on ACPI and UEFI and similarly arcane pieces of convoluted functionality is that no matter how bad things are there's at least some form of documentation and there's a well-understood language at the heart of them. My PhD was in biology, working on fruitflies. They're a poorly documented set of layering violations which only work because of side-effects at the quantum level, and they tend to die at inconvenient times. They're made up of 165 million bases of a byte code language that's almost impossible to bootstrap[1] and which passes through an intermediate representations before it does anything useful[2]. It's an awful field to try to do rigorous work in because your attempts to impose any kind of meaningful order on what you're looking at are pretty much guaranteed to be sufficiently naive that your results bear a resemblance to reality more by accident than design.

The field of bioinformatics is a fairly young one, and because of that it's very easy to be ignorant of its history. Crick and Watson (and those other people) determined the structure of DNA. Sanger worked out how to sequence proteins and nucleic acids. Some other people made all of these things faster and better and now we have huge sequence databases that mean we can get hold of an intractable quantity of data faster than we could ever plausibly need to, and what else is there to know?

Margaret Dayhoff graduated with a PhD in quantum chemistry from Columbia, where she'd performed computational analysis of various molecules to calculate their resonance energies[3]. The next few years involved plenty of worthwhile research that aren't relevant to the story, so we'll (entirely unfairly) skip forward to the early 60s and the problem of turning a set of sequence fragments into a single sequence. Dayhoff worked on a suite of applications called "Comprotein". The original paper can be downloaded here, and it's a charming look back at a rigorous analysis of a problem that anyone in the field would take for granted these days. Modern fragment assembly involves taking millions of DNA sequence reads and assembling them into an entire genome. In 1960, we were still at the point where it was only just getting impractical to do everything by hand.

This single piece of software was arguably the birth of modern bioinformatics, the creation of a computational method for taking sequence data and turning it into something more useful. But Dayhoff didn't stop there. The 60s brought a growing realisation that small sequence differences between the same protein in related species could give insight into their evolutionary past. In 1965 Dayhoff released the first edition of the Atlas of Protein Sequence and Structure, containing all 65 protein sequences that had been determined by then. Around the same time she developed computational methods for analysing the evolutionary relationship of these sequences, helping produce the first computationally generated phylogenetic tree. Her single-letter representation of amino acids was born of necessity[4] but remains the standard for protein sequences. And the atlas of 65 protein sequences developed into the Protein Information Resource, a dial-up database that allowed researchers to download the sequences they were interested in. It's now part of UniProt, the world's largest protein database.

Her contributions to the field were immense. Every aspect of her work on bioinformatics is present in the modern day larger, faster and more capable, but still very much tied to the techniques and concepts she pioneered. And so it still puzzles me that I only heard of her for the first time when I went back to write the introduction to my thesis. She's remembered today in the form of the Margaret Oakley Dayhoff award for women showing high promise in biophysics, having died of a heart attack at only 57.

I don't work on fruitflies any more, and to be honest I'm not terribly upset by that. But it's still somewhat disconcerting that I spent almost 10 years working in a field so defined by one person that I knew so little about. So my contribution to Ada Lovelace day is to highlight a pivotal woman in science who heavily influenced my life without me even knowing.

[1] You think it's difficult bringing up a compiler on a new architecture? Try bringing up a fruitfly from scratch.
[2] Except for the cases where the low-level language itself is functionally significant, and the cases where the intermediate representation is functionally significant.
[3] Something that seems to have involved a lot of putting punch cards through a set of machines, getting new cards out, and repeating. I'm glad I live in the future.
[4] The three-letter representation took up too much space on punch cards

comments

Mankind, it would appear, owe a great deal of their evolutionary advantage to using tools. This ability appears to have been massively amplified by our creation of machine tools.

Some years ago I came across a really peculiar newsgroup post. It was not about technicalities of any sort. It was about history. A beautifully written history of computers. From the Turing machine to the Free Software world, the original author managed to capture all the important events of the computer revolution with a great deal of humor. Re-posting it here, it is just brilliant. The Gospel of Tux (v1.0) In the beginning Turing created the Machine. And the Machine was crufty and bodacious, existing in theory only. And von Neumann looked upon the Machine, and saw that it was crufty. He divided the Machine into two Abstractions, the Data and the Code, and yet the two were one Architecture. This is a great Mystery, and the beginning of wisdom. And von Neumann spoke unto the Architecture, and blessed it, saying, Go forth and replicate, freely exchanging data and code, and bring forth all manner of devices unto the earth. And it was so, and it was cool. The Architecture prospered and was implemented in hardware and software. And it brought forth many Systems unto the earth. The first Systems were mighty giants; many great works of renown did they accomplish. Among them were Colossus, the codebreaker; ENIAC, the targeter; EDSAC and MULTIVAC and all manner of froody creatures ending in AC, the experimenters; and SAGE, the defender of the sky and father of all networks. These were the mighty giants of old, the first children of Turing, and their works are written in the Books of the Ancients. This was the First Age, the age of Lore. Now the sons of Marketing looked upon the children of Turing, and saw that they were swift of mind and terse of name and had many great and baleful attributes. And they said unto themselves, Let us go now and make us Corporations, to bind the Systems to our own use that they may bring us great fortune. With sweet words did they lure their customers, and with many chains did they bind the Systems, to fashion them after their own image. And the sons of Marketing fashioned themselves Suits to wear, the better to lure their customers, and wrote grave and perilous Licenses, the better to bind the Systems. And the sons of Marketing thus became known as Suits, despising and being despised by the true Engineers, the children of von Neumann. And the Systems and their Corporations replicated and grew numerous upon the earth. In those days there were IBM and Digital, Burroughs and Honeywell, Unisys and Rand, and many others. And they each kept to their own System, hardware and software, and did not interchange, for their Licences forbade it. This was the Second Age, the age of Mainframes. Now it came to pass that the spirits of Turing and von Neumann looked upon the earth and were displeased. The Systems and their Corporations had grown large and bulky, and Suits ruled over true Engineers. And the Customers groaned and cried loudly unto heaven, saying, Oh that there would be created a System mighty in power, yet small in size, able to reach into the very home! And the Engineers groaned and cried likewise, saying, Oh, that a deliverer would arise to grant us freedom from these oppressing Suits and their grave and perilous Licences, and send us a System of our own, that we may hack therein! And the spirits of Turing and von Neumann heard the cries and were moved, and said unto each other, Let us go down and fabricate a Breakthrough, that these cries may be stilled. And that day the spirits of Turing and von Neumann spake unto Moore of Intel, granting him insight and wisdom to understand the future. And Moore was with chip, and he brought forth the chip and named it 4004. And Moore did bless the Chip, saying, Thou art a Breakthrough; with my own Corporation have I fabricated thee. Thou thou art yet as small as a dust mote, yet shall thou grow and replicate unto the size of a mountain, and conquer all before thee. This blessing I give unto thee: every eighteen months shall thou double in capacity, until the end of the age. This is Moore s Law, which endures unto this day. And the birth of 4004 was the beginning of the Third Age, the age of Microchips. And as the Mainframes and their Systems and Corporations had flourished, so did the Microchips and their Systems and Corporations. And their lineage was on this wise: Moore begat Intel. Intel begat Mostech, Zilog and Atari. Mostech begat 6502, and Zilog begat Z80. Intel also begat 8800, who begat Altair; and 8086, mother of all PCs. 6502 begat Commodore, who begat PET and 64; and Apple, who begat 2. (Apple is the great Mystery, the Fruit that was devoured, yet bloomed again.) Atari begat 800 and 1200, masters of the game, who were destroyed by Sega and Nintendo. Xerox begat PARC. Commodore and PARC begat Amiga, creator of fine arts; Apple and PARC begat Lisa, who begat Macintosh, who begat iMac. Atari and PARC begat ST, the music maker, who died and was no more. Z80 begat Sinclair the dwarf, TRS-80 and CP/M, who begat many machines, but soon passed from this world. Altair, Apple and Commodore together begat Microsoft, the Great Darkness which is called Abomination, Destroyer of the Earth, the Gates of Hell. Now it came to pass in the Age of Microchips that IBM, the greatest of the Mainframe Corporations, looked upon the young Microchip Systems and was greatly vexed. And in their vexation and wrath they smote the earth and created the IBM PC. The PC was without sound and colour, crufty and bodacious in great measure, and its likeness was a tramp, yet the Customers were greatly moved and did purchase the PC in great numbers. And IBM sought about for an Operating System Provider, for in their haste they had not created one, nor had they forged a suitably grave and perilous License, saying, First we will build the market, then we will create a new System, one in our own image, and bound by our Licence. But they reasoned thus out of pride and not wisdom, not forseeing the wrath which was to come. And IBM came unto Microsoft, who licensed unto them QDOS, the child of CP/M and 8086. (8086 was the daughter of Intel, the child of Moore). And QDOS grew, and was named MS-DOS. And MS-DOS and the PC together waxed mighty, and conquered all markets, replicating and taking possession thereof, in accordance with Moore s Law. And Intel grew terrible and devoured all her children, such that no chip could stand before her. And Microsoft grew proud and devoured IBM, and this was a great marvel in the land. All these things are written in the Books of the Deeds of Microsoft. In the fullness of time MS-DOS begat Windows. And this is the lineage of Windows: CP/M begat QDOS. QDOS begat DOS 1.0. DOS 1.0 begat DOS 2.0 by way of Unix. DOS 2.0 begat Windows 3.11 by way of PARC and Macintosh. IBM and Microsoft begat OS/2, who begat Windows NT and Warp, the lost OS of lore. Windows 3.11 begat Windows 95 after triumphing over Macintosh in a mighty Battle of Licences. Windows NT begat NT 4.0 by way of Windows 95. NT 4.0 begat NT 5.0, the OS also called Windows 2000, The Millenium Bug, Doomsday, Armageddon, The End Of All Things. Now it came to pass that Microsoft had waxed great and mighty among the Microchip Corporations; mighter than any of the Mainframe Corporations before it had it waxed. And Gates heart was hardened, and he swore unto his Customers and their Engineers the words of this curse: Children of von Neumann, hear me. IBM and the Mainframe Corporations bound thy forefathers with grave and perilous Licences, such that ye cried unto the spirits of Turing and von Neumann for deliverance. Now I say unto ye: I am greater than any Corporation before me. Will I loosen your Licences? Nay, I will bind thee with Licences twice as grave and ten times more perilous than my forefathers. I will engrave my Licence on thy heart and write my Serial Number upon thy frontal lobes. I will bind thee to the Windows Platform with cunning artifices and with devious schemes. I will bind thee to the Intel Chipset with crufty code and with gnarly APIs. I will capture and enslave thee as no generation has been enslaved before. And wherefore will ye cry then unto the spirits of Turing, and von Neumann, and Moore? They cannot hear ye. I am become a greater Power than they. Ye shall cry only unto me, and shall live by my mercy and my wrath. I am the Gates of Hell; I hold the portal to MSNBC and the keys to the Blue Screen of Death. Be ye afraid; be ye greatly afraid; serve only me, and live. And the people were cowed in terror and gave homage to Microsoft, and endured the many grave and perilous trials which the Windows platform and its greatly bodacious Licence forced upon them. And once again did they cry to Turing and von Neumann and Moore for a deliverer, but none was found equal to the task until the birth of Linux. These are the generations of Linux: SAGE begat ARPA, which begat TCP/IP, and Aloha, which begat Ethernet. Bell begat Multics, which begat C, which begat Unix. Unix and TCP/IP begat Internet, which begat the World Wide Web. Unix begat RMS, father of the great GNU, which begat the Libraries and Emacs, chief of the Utilities. In the days of the Web, Internet and Ethernet begat the Intranet LAN, which rose to renown among all Corporations and prepared the way for the Penguin. And Linus and the Web begat the Kernel through Unix. The Kernel, the Libraries and the Utilities together are the Distribution, the one Penguin in many forms, forever and ever praised. Now in those days there was in the land of Helsinki a young scholar named Linus the Torvald. Linus was a devout man, a disciple of RMS and mighty in the spirit of Turing, von Neumann and Moore. One day as he was meditating on the Architecture, Linus fell into a trance and was granted a vision. And in the vision he saw a great Penguin, serene and well-favoured, sitting upon an ice floe eating fish. And at the sight of the Penguin Linus was deeply afraid, and he cried unto the spirits of Turing, von Neumann and Moore for an interpretation of the dream. And in the dream the spirits of Turing, von Neumann and Moore answered and spoke unto him, saying, Fear not, Linus, most beloved hacker. You are exceedingly cool and froody. The great Penguin which you see is an Operating System which you shall create and deploy unto the earth. The ice-floe is the earth and all the systems thereof, upon which the Penguin shall rest and rejoice at the completion of its task. And the fish on which the Penguin feeds are the crufty Licensed codebases which swim beneath all the earth s systems. The Penguin shall hunt and devour all that is crufty, gnarly and bodacious; all code which wriggles like spaghetti, or is infested with blighting creatures, or is bound by grave and perilous Licences shall it capture. And in capturing shall it replicate, and in replicating shall it document, and in documentation shall it bring freedom, serenity and most cool froodiness to the earth and all who code therein. Linus rose from meditation and created a tiny Operating System Kernel as the dream had foreshewn him; in the manner of RMS, he released the Kernel unto the World Wide Web for all to take and behold. And in the fulness of Internet Time the Kernel grew and replicated, becoming most cool and exceedingly froody, until at last it was recognised as indeed a great and mighty Penguin, whose name was Tux. And the followers of Linus took refuge in the Kernel, the Libraries and the Utilities; they installed Distribution after Distribution, and made sacrifice unto the GNU and the Penguin, and gave thanks to the spirits of Turing, von Neumann and Moore, for their deliverance from the hand of Microsoft. And this was the beginning of the Fourth Age, the age of Open Source. Now there is much more to be said about the exceeding strange and wonderful events of those days; how some Suits of Microsoft plotted war upon the Penguin, but were discovered on a Halloween Eve; how Gates fell among lawyers and was betrayed and crucified by his former friends, the apostles of Media; how the mercenary Knights of the Red Hat brought the gospel of the Penguin into the halls of the Corporations; and even of the dispute between the brethren of Gnome and KDE over a trollish Licence. But all these things are recorded elsewhere, in the Books of the Deeds of the Penguin and the Chronicles of the Fourth Age, and I suppose if they were all narrated they would fill a stack of DVDs as deep and perilous as a Usenet Newsgroup. Now may you code in the power of the Source; may the Kernel, the Libraries and the Utilities be with you, throughout all Distributions, until the end of the Epoch. Amen.

Just a quick note to remind everyone that the Rcpp class in San Francisco, which I am holding together with Revolution Analytics, will take place a week from today. We are happy to report that the number of registrations has met our initial targets. But as a number of open slots remain, we have decided to offer a few places at discounts of 25% for academics (with code acad1) and 50% for students (with code student). Course details are at the Revolution course page, registration is at the Eventbrite page. And just for completeness, here is what I wrote in the previous announcement:

The format will follow the workshop Romain and I gave during the tutorial day preceding this year's R/Finance conference. The style will once again be hands-on, with copious concrete examples and solid coverage of most aspects of Rcpp and related packages such as RInside, RcppArmadillo and others. The eight-hour schedule contains about six hours of instruction, split into four sessions of around ninety minutes. This leaves ample time for both lunch and coffee breaks, and for informal discussions and Q+A. The one-day class will be offered in San Franciso on Saturday, October 8, 2011. Please see the official course page for more details, concrete location info and maps as well as registration details.

Feel free to contact me at the usual email address with questions. Or with suggestions for the after-party in San Francisco :)

As the administrator for several MTAs I've received my fair share of support requests for "unreadable" e-mails. They generally have an attachment called winmail.dat. This is what all (for exceptions see the rest of this blog post) non-Outlook MUAs can display, when they receive an e-mail with the application/ms-tnef (or application/vnd.ms-tnef) MIME type. TNEF is a propietary format cooked up in Redmond. When chosen, the e-mail in question isn't sent as text/plain or at least text/html, but the entire e-mail is put into an TNEF-encoded attachment, most often called winmail.dat, in which you (most commonly) find the content of the e-mail in RTF, possible attachments are also put into the winmail.dat. The easieast solution for all involved parties is to convince the sender, that her Outlook and/or Exchange setup needs fixing (disabling TNEF in Outlook and Exchange 2003). Sadly this won't work all the time. The other solutions are to either filter the e-mails on your e-mail server and run them through something like ytnef_smtpd.py (please note, that this might be legally problematic for you as a service provider, opening an e-mail, even automatically, is a violation of communication privacy rights in many jurisdictions). Or, if the legal problem is too big or can't be solved through an local server installation (e.g. because there is no server running 24/7), you're left with either using Outlook (probably what Microsoft intended) or you can use a plug-in/add-on for your client. I've tested the LookOut add-on for icedove and it seems to work well. There are of course plug-ins for many other clients, but I can't honestly tell whether they work or not. [UPDATE 2011-09-20]: Fellow Debian Developer Josselin Mouette wrote me, that Evolution is capable of reading TNEF-encoded e-mails out of the box, but he also said that [TNEF e-mails should] be banned by all means.[/UPDATE 2011-09-20] Now, before I close, please always try to convince the sender/postmaster to fix their setup. It has only advantages: text/plain e-mails are smaller, they're less likely to contain malicious code (i.e. no parts hidden away from the virus scanner on the mail server (amavis can read TNEF e-mails, if libconvert-tnef-perl is installed)) and all recipients can open the e-mails with the client of their choice.

Two tiny bits:

• Thanks to Micha Lenk and after a fruitful discussion with Andrew Ruthven, you can finally find the Python bindings for Gnucash in the archive.
• If you want to check how your favourite package evolved over time: All historic build times and disk space information available in the Debian build logs are now imported into our database and are hence visible on log overview pages such as the insighttoolkit one.