hadess asks:

You are in a mall when zombies attack. You have:

1. One weapon
2. One song blasting on the speakers
3. One famous person to fight along side you.

Rock and roll :

1. A chainsaw babe!
2. Disturbed Get down with the sickness
3. Denise Richards

I just wrote about the system administration issues related to the recent Debian SSL/SSH security flaw [1]. The next thing we need to consider is how we can change things to reduce the incidence of such problems. The problem we just had was due to the most important part of the entropy supply for the random number generator not being used due to a mistake in commenting out some code. The only entropy that was used was the PID number of the process which uses the SSL library code which gives us 15 bits of entropy. It seems to me that if we had zero bits of entropy the problem would have been discovered a lot sooner (almost certainly before the code was released in a stable version). Therefore it seems that using a second-rate source of entropy (which was never required) masked the problem that the primary source of entropy was not working. Would it make sense to have a practice of not using such second-rate sources of entropy to reduce the risk of such problems being undetected for any length of time? Is this a general issue or just a corner case? Joss makes some suggestions for process improvements [2]. He suggests that having a single .diff.gz file (the traditional method for maintaining Debian packages) that directly contains all patches can obscure some patches. The other extreme is when you have a patch management system with several dozen small patches and the reader has to try and discover what each of them does. For an example of this see the 43 patches which are included in the Debian PAM package for Etch, also note that the PAM system is comprised of many separate shared objects (modules), this means that the patching system lends itself to having one patch per module and thus 43 patches for PAM isn’t as difficult to manage as 43 patches for a complex package which is not comprised of multiple separate components might be. That said I think that there is some potential for separating out patches. Having a distinction between different types of patches might help. For example we could have a patch for Makefiles etc (including autoconf etc), a patch for adding features, and a patch for fixing bugs. Then people reviewing the source for potential bugs could pay a lot of attention to bug fixes, a moderate amount of attention to new features, and casually skim the Makefile stuff. The problem began with this mailing list discussion [3]. Kurt’s first message starts with “When debbuging applications” and ends with “What do you people think about removing those 2 lines of code?“. The reply he received from Ulf (a member of the OpenSSL development team) is “If it helps with debugging, I’m in favor of removing them“. It seems to me that there might have been a miscommunication there, Ulf may have believed that the discussion only concerned a debugging built and not a build that would eventually end up on millions of machines. It seems possible that the reaction would have been different if Kurt had mentioned that he wanted to have a single source tree for both debugging and for regular use. It also seems likely that his proposed change may have received more inspection if he had clearly stated that he was doing to include it in Debian where it would be used by millions of people. When I am doing Debian development I generally don’t mention all the time “this code will be used by millions of people so it’s important that we get it right“, although I do sometimes make such statements if I feel that my questions are not getting the amount of consideration from upstream that a binary package destined for use by millions of people deserves. Maybe it would be a good practice to clarify such things in the case of important packages. For a package that is relevant to the security of the entire distribution (and possibly to other machines around the net - as demonstrated in this case) it doesn’t seem unreasonable to include a post-script mentioning the scope of the code use (it could be done with an alternate SIG if using a MUA that allows selecting from multiple SIGs in a convenient manner). In the response from the OpenSSL upstream [4] it is claimed that the mailing list used was not the correct one. Branden points out that the openssl-team mailing list address seems not to be documented anywhere [5]. One thing to be learned from this is that distribution developers need to be proactive in making contact with upstream developers. You might think that building packages for a major distribution and asking questions about it on the mailing list would result in someone from the team noticing and mentioning any other things that you might need to do. But maybe it would make sense to send private mail to one of the core developers, introduce yourself, and ask for advice on the best way to manage communication to avoid this type of confusion. I think that it is ideal for distribution developers to have the phone numbers of some of the upstream developers. If the upstream work is sponsored by the employer of one of the upstream developers then it seems reasonable to ask for their office phone number. Sometimes it’s easier to sort things out by phone than by email. Gunnar Wolf describes how the way this bug was discovered and handled shows that the Debian processes work [6]. A similar bug in proprietary software would probably not be discovered nearly as quickly and would almost certainly not be fixed in such a responsible manner. Update: According to the OpenSSL project about page [7], Ulf is actually not a “core” member, just a team member. I had used the term “core” in a slang manner based on the fact that Ulf has an official @openssl.org email address.

• [1] http://etbe.coker.com.au/2008/05/18/debian-ssh-problems/


• [2] http://np237.livejournal.com/17981.html


• [3] http://marc.info/?t=114651088900003&r=1&w=2


• [4] http://www.links.org/?p=328


• [5] http://advogato.org/person/branden/diary/5.html


• [6] http://gwolf.org/node/1743


• [7] http://www.openssl.org/about/

Share This

(tl;dr: if you have a few minutes, please add information here) Joss, the problem with the new package formats is that there is nothing that actually uses the additional information in a way that adds significant new functionality, so the net result of the change was that we throw away the information at a different layer in our software stack, and one of the interfaces got a lot more complicated in the process. One possible application would be a "poor man's patch tracking" inside the BTS, perhaps with a new state "fixed in patch". I can see two ways of implementing that:

1. by extending the interface of the "new" package formats that Debian,Ubuntu bug numbers are attached to the actual patch files and having the archive maintenance software extract and process that information (reject packages that add a patch for a bug without closing it in the changelog, notify the BTS), or
2. by leaving the package format untouched and simply adding a regex matching "Fixes: #nnnnnn" that is reported to the BTS as "we have added a patch", so the submitter is notified that the bug is gone for him/her; the bug is then closed in the changelog of the upload removing the patch.

The former approach also allows us to link to patches from BTS pages, which the latter doesn't, so there could be actual benefit here if we believe it is worth the additional complexity. (Update: Rapha l thinks it is. I like the idea of a package format with separate patches a lot more in this context than I did without it, but still my fear that it will actually be perceived as sanctioning large patchsets still remains.) About mandatory co-maintenance: the problem isn't "helping". We have plenty of people with commit access to packages they don't even remotely understand who are really helpful (not). The problem is that someone needs to actually read all the commit logs and understand what the changes do in this context. In most cases, that person or group would be upstream, not a DD. My first impression after reading the patch was "adding uninitialized data to the entropy pool is pointless/harmful as it is not random, so this patch makes sense", because the loop around it was not contained in the patch. Obviously I'm not an OpenSSL developer. There is nothing Debian could have done internally to verify the correctness of this patch that would properly scale to the entire archive, even if we put "more emphasis on security". The only solution I see is reporting every patch to upstream immediately and getting affirmation that it is correct. This, however means that we need to produce patches that upstream can accept. For obvious code bugs, that is simple, but for integration patches like paths it is not sufficient to replace one string with another, but rather make it configurable in some place that can be reached from debian/rules. In an ideal world, we end up with very few Debian specific patches, so essentially we are talking about adding functionality to dpkg that we don't want to use. I've started a page in the Debian Wiki, Getting Packaged with an outline of a possible document aimed at upstream developers that should list the typical problems we run into and how to avoid them.

There are obviously some things we need to remind if we don t want something like the OpenSSL debacle to happen again. It doesn t mean we need to throw stones nor to rush into changing our processes without thinking. However, there are already some things that should be obvious but unfortunately are not.

1. Shipping a giant diff.gz that contains all changes in one, putting security fixes, policy fixes, bug fixes, cosmetic changes and autotools files at the same level, is not something we should accept anymore. Improvements in the dpkg-source format are much welcome in this direction, but they are useless if maintainers don t use them. Neither a VCS nor a build tool will be able to know which line of the changes is related to which bug. Only the maintainer can.
2. Core packages should all have co-maintainers. This is pretty much stating the obvious, and is much easier said than done. The OpenSSL case is one of the best examples here: Kurt is not one of those who refuses help, but frankly, would you want to maintain that package? Having already maintained packages with messy code, upstreams not understanding at all the needs of a distributor, avalanches of security alerts and randomly-changing ABIs, I can tell you this is no fun like it can be to hack on a desktop environment or a device driver. The only sane reason to do this is that you need the package to work. The only visible result you get from your work is that programs are not randomly crashing.
   I have no magic recipe to propose so that more people help with such packages, and that s where we need to be really innovative. Cross-distribution teams, mandatory co-maintainership on a core package for each DD these (and all ideas I have not heard of) are the experiments we should start now.
3. Patching bad code leads to unpredictable results. What maintainer of a complex package has not introduced a new bug while trying to fix another one? Even when a piece of code is maintained by uncooperative developers, is not commented, uses arcane variable names or is impossible to understand without having contributed 3 winning entries to the IOCCC, it needs to evolve. And in these cases, it is only a matter of time until such things happen.
   Don t get me wrong: I m not trying to put the blame on upstream here. They have contributed very valuable code to the community and their work helped in the considerable widespread of cryptography. It s just that their code is not enough for our needs. If we can t patch it safely (and I m now convinced we can t), maybe we need to focus on alternatives and help them getting used by crypto-related packages. The code in GnuTLS and NSS is not necessarily better, but most (if not all) patches Debian needs to apply to them are build and portability fixes.
4. Unless Debian-specific, 1 patch = 1 bug in the upstream tracker. This should be obvious, but given the number of patches that are never forwarded, it doesn t seem so. You should not only give a chance for upstreams to review the patch, but you need them to track it, and you must give them the chance to review it anytime someone else stomps on a similar issue. If upstream does not have a bug tracker, they probably think their software has no bugs. Which means they are not trustworthy, and we go back to point 3.
5. We need to give more priority to security. Issues in the security team seem now fixed for good and they have been doing an awesome work. There isn t much left to do so that packages are all built with security-hardening features, but it still needs to be done. And there is much more to do so that we can provide out of the box a decent SELinux setup, or, if it turns out unrealistic to do, a decent system hardening setup using another framework. I know the SELinux zealots will jump on their high horses to explain that their framework is better, but the current situation where it is impossible for the average system engineer to setup a Debian-based MAC system is much worse than having a suboptimal setup that already works.

All in all, this incident has a great impact on Debian s image. If we don t react accordingly, adapting our processes and our system to match what our users expect from us and they expect the best they will turn away from us. With very good reasons to do so. Update : It seems OpenSSL does have a bug tracker. Thanks Kurt for pointing me to it.

Yesterday I stumbled about files and dead symlinks left in /usr/lib/python2.3/site-packages/ on my Sid box. These files/symlinks seem to have been shipped/created(?) by:

python-ldap python-cairo python-crypto kiki python-foomatic python-mysqldb
python-logilab-common python-egenix-mxtools python-numarray python-pygresql
python-imaging-sane python-imaging python-xml python-reportlab

Deleting /usr/lib/python2.3 (dpkg -S didn’t show any package relationship nor did I find something in /var/lib/dpkg/info/) and reinstalling the above mentioned packages did not recreate the files/symlinks. So it seems the directory can be safely removed. Maybe I missed some announcement or one (or more) packages need to be fixed. No time to examine it atm. Update This is known as Debian bug #409390. Thanks to Josselin Mouette for the information.

$ sudo dpkg --set-selections
[sudo] password for joss: 
dpkg hold

Introducing biometric authentication in our systems is a very good idea. It is the key to two-factor authentication which, while not solving all security issues, can bring a giant leap in terms of security when done correctly.

Everything is in the correctly.

And I can t say I m impressed with the way people are developing fingerprint authentication systems. Let s look at the currently emerging standard, fprint, and its security notes:

In it's current state, fprint is not a very secure system: this data is stored on disk in unencrypted form. This data is not readable by other users, however it is possible that the super-user can access it, and also someone with local access could move the disk to another system in order to gain access to the whole disk.

This statement is followed by considerations on how to protect this data from being read, much like passwords are protected by MD5 hashes.

Wait Yes, they are treating biometrical data like a password. Password-based security relies on the fact it isn t easy to obtain your password without spying on you or torturing you. But obtaining your fingerprint? Unless you ve been wearing gloves for years, this is absolutely trivial. And this is how people focus on the wrong issues.

Biometric data is not a password, and it requires a radically different approach to authentication. You can t just expect the person in front of the computer to provide some data corresponding to the fingerprint. You have to ensure that the person in front of the computer is the one with the correct fingerprint. Biometric devices manufacturers have made impressive efforts to ensure fake fingers or cut fingers can t be used, but there is still a giant hole in the security model. While you can now be reasonably sure that the device will only return data corresponding to the person in front of it, you also need to ensure the data processed by the computer comes from the device.

Guess what? These devices are USB-based. And given how USB works, there is nothing that prevents an attacker to plug a custom device in the USB port and send the data he wants to the authentication system. Good job, guys. While you re busy hiding easily available data, the house s door is still wide open.

#451480 really made my day. I'll let you judge for yourself:

Package: debhelper
[snip]

When building a package and having libc6 2.7.0exp8 installed (from
experimental), debhelper (dh_shlibdeps ?) does not compute correct
dependencies:
Depends: libc6 (>= 2.7-1), xserver-xorg-core (>= 2:1.4)

The package is then uninstallable.

This strip is dedicated to lool.

If you know of an occasional passerby who takes the time to extract the contents of Debian source packages to see whether the RFCs are here or not, please let us know. We are always looking for new potential contributors.

— Josselin Mouette

This article is interesting to relate to thread patterns and has some great "authorline" graphs of how different people post to threads.

This is the sum of the votes for the three fascist candidates at yesterday's French election. Not forgetting the 5.79 % of trotskist candidates, you can see how much the French like democracy. All in all, only 49.12 % of the French voted for a democratic candidate (christian-democrat, social-democrat, communist, green, post-globalization). Think about it before saying again that France is the country of human rights. This is confirming that after five years of Sarkozy and his omnipresent police - regularly pinpointed by Amnesty International - we're already the country of human half-rights.

Joss, you made my day.

Josselin, I feel flattered that I play a role in your latest Debian cabal strip, I found the past two very funny. However, I don't get the joke of the third one: I do not have any package in Debian using debconf. Are you mistaking me for someone else?