I'm currently considering how to allow users to log on with different attributes as user names, e.g. with their "real" user name or their mail adress. Unfortunatly as described on http://httpd.apache.org/docs/2.2/mod/mod_authnz_ldap.html#authldapurl, though RFC 2255 allows a comma-separated list, only the first attribute is used. Now, of course an idea would be to specify all different attributes as a new "loginas"-type one. Another solution would be to use ldap overlay modules, and just convert them "on the fly". Better ideas would be welcome.

aba@ries:~$ grep final update_out/update.OUTPUT_py   cut -f 2 -d ' '  tr , '\n' wc -l
901

This means 901 packages went from unstable to testing with the most recent britney run (though counting each binNMUed source package as ~12 packages here). Only about 200 packages which are otherwise ready for migration tried to go to testing, the vast majority reached testing. Major reason for this move is that now libselinux was ready, and we were able to migrate libselinux, pango, glib2.0 and ocaml which dependend on all of them (the last transition being responsible for about 700 packages). Good news for armel as well:

start: 61+442: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-11:p-0:s-0:s-12:a-442
  end: 60+367: i-2:a-0:a-0:a-21:h-11:i-0:m-4:m-10:p-0:s-0:s-12:a-367

So some progress as well - it's a steady move towards a reasonable uninstallability count (the last number being armel).

Ldap seems like a good database to store accounts in: Lots of tools can use it, it's easily integrated into pam, and lots more. But one sometimes just hits borders in the data model of openldap: There are no semantics available. So, let me give you an example. Take e.g. the dnsZoneEntry attribute in the debian.org-ldap. Actually, one wants to allow anyone to write into that attribute in his own entry. But - and that's a big but - one wants to have two additional checks before writing: One is that one cannot claim any dns entry already used by someone else. And the other is that the format needs to comply to specific standards. Now the question is, is there some way to use ldap with more semantics? Or does one need to write ones own backend to do that (and in all the non-plain-db*-backends, it is specified that one needs to write his own authorization code as well). Or some other recommended way to do that? (Or is the only existing incarnation of that ActiveDirectory?) Update: Thanks to Faidon Liambotis I know now about the overlays in openldap. The standard overlays unique and constraint will probably solve the case above - of course, I have a few more complex cases, but that are at least good starters.

trying: mysql-dfsg-5.0
accepted: mysql-dfsg-5.0
   ori: 71+890: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-890
   pre: 71+887: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-887
   now: 71+773: i-3:a-0:a-0:a-23:h-13:i-0:m-8:m-10:p-0:s-0:s-14:a-773

In other words, the transition of mysql-dfsg-5.0 to testing has reduced the uninstallability on armel from about 900 packages to less than 800. Still some way to go, but good progress.

I'm looking for some time for a good user self management. "Good" means: User can add their own accounts, gets approved (or not), can recover the password, ... Of course, there are plenty implementations of that available in the market - most wikis have that, as well as lots of other applications. But all of them are embedded-only, i.e. one has to use that web application and gets "vendor-locked". Isn't there anything open source and standalone available, following the basic unix principles of "do one thing and do it well"?

For my research I wanted to know how to obtain the exact number of Debian developers. Thanks to help from Andreas Barth and Manoj Srivastava, I can now document the procedure:

$ ldapsearch -xLLLH ldap://db.debian.org -b ou=users,dc=debian,dc=org \
  gidNumber=800 keyFingerPrint \
    sed -rne ':s;/^dn:/bl;n;bs;:l;n;/^keyFingerPrint:/ p;bs ' \
    wc -l
1049

This actually seems enough as I do not recall any new maintainers being added since the last call for votes, which gives 1049 as well. Andreas told me to count the number of entries in LDAP with GID 800 and an associated key in the Debian keyring. Manoj's dvt-quorum script also takes the Debian keyrings (GPG and PGP) into account, so I did the same:

$ ldapsearch -xLLLH ldap://db.debian.org -b ou=users,dc=debian,dc=org \
  gidNumber=800 keyFingerPrint \
    sed -rne ':s;/^dn:/bl;n;bs;
              :l;n;/^keyFingerPrint:/ s,keyFingerPrint: ,,p;bs ' \
    sort -u > ldapfprs
$ rsync -az --progress \
  keyring.debian.org::keyrings/keyrings/debian-keyring.gpg \
  ./debian-keyring.gpg
$ gpg --homedir . --no-default-keyring --keyring debian-keyring.gpg \
  --no-options --always-trust --no-permission-warning \
  --no-auto-check-trustdb --armor --rfc1991 --fingerprint \
  --fast-list-mode --fixed-list-mode --with-colons --list-keys \
    sed -rne 's,^fpr:::::::::([[:xdigit:]]+):,\1,p' \
    sort -u > gpgfprs
$ rsync -az --progress \
  keyring.debian.org::keyrings/keyrings/debian-keyring.pgp \
  ./debian-keyring.pgp
$ gpg --homedir . --no-default-keyring --keyring debian-keyring.pgp \
  --no-options --always-trust --no-permission-warning \
  --no-auto-check-trustdb --armor --rfc1991 --fingerprint \
  --fast-list-mode --fixed-list-mode --list-keys \
    sed -rne 's,^[[:space:]]+Key fingerprint = ,,;T;s,[[:space:]]+,,gp' \
    sort -u > pgpfprs
$ sort ldapfprs pgpfprs gpgfprs   uniq -c \
    egrep -c '^[[:space:]]+2[[:space:]]'
1048

MAN OVER BOARD! Who's the black sheep? Update: In the initial post, I forgot the option --fixed-list-mode and hit a minor bug in gnupg. I have since updated the above commands. Thus, there is no more black sheep and the rest of this post only lingers here for posterity.

while read i; do
  grep "^$ i $" pgpfprs gpgfprs   echo $i >&2
done < ldapfprs >/dev/null

which returns 9BF093BC475BABF8B6AEA5F6D7C3F131AB2A91F5

$ gpg --list-keys 9BF093BC475BABF8B6AEA5F6D7C3F131AB2A91F5
pub   4096R/AB2A91F5 2004-08-20
uid                  James Troup <james@nocrew.org>

our very own keyring master James Troup. So has James subverted the project? Is he actually not a Debian developer? Given the position(s) he holds, does that mean that the project is doomed? Ha! I am so tempted to end right here, but since my readers are used to getting all the facts, here's the deal: James is so special that he gets to be the only one to have a key in our GPG keyring which can be used for encryption, or so I found out as I was researching this. Now this bug in gnupg actually causes his fingerprint not to be printed. Until this is fixed (if ever), simply leave out --fast-list-mode in the above commands. NP: Oceansize: Effloresce

During the recent weeks, we started to hunt and hint large batches of packages together to testing (and this blog is just to show a bit of what is part of the tasks of release team members). This started with a large jasper hint, continued with a very large poppler / texlive / jack / abiword-hint (which moved about 75% of the packages that were ready for testing migration in at the same time), then moved a more recent linux-2.6 version in, and now finally ghc6 / haskell. However, now we reached a point where a few more release teamisch hints don't help anymore: The next large blocker is gcj-4.1 / gccdefaults / ecj. This is now blocked by different bugs on different architecture:

• mips* needs to have xulrunner built first to build gcj-4.1. This is discussed in bug #428582, and the porters are busy working on it.
• hppa fails for unknown reasons - probably it needs some porting. That should be discussed in bug #431865 but until now there was no answer from the hppa porters.
• arm already reached the state of being out-of-date to a degree that it needs bootstraping by hand - see http://lists.debian.org/debian-arm/2007/07/msg00014.html for reference.

Update: Aurelien Jarno informed me that gcj-4.1 had to be bootstrapped on all architectures, but it hasn't happened yet on arm due to bugs in gcj.

During the recent weeks, we started to hunt and hint large batches of packages together to testing (and this blog is just to show a bit of what is part of the tasks of release team members). This started with a large jasper hint, continued with a very large poppler / texlive / jack / abiword-hint (which moved about 75% of the packages that were ready for testing migration in at the same time), then moved a more recent linux-2.6 version in, and now finally ghc6 / haskell. However, now we reached a point where a few more release teamisch hints don't help anymore: The next large blocker is gcj-4.1 / gccdefaults / ecj. This is now blocked by different bugs on different architecture:

• mips* needs to have xulrunner built first to build gcj-4.1. This is discussed in bug #428582, and the porters are busy working on it.
• hppa fails for unknown reasons - probably it needs some porting. That should be discussed in bug #431865 but until now there was no answer from the hppa porters.
• arm already reached the state of being out-of-date to a degree that it needs bootstraping by hand - see http://lists.debian.org/debian-arm/2007/07/msg00014.html for reference.

Debian had OpenMPI package since early last year when Florian Ragwitz made some initial stabs at packaging. The package has seen a number of NMU and patches since then, but was generally getting cobwebs ... which was too bad because OpenMPI seems to have some wind behind its sails upstream. Unfortunately, little of that got packaged for Debian. After some discussions on and around the debian-science list, a new maintainer group was formed on Alioth under the pkg-openmpi name. Tilman Koschnick (who had already helped Florian with patches), Manuel Prinz, Sylvestre Ledru and myself have gotten things in good enough shape in reasonably short time. And I have just uploaded a lintian-clean package set openmpi_1.2.3-0 to Debian, where it is expected to sit in the NEW queue for a little bit before moving on to the archive proper. The changelog entry (which will appear here eventually) shows twelve bugs closed. Our plan is to provide a stable and well maintained MPI implementation for Debian. OpenMPI is the designated successor to LAM, and apart from MPICH2, everybody seems to have thrown their weight behind OpenMPI. So we will try to work with the other MPI maintainers to come up with sensible setups, alternatives priorities and the likes. If you are interested in MPI and would like to help, come join us at the Alioth project pkg-openmpi. Last but not least, thanks to Florian for the initial packaging, and to Clint Adams, Mark Hymers, Andreas Barth, and Steve Langasek (twice even) for NMUs.

A new version of svn-buildpackage has appeared in unstable now. If you use this package, please test it, and if something bad happens, please file appropriate bugs and alert the release team. Thanks.

I dived through the list of open RC issues, and found some without any (real) activity for at least 20 days. Please remember: If we want to release Etch, we need to get rid of any such bug prior to release.

• libglide2: Reproducible segmentation fault with Voodoo2: Latest info is from December 1st that a driver needs forward porting
• initramfs-tools: [powerpc64] i-t tries to mount RAID/LVM stuff before the disk are up -> unbootable: Last mail from December 11st with suggestions from Sven
• libmail-mboxparser-perl: FTBFS: build times out during test (t/3_while): Possible workaround already in the bug log
• lvm2: several issues with devmapper devices: The maintainer didn't care to comment at all until now
• parted: NTFS (partition) not recreated correctly after resize:incorrect start sector: lots of discussions on December 16th, quiet since then

One library is lying on ia64 - it tells libtool to have a link to /usr/lib/libasound.la, but there is no dependency to that on the package. Please compare http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=ia64&stamp=1166917613&file=log with http://buildd.debian.org/fetch.cgi?&pkg=gcompris&ver=8.2.2-1&arch=s390&stamp=1166227920&file=log. Now, one needs to locate the lying package and fix it.

That sometimes journalists have obvious reading problems, well, isn't a too new thing. One has to deal with that, e.g. by appending more text to the cited blog entry. If someone inside the Debian community fails for the same is way more disappointing for me. I have never stated that "[I] couldn't compensate this" - that is just totally wrong. I think that dunc-tank helped us with release of Etch, but the help could have been greater if some people wouldn't behave as childish as they do. And I also try to look both ways, and not be singled-minded about it. There is another thing I learned as well during the last 12 hours - people don't appreciate if one tries to summarise openly what happened, and look both ways. Is it so wrong to look in both directions, and not only say "that and that were the improvements"? I thought at least within Debian we don't want to fall the usual management mistake of only speaking about how great everything is, but be honest to ourselfs. People starting to rip only half of the words out, citing them out of the context, and inventing new quotes don't help with that.

Monday morning started with seeing lots of brand new release bugs about attr being broken so bad that "ls -la" failed, and also buildds were affected negatively. As a consequence, I few-hour-NMUed this package back to a sane (i.e. working) state. As any library maintainer, please remember: Whenever you drop a symbol - whichever this symbol is this means: You need to bump SONAME. Or just use library versioning from the beginning. Failing to do so can lead break any program linking to your code, which is especially bad if it is ls and cp. Rest of the day was mostly spent on further libpng-analysis. There is now a nice table on http://ries.debian.org/~aba/png.abis which symbols were exported in which version. We are progressing into resolving the libpng problems now.

This is the first of the two "larger" reports during the release management funding experiment. I suppose I don't need to be more specific about what I did in detail - my blog postings in http://blogs.turmzimmer.net/en/debian/etch-rm/ cover that already. The thing that ate most of my time are the release critical bugs. While it is possible for me on normal days to fix one bug every few days, it was now possible to fix a couple of bugs per day. This sounds rather good, but at the same time took away most of my time for other things I should have also worked on - that even involved blogging about my work. (Also in looking back that was still a good decision to work so much an release critical bugs - it would have been just more convenient if that wasn't necessary so much.) Looking back, I started with improving tools. I think that this was a good decision, good tools make work easier - and also, everyone can use the better tools, so they multiply their benefit. And e.g. the structural changes I did to bts.turmzimmer.net to allow "show also related bugs" were so deep that I probably wouldn't have been able to do them, unless I reserve a couple of hours in a row. In other words, being able to explicitly set some time aside for Debian which is not eaten up by the daily routine tasks (which involves reading quite much mail, BTW), is a good thing. So, looking at the status changes during the time I spent full-time on release issues I think it worked well. Of course, not everything is perfect, but there is a clear improvement. On the other hand, there was a large disadvantage of the whole experiment: Some people who used to do good work reduced their involvement drastically. There was nothing I could do about, and that happened way before I started full-time on release, but on the global picture that still counts. So, as a first summary, I am happy with my own involvement, but that doesn't necessarily apply to the full experiment. Update: There are media rumours floating around that "[Etch has] been delayed because some developers have deliberately slowed down their work". This doesn't reflect what I said. I just noted that the dunc-tank experiment has positive and negative effects, and we shouldn't watch only one side - whichever that side is. The reasons for the release being delayed from the original planned date has other reasons, please read the mails on debian-devel-announce for details (also all linked on http://release.debian.org/). And, I'm quite happy with the involvement of most Developers in the release. (And this paragraph isn't part of what this blog posting should be about really, but as it is cited, it is still here ...)

(This blog entry is a bit delayed because I have been on the QA meeting.) On Monday and Tuesday last week we did one important step: We sent out the release update that etch is fully frozen now. Preparing that, and then handling the incoming freeze requests took some of the time. Of course, some of the regular RC bug squashing happened as well - but that is rather a routine task now.

During current QA meeting, Amaya did a nice picture of us all.

People who have following our release update mails have seen that we are tracking different numbers this time. There is the total number of RC bugs relevant for etch, which is currently 122. And there is the number of RC bugs relevant for both etch and sid (i.e. "not handled yet"-bugs), which is currently 78 - and that number was basically the only number we could track for release of sarge. (Then there are also the numbers of bugs only in etch, and the number of aged bugs relevant for etch and sid - please see http://bts.turmzimmer.net/graph-small.png for the full stats.) So, if you look back we have written "RC bugs below 80" as precondition in our timeline. We copied that from the sarge release, were we had "~85 RC bugs" at time of freeze. During detailed planing, it became obvious that if we want to basically "go by the same numbers", that is (and can only be) the number of RC bugs applying to both etch and sid - as that was the only stats we really had during sarge release planing (please remember, we didn't had version tracking at that time, at the moment where a bug fix was uploaded to sid, the bug disappeared at all, unless someone reopened the bug report and tagged it sarge). So, we agreed that we can and should freeze Etch as soon as the combined number is below 80, and the number of fixes waiting for transition is not out of control (and a few other issues are resolved, please see the previous mail to debian-devel-announce for that). Please allow me also a personal remark: It is really sad when somone tries to hurt active Developers working on the release, while not contributing anything except spite. I don't think that is adequate behaviour.

Yay! Release team reported that Etch (current testing branch) is now frozen, Andreas Barth announced it.