Blogging about recent Friday is a bit late now, but there you are: On Friday, I was busy preparing the full freeze for Etch - this included e.g. polishing the announcement mail as well as letting linux 2.6.18 finally into Etch (and breaking some packages we cannot remove now), and of course more RC bug squashing. Thankfully, most of that worked out well, so we were able to freeze Etch today.

To say it with the words of Andreas Barth (one of the Debian release managers):

Etch is now frozen! Wheeeeeee!!!

On Monday, I sent out the next release update (which was under preparation for a few days). Basically, Debian is doing quite well - we just have a few items left open before we can have the full freeze. Of course, I worked also on one of the preconditions, namely on RC bugs - the usual trying to reproduce, uploading fixes, encouraging other people to upload fixes, easying testing transition, ...

I have been really busy until two weeks ago [Shameless plug: If you need a (technical) project manager and/or hands-on Basis person to get your SAP R/3 system upgraded to ECC6, get in touch.]. Which means I've been a bit slack - so slack in fact that Andreas Barth had to resolve an RC bug in one of my packages - mea culpa.

Ridden by guilt I decided to finally do what I've been wanting to for some time: Putting the steps together to do a Debian test upgrade using QEMU. I chose QEMU because it is free and readily comes with Debian. It is slower than VMware, at least without the non-free kernel module, but still usable on somewhat reasonable hardware. The actual upgrade steps given below are independent of the virtualisation technology used, so they will apply to VMware as well.

Without further ado, here goes:

• install the qemu package
• optionally install the kqemu module:
• install the kqemu-source package (from 'non-free')
  
• run 'module-assistant prepare'
• run 'module-assistant auto-build kqemu' (Daniel, in case you read this: This may be easier than running make-kpkg as suggested in the README.)
  
• install built package with 'dpkg -i kqemu-modules-[kernel version]'
• possibly change permisisons/ownership of '/dev/kqemu' to allow non-root users
  
• run 'depmod -a' and 'modprobe kqemu' to load module
  
• create a disk image (Using qcow means we are not using a sparse file, so compressing and decompressing does not inflate the file.): qemu-img create -f qcow hda_40GB_sarge.qcow 40G
• get a sarge install CD, e.g. 31r4 netinst
• install the initial sarge system:
• start qemu e.g. like this: qemu -m 512 -hda ./hda_40GB_sarge.qcow -cdrom ./sarge-31r4-i386-netinst.iso -boot d
• start the installer and go through the normal motions
• restart qemu like this: qemu -m 512 -hda ./hda_40GB_sarge.qcow
• finish the installation:
• select a mirror (Note that DNS is not setup - fix this or use IP addresses for repositories. Also, something like apt-proxy comes in quite handy.)
• choose 'manual package selection' in the installer
• update the packages as suggested by aptitude
• quit aptitude and finish the installation
• now would be a good time to take a copy of the fresh sarge install before continuing:
• halt the (qemu) system
• copy the image file, e.g.: cp hda_40GB_sarge.qcow hda_40GB_sarge_pristine.qcow
• compress the copy (The uncompressed image is ~ 1GB, the bzip'ed one ~ 100MB.)
  
• install X:
• start aptitude, select 'x-window-system-core' and install
• leave all X config settings standard, except:
• mouse device: /dev/psaux
• mouse type: PS/2
• screen size: 1024x768 (or similar)
• install Gnome (That's just what I use, KDE or others are equally fine.):
• start aptitude and select the following packages and install:
  
• 'gnome-desktop-environment'
• 'xscreensaver'
• 'gdm'
• 'gdm-thmes'
• 'synaptic'
• 'menu'
• 'menu-xdg'
• quit aptitude
• restart gdm: invoke-rc.d gdm restart
• now might be a good time to take another copy of the sarge install before continuing:
• halt the (qemu) system
• copy the image file, e.g.: cp hda_40GB_sarge.qcow hda_40GB_sarge_gnome.qcow
• compress the new copy (The uncompressed image is now ~ 2GB, the bzip'ed one ~ 540MB.)
  
• install other software as desired that you want test the upgrade with, e.g.:
• 'mondo'
• 'mondo-doc'
• perform the upgrade (following the preliminary Release Notes):
• get to the command line in the qemu system, e.g. by running 'invoke-rc.d gdm stop' in a gnome-terminal
• change apt source from sarge to etch
• run 'apt-get update'
• upgrade the kernel by running 'aptitude install linux-image-2.6-486' (This is not what the Release Notes say but it works for me.) (Say 'Yes' to removing the running kernel.)
• upgrade aptitude by running 'aptitude update aptitude'
• upgrade the system by running 'aptitude -f --with-recommends dist-upgrade'
• if applicable, rerun and keep rejecting offerings to resolve conflicts until the one upgrading mondo is offered (I'll have to look into this.)
• run 'aptitude -f --with-recommends install gnome-desktop-environment' to reinstall Gnome

All in all, things seem to work ok. However, this is far from hitting the friendly green upgrade button and it just happens. So, I thought I try the next best thing to the friendly green upgrade button which is synaptic. Doing an upgrade with synaptic does actually work quite smoothly. It needs to be restarted a few times, leaves some cruft in terms of obsolete and orphan packages and the reboot doesn't really work from within Gnome, but other than that it is ok. Most notably, it didn't leave me with an unusable system in the middle of it all.

Robert Collins made some interesting remarks about the challenges of upgrades last week when we had dinner with Martin Krafft and a number of other great people. It looks like the Ubuntu folks are working on improving update-manager but also the underlying infrastructure to smooth out the upgrade process. Maybe there could be an opportunity to work together on this and achieve a situation where upgrades become as smooth as installs are now due to the fantastic work of the d-i people.

There is a long-standing RC bug libglide2: Reproducible segmentation fault with Voodoo2 where it is easy to reproduce this bug if you have the hardware (warning: it will crash your computer), but - well, it just needs to be done.

During my cruise through release critical bugs, one sometimes wonder how often "team" should be translated with "toll, ein anderer macht's" (great, someone else will do it). Looking at bugs like e.g. #399589 - asterisk-oh323: Asterisk OH-323 crashes latest asterisk package, it doesn't seem to demanding to expect also in a team-maintained environment that any of the maintainers recheck (or at least send mail) to the person reporting the bug within a week. However, the bug was reported on November 20th (that's a Monday), and nothing happend up to now from their side - means: Nothing within 11 days. Unfortunatly, there are more so buggy packages from the VoIP-Team - most can just be moved out from Etch, but it is always unsatisfactory to remove packages.

On Wednesday, I send out the announcement about how (and which) non-free packages can be autobuild. Of course, that was not the only action - some more time was invested in fixing RC-bugs, and, unfortunatly, in finding an RC-bug within apt-get source. I also transfered the "which programs are uninstallable in etch"-program to the new ftp-master - and the good news is we only have left two programs uninstallable on arm (ignoring arch=all except on i386).

One of the good news is that PHP 5.2 finally got into testing. A bit of handholding PHP was part of my Tuesdays tasks (together with a removal of php-imagick, which I also NMUed afterwards, and which I need to handhold to go back into testing now). Tuesday allowed me to also upload a few more RC bug fixes. Besides of that, lots of the usual gruntwork and small items were done - nothing special to talk about, just answering couple of mails, reviewing packages, and all the rest of daily routine job (which becomes quite much shortly prior to release).

In case you are maintainer and have any RC bug sitting with your package, and you have already commited a resolution to your source repository, please upload your package in a timely manner. It is really annoying if such bugs sit there, and are not resolved (and have their negative impact about our users!), and at the same time, it wouldn't be too difficult for the maintainers to upload the fix.

I have been away for 5 days without any net access (and had a few hours of mail reading on Sunday evening). On Monday morning, the trend on the RC-Graph didn't looked too good , so I invested resolving RC bugs. A couple of NMUs later the graph now looks better - but of course, there is much more that needs to be done. Beside fixing packages, I did the "usual" small jobs, including reviewing cyrus-sasl2, answering lots of mails, and so on.

Monday morning started with setting up two more graphical stats about the number of Release Critical bugs. They contain how many RC bugs affect etch currently, how many affect only etch (i.e. usually untransitioned fixes), how many affect both suites, and how many affect both suites after 20 days. Both graphs are available for the last 3 weeks and 3 months, respectively at http://bts.turmzimmer.net/graph-small.png and http://bts.turmzimmer.net/graph-large.png. As you can see with the large graph, we did some important progress. As the small graphs show, we should try to get better again. :) So, let's work together on that, and let all three curves go down again. After that was done, some normal RC bug squashing started - nothing special about, just "another few bugs gone".

There are a couple of bugs around where it would be a shame to remove the package (or where it is next to impossible), which are hard to fix for outsiders but should be way easier for people inside:

• #397623: During build of gimp-help, xsltproc tries to download the DTD from the web address. Anyone knowing how to prevent that could probably fix the bug within minutes.
• #397636: open-iscsi: This seems to need just a new upstream version, but nothing for somebody who doesn't use open-iscsi himself. Update: Martin (zobel) will take care if the maintainer doesn't fix this bug.
• #397655: debiandoc-sgml has issues generating cjk-packages correctly.

There are a couple of more such bugs available, please feel free to visit our RC bugs list yourself.

KPilot has a long-standing RC bug report about problems with syncing events and addressbook. There are now some test packages available at http://mirror.pusling.com/kdepim/kdepim_4:3.5.5.dfsg.1-2~sune1/ that should work, so please give it a try. (Writing about this because pusling doesn't blog.)

On Thursday, the release update we have been busy preparing was finally sent out by Steve. This has clarified some of the questions people have been asking for some time. Also, more comments on the release notes have been merged. We are now at an state where I'm happy to let users work with the release notes - though one large section still needs work: What is the appropriate order for upgrades of kernel and packages. After that was done, I started myself with some upgrade tests to see how the packages really behave. More on that on one of the next days ...

Another day spent mostly on documentation. It seems incredible how much time it takes to get things right, but still - it is an very important task, as our users rely on good documentation. The release notes are now mostly ready, except for some more information on the kernel/udev/... (and thanks for Dann for so much input on the kernel situation). Thursday will be spent on merging some comments on the release notes which I got in the late evening, and trying out some upgrade scenarios.

After a day off yesterday I spent some more day on release today. Some was to work on our proposed mail to debian-devel-announce (hopefully, we can finally send it out today). More time was spent on updating the release notes, and work on the submitted requests what should be included. Now the only requests still open are connected with the kernel/udev (and I asked the kernel list for some more input), security situation with php and mozilla, related to sarge (which probably translates into a "wontfix"), and a few minor issues. That's good news because we want to have release notes available now as we have an Release Candidate of the installer published. And as always, some time is spent on the thousands minor little things that are so time-expansive. But still need to be done.

Experimental is autobuild, but on an best-effort basis (which is a different thing than what happens with unstable). In other words, it can happen that a package is not build on all architectures, and nobody does anything to fix that. If that happens to one of your packages, please contact us (Martin Zobel or me), and we'll try to fix that. I just have rescheduled glibc in experimental on two architectures.

Today, I continued chasing easy apache2 bugs. The following bugs seem to be easy to fix (and should be fixed in my opinion prior to Etch).

• #396782: apache2.2-common: Upgrade adds "default" virtual host; breaks existing config: the problem is that apache2.2-common is new in any case, as we had apache2-common in sarge. One possible fix would be to detect if any site is active in /etc/apache2/sites-enabled (e.g. via find) and add the default site only if none is there at all.
• #396265: apache2: mod_proxy_ajp connection reuse bug: This problem is perhaps not specially easy, but - there is a patch upstream, and it seems to have some security impact as well.
• #395936: Apache2 SSL service stopped working since upgrade to 2.2.3-2: -D SSL went amiss when apache2 and apache2-ssl were merged - in case we can't fix that with reasonable effort, it should be documented in the etch release notes
• #391813: apache2.2-common: man page for apache2 missing in apache2.2-common: obvious fix
• #338472: apache2: Move /server-info and /server-status conf to mods-available: obvious fix, but that's a policy decision

Some more bugs were obviously outdated, so I closed them. I also started collecting information for the release notes about apache2.

On Friday, I spent most of my time on apache2. I originally didn't intend to spend so much time there, but as apache2 is an important package, and we still have lots of bug reports flowing in, I decided that spending a bit more time than the minimum effort is useful. I commited an NMU to fix a first row of 8 bug reports (which boiled down to 3 changes), but I expect to work more on apache during the weekend and on Monday.

On Thursday, I did lots of "small" items that needed be done done for quite some time: I checked the new hosting location for volatile (thanks to the physics department of the Ludwig-Maximilians-Universität München for sponsoring), (hopefully) fixed the wanna-build setup for debian-edu (and are now hung at uploading), and so on. Also, I had a short look at the apache2-bugs count, and noticed this package needs serious QA activities. All "good" activities, but obviously not really "working on the release", so I mark Thursday as day off, and will add another day at the end of my schedule.