Fedora 17 will be shipping next week. It's got a bunch of new features, none of which I contributed to in the slightest. What I did work on was improving our support for installation on x86 Apple hardware. There's still a few shortcomings in this so it's not an announced or supported feature, but it's sufficient progress that it's worth writing about.

There's a few ways that Apple platforms differs from normal PC hardware. The first is that Apples are very much intended to be EFI first and BIOS second, while that's still not really true for commodity PC hardware. Apple launched Boot Camp shortly after they started shipping x86 Macs (and shortly after I'd got Ubuntu running on one for the first time[1]) but the focus of Boot Camp has always been to be good enough to boot Windows and not much else[2]. The other is the integration with the boot environment. You can boot Linux easily enough by setting the standard EFI boot variables, but if you ever boot back into OS X it's easy to trigger deletion of those. There's then no straightforward way of resetting them, and you're left having to recover your installation.

The other difficult bit has been actually starting the installer at all. If you're happy to use BIOS emulation than this can be done without too much misery, but you're then left with dealing with Apple's custom synchronised GPT/MBR. More modern Macs will boot via the standard EFI mechanisms, but there's a range of problems that can be caused that way. Fedora 17 is the first Linux distribution to ship install media that will EFI boot a Mac when either written to optical media or a USB stick[3]. Put the install media in your system and then either select it in the OS X Startup Disk preferences menu and reboot, or reboot and hold down the alt key. In the latter case there'll be a nice Fedora logo. Click on it and it'll boot.

The biggest difference between Apple hardware and anyone else is that we'd normally put the bootloader in a FAT partition that's shared with any other installed operating systems. That does actually work on Apples, but then you run into the earlier point I mentioned. The Apple startup picker that you get by holding down the alt key doesn't pay any attention to the standard EFI boot variables, so it won't show a FAT partition merely because it's got an EFI bootloader on it. The same is true of the OS X Startup Disk preferences. The best way to get a drive to appear in the menu is to make it look like an OS X drive.

This was problematic in a few ways, due to the requirement for an OS X drive to be HFS+. The first was that we didn't have any support for that in our installer, so work had to be done there. The second was that the state of HFS+ was woefully poor in Linux[4]. Linux will refuse to write to an HFS+ filesystem if it hasn't been cleanly unmounted or fscked, and since we can't rely on everyone always shutting their machine down cleanly that means a working fsck.hfsplus. In theory we were shipping one of those. In practice, it reliably segfaulted on 64-bit systems[5]. So I had to package the latest version of Apple's code, and doing that involved dealing with the fact that Apple now use blocks almost as much as Grub 2 uses nested functions, and that meant using Clang and that meant dealing with a bug where Clang segfaulted if it had been built with gcc 4.7[6], but finally once all of those yaks had been shaven we could trust our filesystem.

Of course, more problems existed. You can't just drop a bootloader onto an HFS+ partition and expect it to work. You need to write its inode value into the superblock. Trivial, except that if you do this from userspace then the kernel kindly overwrites your update when you unmount the disk and it updates the superblock. That required a mechanism for updating that data via the kernel, which meant adding an ioctl. This would have been fine, except the OS X Startup Disk preference looks for a bootloader in a location other than where Fedora installs it. Changing our bootloader install path wasn't a great option, so the easiest thing to do was just to link them together. Except HFS+ doesn't really support hardlinks. When you hardlink something the original file gets moved into a magic directory in the filesystem root and the links are special files that refer to it. And it turns out that it's not actually the inode that the firmware wants when finding the bootloader, it's the catalogue ID, and these aren't the same when hardlinks are involved. So, another kernel patch.

All that was left for the boot partition then was adding an icon and a drive label. We already had code for generating the icon, and the drive label wasn't that difficult. Hurrah!

Ok, so we can construct a filesystem that (a) appears in the bootpicker, and (b) appears in the OS X Startup Disk preferences. We can even put a bootloader on it. This led to the next problem. The bootloader started without any trouble. But the bootloader couldn't read any files off the boot partition, including its configuration. This should have been obvious with hindsight - the problem was simply that grub legacy[7] doesn't support HFS+. Writing an HFS+ driver seemed like an excessive amount of work, especially since the firmware already supported reading HFS+, so instead I wrote a simple grub filesystem driver that uses the UEFI interfaces to read files.

A working bootloader! One more problem there. grub looks in its startup directory to find its configuration file. We have two "copies" of grub hardlinked to each other, each looking in a different directory for configuration. Hardlinking the config files together worked fine, except that most tools that update config files have this irritating habit of writing to a temporary file and moving that over the original, thus breaking the hardlink. Easily solved by using a symlink instead, except that the UEFI filesystem semantics don't really support symlinks because UEFI only really envisaged using FAT. If you use Apple's UEFI HFS+ driver to open a symlink, you get a short file containing the target path of the symlink. Rather less than ideal, and a rather gross hack to cope with it.

All set now, other than a bug in some older Mac firmware where read would return BUFFER_TOO_SMALL without returning the real buffersize and the odd way that Apple treat CDs. Oh, and a bug in Apple's Broadcom wifi driver that could overwrite the kernel, and case-sensitivity, something that obviously isn't something you often hit on UEFI when all you usually see is FAT.

Kernel-side, we had a couple of things. Obviously there the bugs I'd mentioned in the past, but those had already been dealt with. New issues included the fact that we were frequently picking the wrong framebuffer address, especially on machines with multiple GPUs. Fixed now, along with another issue where we'd sometimes get confused by models with different GPU configurations. And at that point, most of the implementation work was done.

Of course, there was also the work of integrating this into the tools that we use to generate our install media, and I'd like to thank Will Woods, Brian Lane and Dennis Gilmore for the work they put into that, along with anyone I've forgotten. Tom Calloway handled getting the device label graphics generated right before deadline. A cast of thousands helped with testing.

So why did I mention shortcomings? First, this only works on machines with 64-bit firmware. Early 64-bit Apples still had 32-bit firmware. In theory we can support that case - in practice it's a moderate amount of writing, including some mildly awkward kernel code. But anything later than mid-2007 should have 64-bit firmware, so it's not a massive concern. Second, we seem to have fairly significant trouble with Radeon hardware on a lot of Macs. We fixed one bug there, but something's still going wrong in setup and you'll frequently end up with a black screen. And thirdly, I'm sure there's some other machines that still don't work for (as yet) undetermined reasons.

With luck we'll get these things dealt with by Fedora 18. But even with these flaws, Fedora 17 will work fine on a lot of Apple hardware, and testing it is as simple as downloading and booting a live image. Bugs go in the usual place.

[1] Note my charming belief that the Ubuntu installer would fully support this hardware via EFI in the near future. 6 years ago. It still doesn't.
[2] This can be fairly easily seen from little things like the name "Windows" being hardcoded into every UI that sees a Boot Camp drive.
[3] I cover this in more depth here
[4] To be fair, it mostly still is - there's plenty of work to be done there.
[5] Approximately nobody had noticed this, which is all kinds of unreassuring.
[6] Thanks to Kalev Lember for dealing with that
[7] Still the EFI bootloader in Fedora 17. We've already swapped Fedora 18 over to grub 2.

comments

The Gnome shell is a great step. Finally the Linux default Desktop is not only useful (as Gnome 2.X used to be), but also stylish and modern. It's a different kind of desktop, which seems to irk desktop habits of some peoples. I'm proud to show it off. This happened for the last time years ago while doing some funny config quirks of fvwm. Gnome 3.x is cool. (: Also the customisability via all the extensions is fun. What I do miss is the gnome weather (There used to be an incarnation with plenty of geographical sites, which got somehow got axed later.) and more packaged extensions for out of the box deployments (wish #661782). I seem to be too lunatic to find session settings in "System Settings" for having startup applications run or the used applications reopened (please yes evince show the same pdfs). The other minor critic is that "System Settings" is quite empty, but has on each system a funny advertisement for "Wacom Tablets". Ah and after a myriad of suspends and network-manager stops/restarts gnome-shell likes to segfault, which is no nuisance as it restarts.

The other day, I was a bit puzzled over a seemingly simple task: Enable core files to be generated from a PostgreSQL instance running on Debian. That instance has unfortunately been segfaulting on occasion, but never left a core file.

Now in principle it is clear that

ulimit -c unlimited

is the incantation to get this done. But where do you put this? You could hack it into the init script, but that seemed a bit ugly, and I wanted a sustainable solution.

A useful thing in the meantime is to check the current settings. That information is available in /proc/$PID/limits with the PID of the postmaster process (or any child process, really), and it looked like this to begin with:

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            ms        
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
...

Use sudo grep core /proc/$(sudo cat /var/run/postgresql/8.4-main.pid)/limits if you want it automated.

So it's good to know that we only need to set the soft limit.

One way to configure this properly would appear to be in /etc/security/limits.conf. There you can add a line like

*               soft    core            unlimited

to enable core dumps globally. I'm not actually sure whether that would work if the service is started during the boot without PAM. In any case, I didn't want to enable core files globally; who knows what that would lead to.

One could replace the * by a user name, such as postgres, and then enable pam_limits.so in /etc/pam.d/su. But the postgresql init script in Debian is nested about four levels deep, so it wasn't clear whether it called su at all.

Now as it turns out, the init script ends up changing the user using this Perl code:

$) = $groups;
$( = $gid;
$> = $< = $uid;

(see change_ugid in /usr/share/postgresql-common/PgCommon.pm), so the whole PAM line of thought wasn't going to work anyway. (Other packages such as pgbouncer and slony1 do got through su, so that would be a solution for those.)

The way to solve this is the pg_ctl -c option, which sets the soft limit for core files to unlimited. And the way to pass this option through the init script maze is the file /etc/postgresql/8.4/main/pg_ctl.conf, which should contain a line like this:

pg_ctl_options = '-c'

Then restart postgresql, and check /proc/$PID/limits again:

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            ms        
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     

OK.

Another thing that's recommendable in this context is to change the core file names to have a unique element, so that if multiple backends crash before you can take a look, they don't overwrite each other's core files. The core(7) man page explains the configuration options; I went with this sysctl setting:

kernel.core_pattern = core.%e.%p

which includes process name and PID. The PID file still ends up in the data directory of the PostgreSQL instance, which could also be changed, but I didn't find it necessary.

Stick the above line in /etc/sysctl.d/local.conf and reload with

service procps force-reload

I actually use a setting like that on all machines now; it's just nicer.

OK, and now I'll wait for the next core file. Or not.

When we added s390x to the main archive, coming from Debian Ports, we were unlucky. A new glib release had assumptions that weren't true on 64bit big endian architectures and it entered the archive just a few days before we made the initial import. This weekend we finally got a new major release into Debian unstable that fixed these issues. So we're almost on par with s390 now. It all untangled quite nicely after glib-networking was able to complete its testsuite. Only one build-dependency loop between nautilus and tracker had to be broken manually.

So what's left? There's a bunch of usertagged bugs (with both general FTBFSes and arch-specific issues; kudos to Aur lien Jarno providing a lot of patches) and we still need to file some, like iceweasel segfaulting during its build. That's important because another bunch of packages needs it to build (well, mozjs and/or xulrunner, or some package that needs those).

Some of my recent time has been devoted to making our boot media more Mac friendly, which has entailed rather a lot of rebooting. This would have been fine, if tedious, except that some number of boots would fall over with either a clearly impossible kernel panic or userspace segfaulting in places that made no sense. Something was clearly wrong. Crashes that shouldn't happen are generally an indication of memory corruption. The question is how that corruption is being triggered. Hunting that down wasn't terribly easy.

My first thought was that we were possibly managing to load the kernel over a region used by UEFI code. UEFI defines two types of code - boot services and runtime services. While runtime services code and data must be preserved by the OS, in theory boot services code and data is available to the OS once the firmware has exited. In practice, that's not true. It seemed entirely possible that the kernel might be ending up on top of some of that boot services code or data and getting trodden on. Grub now has code to avoid putting the kernel on boot services, so testing the latest code seemed like a good plan. But no, crashes still happened.

That pretty much ruled out the bootloader. My next thought was that executing some of the firmware code was triggering a write to some other memory that contained the kernel. Josh Boyer suggested the next trick, which was to try marking the kernel read-only to see whether anything was hitting it. x86 lets you mark pages as read-only - any attempt to write to them should take a fault. UEFI functions are executed in the context of the kernel, so share the same page tables. That let me rule this out, since everything still went just as wrong and I wasn't taking an extra fault first.

However, at this point I was reasonably happy that it wasn't the kernel itself being overwritten - faults were occurring in userspace code as well. That was a pretty strong indication that what was happening was continuing to happen once userspace had started, so it wasn't a direct response to a firmware call. I made sure of that by stubbing out all the calls that could be triggered after kernel initialisation, and saw the same failures. Once all attempts to be clever have failed, it's time to just start using brute force. The kernel lets you reserve areas of RAM by passing arguments like memmap=0xlength$0xstart to block length bytes starting at start from being used. It took a while, but I finally found a 256MB range that made a difference - reserving it resulted in the machine booting reliably, letting the OS use it resulted in occasional crashes.

Definite progress. Comparing that memory range to the EFI memory map was helpful. There were several blocks of UEFI boot services data present there, which really seemed like too much of a coincidence. By reserving each of them in turn, I'd traced it down to a single 31MB region of boot service data - that is, memory reserved by the firmware for use by the UEFI boot services. Per spec, this is available to the OS once the boot environment has been exited. Nothing other than the OS should be touching this after boot, but something clearly was. Tracking down what was far easier than I expected, although the first attempt was a failure. Setting it read-only should have triggered a fault, but didn't. That was rather confusing. But, rather than give up, I patched the kernel to fill the region with 0xff at kernel init. Then I booted the system, read it back and looked for values that weren't 0xff. I got this:

00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
001568a0  ff ff ff ff ff ff ff ff  ff ff ff ff 84 00 00 00   ................ 
001568b0  00 20 a7 ac 46 00 00 00  00 00 00 00 00 00 06 01   . ..F........... 
001568c0  c2 0b 0c 00 ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
001568d0  ff ff 0a 04 f0 03 82 0d  40 00 00 00 ff ff ff ff   ........@....... 
001568e0  ff ff 00 21 00 36 9a 80  ff ff ff ff ff ff 00 7e   ...!.6.........~ 
001568f0  00 09 43 48 41 2d 47 75  65 73 74 01 04 02 04 0b   ..CHA-Guest..... 
00156900  16 32 08 0c 12 18 24 30  48 60 6c 2d 1a 0e 18 1a   .2....$0H l-.... 
00156910  ff ff 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156920  00 00 00 00 00 00 00 dd  09 00 10 18 02 00 10 01   ................ 
00156930  00 00 dd 1e 00 90 4c 33  0e 18 1a ff ff 00 00 00   ......L3........ 
00156940  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156950  00 00 bd ea f8 b3 ff ff  ff ff ff ff ff ff ff ff   ................ 
00156960  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
022fb000

That's a lot of 0xffs (around 31MB of them) with one small section that contains an 802.11 probe packet with the SSID of the hospital across the road from my house. Apple support network booting off wireless networks. It seems that the firmware brought up the wireless card, associated with this network (it's the only public one nearby) and then left the card DMAing packets into RAM. The read-only page attribute only applies to CPU-initiated accesses, so it could do this without triggering a page fault. It also explained why it was so random - whether memory corruption occurred would depend on whether a packet appeared between that memory being used by the OS and the kernel reinitialising the wireless card. It certainly explains why I couldn't reproduce it when I left the machine repeatedly rebooting on the bus home.

How do we fix this? Unsure. With luck disconnecting the UEFI driver in the bootloader should quiesce the hardware, but without testing I'm not sure of that yet. For now it's just another example of firmware managing to break expectations in deeply strange ways.

comments

This is half rant, half warning to others. Relative merrits of the IBM ThinkPad X31 vs the Lenovo ThinkPad X220... Old IBM ThinkPad X31 which I abused for years:

• Built like a tank
• 4:3 screen at 1024x768
• Fell down several times
• Got doused in coffee and coke
• Has been invaded (as in "filled to the brim") by Indian ants which ate out all the coffee and coke over about a week of constant occupation (even while the machine was running; I had no choice) years later
• Got hit by a truck's rear view mirror (~40 km/h)
• Got stepped upon repeatedly (not by me)
• Experienced everything from -25 to +35-40 degrees Celsius
• Experienced very high and extremely low humidity
• Survived uncounted kilometers of very rough off-road driving
• Got sprayed with sea water repeatedly
• Prolonged exposure to various kinds of abrasive sand
• I probably forgot other abuse

I upgraded the RAM, swapped the battery once and the HDD twice. If not for a loose power socket, I would still use it. Contrast my new Lenovo ThinkPad X220 after a few months of pampering:

• You can remap the blue button to something useful (nice!)
• Wide-screen 16:9 at 1366 x 768 with no upgrade option (we are talking 5 years of technological advancement being ignored)
• Cardbus slot, but only one mini and one micro PCI slot so Wi-Fi, UMTS, SSD, and spindle disk at the same time are impossible
• Charge LED can only be seen from the outside which is great when my battery starts running low. Other people do care more about my battery status than myself, obviously! Making the LED visible from both sides would cost next to nothing. This is sloppy/bad design and nothing else. I am unable to understand why anyone who ever used a portable computer would do this. Argh!
• Sleep LED can only be seen from outside
• Use of built-in 3G or GPS can totally lock up device and/or make KDE/X quit and go back to login screen
• Random segfaults of programs that cause high network load via Wi-Fi (Chrome is especially queasy)
• Random ssh disconnects when using Wi-Fi
• Random sound artifacts during high network load via Wi-Fi
• All of the above, but less often, when using wired network
• Random segfaults and other magic issues after more than a few hours of uptime and/or warmth
• Three distinct USB ports (1 x USB 3.0, 1 x USB 2.0, 1 x USB 2.0 with power pass-through) which do or do not work correctly depending on uptime and temperature
• Makes weird noises
• On-desk combined power brick and USB hub broken after a few months of sitting on my desktop and being used once every week

Google tells me there are several people with random issues with their X220, especially when the mini-PCI slot is being used in a machine with USB 3.0. Removing the UMTS/GPS module did help, but still, this means no SSD for me as that's the slot I would have to use. Recently, things have been degrading very quickly to the point where I refuse to do even simple things, like write and push blog posts, on this machine. I am not sure if I am simply unlucky and received a dud, if the X220 is a model with issues, or if this is indicative of a general trend within Lenovo, but it's time to contact Lenovo support. If I had known all of this in advance, I would simply have soldered the power socket back into my X31 and held out for the X221. Or something else, entirely.

as expected this week was characterised by fixing bugs around the perl 5.14 transition. besides that I've also uploaded a couple of NMUs with patches kindly provided in the BTS by some tireless bug fixers.

• #554047 src:cellwriter: "FTBFS with binutils-gold"
  add patch from Ubuntu / Mahyuddin Susanto, upload to DELAYED/2
• #628500 src:prima: "prima: FTBFS with perl 5.14: error: 'sv_undef' undeclared"
  patch to change functions that rely on removed PERL_POLLUTE, upload to DELAYED/2 (perl-5.14)
• #629284 src:libpod-constants-perl: "libpod-constants-perl: FTBFS with perl 5.14: Can't use string [...] as an ARRAY ref"
  patch Makefile.PL (older and newer EUMM) (pkg-perl, perl-5.14)
• #636268 src:libdbix-class-timestamp-perl: "libdbix-class-timestamp-perl: FTBFS with perl 5.14: test failures"
  upload new upstream release (pkg-perl, perl-5.14)
• #636271 src:libtest-www-declare-perl: "libtest-www-declare-perl: FTBFS with perl 5.14: tests fail"
  new patch to work with perl 5.14 regexp (pkg-perl, perl-5.14)
• #636517 src:libregexp-optimizer-perl: "libregexp-optimizer-perl: FTBFS with perl 5.14: test failures"
  new patch to work with perl 5.14 regexp (pkg-perl, perl-5.14)
• #636520 src:libtest-html-content-perl: "libtest-html-content-perl: FTBFS with perl 5.14: tests fail"
  send a patch to the BTS, uploaded by the maintainer (perl-5.14)
• #636521 src:libtest-log4perl-perl: "libtest-log4perl-perl: FTBFS with perl 5.14: tests fail"
  new patch to work with perl 5.14 regexp (pkg-perl, perl-5.14)
• #636522 src:libtest-simpleunit-perl: "libtest-simpleunit-perl: FTBFS with perl 5.14: tests fail"
  new patch to work with perl 5.14 regexp (pkg-perl, perl-5.14)
• #643040 src:abcm2ps: "abcm2ps: FTBFS: dpkg-buildpackage: error: dpkg-source -b abcm2ps-5.9.22 gave error exit status 2"
  backup/restore Makefile in debian/rules, upload to DELAYED/2
• #643359 src:blktool: "blktool: FTBFS: util.c:31:3: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643362 src:cellwriter: "cellwriter: FTBFS: src/statusicon.c:215:17: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643369 src:crack-attack: "crack-attack: FTBFS: callbacks.cxx:116:38: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643370 src:d52: "d52: FTBFS: d52pass2.c:935:3: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from peter green, upload to DELAYED/2
• #643374 src:dis51: "dis51: FTBFS: pass2.c:186:4: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643375 src:ebview: "ebview: FTBFS: dialog.c:72:6: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643383 src:gfan: "gfan: FTBFS: application.cpp:548:22: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643384 src:gfccore: "gfccore: FTBFS: error.cc:116:52: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #643413 src:jack-tools: "jack-tools: FTBFS: jack.dl.c:20:3: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from M nica Ram rez Arceda, upload to DELAYED/2
• #643417 src:libccscript3: "libccscript3: FTBFS: compiler.cpp:1147:38: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Ilya Barygin, upload to DELAYED/2
• #643448 src:nvramtool: "nvramtool: FTBFS: hexdump.c:94:7: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from peter green, upload to DELAYED/2
• #643457 src:pidgin-librvp: "pidgin-librvp: FTBFS: rvp.c:4271:3: error: format not a string literal and no format arguments [-Werror=format-security]"
  add patch from Eric Alexander, upload to DELAYED/2
• #646286 src:genders: "genders: FTBFS: crti.o: No such file or directory"
  send a patch to the BTS, uploaded by the maintainer (perl-5.14)
• #646297 src:spread: "spread: FTBFS: Spread.xs:463:4: error: format not a string literal and no format arguments [-Werror=format-security]"
  add format argument, do a QA upload (perl-5.14)
• #648858 src:libdbd-oracle-perl: "libdbd-oracle-perl: needs to be rebuilt against perl 5.14"
  manually rebuild against perl 5.14 (pkg-perl, perl-5.14)
• #649055 libgnome2-gconf-perl: "libgnome2-gconf-perl: FTBFS (Failed 1/1 test programs. 0/0 subtests failed)"
  add patch from Colin Watson (pkg-perl, perl-5.14)
• #649058 libdata-alias-perl: "libdata-alias-perl: FTBFS on mips powerpc s390 sparc"
  add patch from Niko Tyni (pkg-perl, perl-5.14)
• #649180 src:libmouse-perl: "libmouse-perl: swap libextutils-parsexs-perl/perl dep"
  upload fixed package prepared by Alessandro Ghedini (pkg-perl, perl-5.14)

Executive summary Debian should consider:

• less fine-grained software packaging for overall simplification of the system and its development
• less flexibility at the lower levels of the software stack, again for overall simplicity
• reducing friction in the development workflow, especially by making it possible to branch and merge at the whole system level

Introduction I've taken a job with Codethink, as part of a team to develop a new embedded Linux system, called Baserock. Baserock is not a distribution as such, but deals with many of the same issues. I thought it might be interesting to write out my new thoughts related to this, since they might be useful for proper distributions to think about too. I'll hasten to add that many of these thoughts are not originally mine. I shamelessly adopt any idea that I think is good. The core ideas for Baserock come from Rob Taylor, Daniel Silverstone, and Paul Sherwood, my colleagues and bosses at Codethink. At the recent GNOME Summit in Montreal, I was greatly influenced by Colin Walters. This is also not an advertisment for Baserock, but since my of my current thinking come from that project, I'll discuss things in the context of Baserock. Finally, I'm writing this to express my personal opinion and thoughts. I'm not speaking for anyone else, least of all Codethink. On the package abstraction Baserock abandons the idea of packages for all individual programs. In the early 1990s, when Linux was new, and the number of packages in a distribution could be counted on two hands in binary, packages were a great idea. It was feasible to know at least something about every piece of software, and pick the exact set of software to install on a machine. It is still feasible to do so, but only in quite restricted circumstances. For example, picking the packages to install a DNS server, or an NFS server, or a mail server, by hand, without using meta packages or tasks (in Debian terminology), is still quite possible. On embedded devices, there's also usually only a handful of programs installed, and the people doing the install can be expected to understand all of them and decide which ones to install on each specific device. However, those are the exceptions, and they're getting rarer. For most people, manually picking software to install is much too tedious. In Debian, we've realised this a great many years ago, and developed meta packages (whose only purpose is to depend on other packages) and tasks (solves the same problem, but differently). These make it possible for a user to say "I want to have the GNOME desktop environment", and not have to worry about finding every piece that belongs in GNOME, and install that separately. For much of the past decade, computers have had sufficient hard disk space that it is no longer necessary to be quite so picky about what to install. A new cheap laptop will now typically come with at least 250 gigabytes of disk space. An expensive one, with an SSD drive, will have at least 128 gigabytes. A fairly complete desktop install uses less than ten gigabytes, so there's rarely a need to pick and choose between the various components. From a usability point of view, choosing from a list of a dozen or two options is much easier than from a list of thirty five thousand (the number of Debian packages as I write this). This is one reason why Baserock won't be using the traditional package abstraction. Instead, we'll collect programs into larger collections, called strata, which form some kind of logical or functional whole. So there'll be one stratum for the core userland software: init, shell, libc, perhaps a bit more. There'll be another for a development environment, one for a desktop environment, etc. Another, equally important reason, to move beyond packages is the problems caused by the combinatorial explosion of packages and versions. Colin Walters talks about this very well. When every system has a fairly unique set of packages, and versions of them, it becomes much harder to ensure that software works well for everyone, that upgrades work well, and that any problems get solved. When the number of possible package combinations is small, getting the interactions between various packages right is easier, QA has a much easier time to test all upgrade paths, and manual test coverage improves a lot when everyone is testing the same versions. Even debugging gets easier, when everyone can easily run the same versions. Grouping software into bigger collections does reduce flexibility of what gets installed. In some cases this is important: very constrainted embedded devices, for example, still need to be very picky about what software gets installed. However, even for them, the price of flash storage is low enough that it might not matter too much, anymore. The benefit of a simpler overall system may well outweigh the benefit of fine-grained software choice. Everything in git In Baserock, we'll be building everything from source in git. It will not be possible to build anything, unless the source is committed. This will allow us to track, for each binary blob we produce, the precise sources that were used to bulid it. We will also try to achieve something a bit more ambitious: anything that affects any bit in the final system image can be traced to files committed to git. This means tracking also all configuration settings for the build, and the whole build environment, in git. This is important for us so that we can reproduce an image used in the field. When a customer is deploying a specific image, and needs it to be changed, we want to be able to make the change with the minimal changes compared to the previous version of the image. This requires that we can re-create the original image, from source, bit for bit, so that when we make the actual change, only the changes we need to make affect the image. We will make it easy to branch and merge not just individual projects, but the whole system. This will make it easy to do large changes to the system, such as transitioning to a new version of GNOME, or the toolchain. Currently, in Debian, such large changes need to be serialised, so that they do not affect each other. It is easy, for example, for a GNOME transition to be broken by a toolchain transition. Branching and merging has long been considered the best available solution for concurrent development within a project. With Baserock, we want to have that for the whole system. Our build servers will be able to build images for each branch, without requiring massive hardware investment: any software that is shared between branches only gets built once. Launchpad PPAs and similar solutions provide many of the benefits of branching and merging on the system level. However, they're much more work than "git checkout -b gnome-3.4-transition". I believe that the git based approach will make concurrent development much more efficient. Ask me in a year if I was right. Git, git, and only git There are a lot of version control systems in active use. For the sake of simplicity, we'll use only git. When an upstream project uses something else, we'll import their code into git. Luckily, there are tools for that. The import and updates to it will be fully automatic, of course. Git is not my favorite version control system, but it's clearly the winner. Everything else will eventually fade away into obscurity. Or that's what we think. If it turns out that we're wrong about that, we'll switch to something else. However, we do not intend to have to deal with more than one at a time. Life's too short to use all possible tools at the same time. Tracking upstreams closely We will track upstream version control repositories, and we will have an automatic mechanism of building our binaries directly from git. This will, we hope, make it easy to follow closely upstream development, so that when, say, GNOME developers make commits, we want to be able to generate a new system image which includes those changes the same day, if not within minutes, rather than waiting days or weeks or months. This kind of closeness is greatly enhanced by having everything in version control. When upstream commits changes to their version control system, we'll mirror them automatically, and this then triggers a new system image build. When upstream makes changes that do not work, we can easily create a branch from any earlier commit, and build images off that branch. This will, we hope, also make it simpler to make changes, and give them back to upstream. Whenever we change anything, it'll be done in a branch, and we'll have system images available to test the change. So not only will upstream be able to easily get the change from our git repository, they'll also be easily verify, on a running system, that the change fixes the problem. Automatic testing, maybe even test driven development We will be automatically building system images from git commits for Baserock. This will potentially result in a very large number of images. We can't possibly test all of them manually, so we will implement some kind of automatic testing. The details of that are still under discussion. I hope to be able to start adding some test driven development to Baserock systems. In other words, when we are requested to make changes to the system, I want the specification to be provided as executable tests. This will probably be impossible in real life, but I can hope. I've talked about doing the same thing for Debian, but it's much harder to push through such changes in an established, large project. Solving the install, upgrade, and downgrade problems All mainstream Linux distributions are based on packages, and they all, pretty much, do installations and upgrades by unpacking packages onto a running system, and then maybe running some scripts from the packages. This works well for a completely idle system, but not so well on systems that are in active use. Colin Walters again talks about this. For installation of new software, the problem is that someone or something may invoke it before it is fully configured by the package's maintainer script. For example, a web application might unpack in such a way that the web server notices it, and a web browser may request the web app to run before it is configured to connect to the right database. Or a GUI program may unpack a .desktop file before the executable or its data files are unpacked, and a user may notice the program in their menu and start it, resulting in an error. Upgrades suffer from additonal problems. Software that gets upgraded may be running during the upgrade. Should the package manager replace the software's data files with new versions, which may be in a format that the old program does not understand? Or install new plugins that will cause the old version of the program to segfault? If the package manager does that, users may experience turbulence without having put on their seat belts. If it doesn't do that, it can't install the package, or it needs to wait, perhaps for a very long time, for a safe time to do the upgrade. These problems have usually been either ignored, or solved by using package specific hacks. For example, plugins might be stored in a directory that embeds the program's version number, ensuring that the old version won't see the new plugins. Some people would like to apply installs and upgrades only at shutdown or bootup, but that has other problems. None of the hacks solve the downgrade problem. The package managers can replace a package with an older version, and often this works well. However, in many cases, any package maintainer scripts won't be able to deal with downgrades. For example, they might convert data files to a new format or name or location upon upgrades, but won't try to undo that if the package gets downgraded. Given the combinatorial explosion of package versions, it's perhaps just as well that they don't try. For Baserock, we absolutely need to have downgrades. We need to be able to go back to a previous version of the system if an upgrade fails. Traditionally, this has been done by providing a "factory reset", where the current version of the system gets replaced with whatever version was installed in the factory. We want that, but we also want to be able to choose other versions, not just the factory one. If a device is running version X, and upgrades to X+1, but that version turns out to be a dud, we want to be able to go back to X, rather than all the way back to the factory version. The approach we'll be taking with Baserock relies on btrfs and subvolumes and snapshots. Each version of the system will be installed in a separate subvolume, which gets cloned from the previous version, using copy-on-write to conserve space. We'll make the bootloader be able to choose a version of the system to boot, and (waving hands here) add some logic to be able to automatically revert to the previous version when necessary. We expect this to work better and more reliably than the current package based one. Making choices Debian is pretty bad at making choices. Almost always, when faced with a need to choose between alternative solutions for the same problem, we choose all of them. For example, we support pretty much every init implementation, various implementations of /bin/sh, and we even have at least three entirely different kernels. Sometimes this is non-choice is a good thing. Our users may need features that only one of the kernels support, for example. And we certainly need to be able to provide both mysql and postresql, since various software we want to provide to our uses needs one and won't work with the other. At other times, the inability to choose causes trouble. Do we really need to support more than one implemenation of /bin/sh? By supporting both dash and bash for that, we double the load on testing and QA, and introduce yet another variable to deal with into any debugging situation involving shell scripts. Especially for core components of the system, it makes sense to limit the flexibility of users to pick and choose. Combinatorial explosion d j vu. Every binary choice doubles the number of possible combinations that need to be tested and supported and checked during debugging. Flexibility begets complexity, complexity begets problems. This is less of a problem at upper levels of the software stack. At the very top level, it doesn't really matter if there are many choices. If a user can freely choose between vi and Emacs, and this does not add complexity at the system level, since nothing else is affected by that choice. However, if we were to add a choice between glibc, eglibc, and uClibc for the system C library, then everything else in the system needs to be tested three times rather than once. Reducing the friction coefficient for system development Currently, a Debian developer takes upstream code, adds packaging, perhaps adds some patches (using one of several methods), builds a binary package, tests it, uploads it, and waits for the build daemons and the package archive and user-testers to report any problems. That's quite a number of steps to go through for the simple act of adding a new program to Debian, or updating it to a new version. Some of it can be automated, but there's still hoops to jump through. Friction does not prevent you from getting stuff done, but the more friction there is, the more energy you have to spend to get it done. Friction slows down the crucial hack-build-test cycle of software development, and that hurts productivity a lot. Every time a developer has to jump through any hoops, or wait for anything, he slows down. It is, of course, not just a matter of the number of steps. Debian requires a source package to be uploaded with the binary package. Many, if not most, packages in Debian are maintained using version control systems. Having to generate a source package and then wait for it to be uploaded is unnecessary work. The build daemon could get the source from version control directly. With signed commits, this is as safe as uploading a tarball. The above examples are specific to maintaining a single package. The friction that really hurts Debian is the friction of making large-scale changes, or changes that affect many packages. I've already mention the difficulty of making large transitions above. Another case is making policy changes, and then implementing them. An excellent example of that is in Debian is the policy change to use /usr/share/doc for documentation, instead of /usr/doc. This took us many years to do. We are, I think, perhaps a little better at such things now, but even so, it is something that should not take more than a few days to implement, rather than half a decade. On the future of distributions Occasionally, people say things like "distributions are not needed", or that "distributions are an unnecessary buffer between upstream developers and users". Some even claim that there should only be one distribution. I disagree. A common view of a Linux distribution is that it takes some source provided by upstream, compiles that, adds an installer, and gives all of that to the users. This view is too simplistic. The important part of developing a distribution is choosing the upstream projects and their versions wisely, and then integrating them into a whole system that works well. The integration part is particularly important. Many upstreams are not even aware of each other, nor should they need to be, even if their software may need to interact with each other. For example, not every developer of HTTP servers should need to be aware of every web application, or vice versa. (It they had to be, it'd be a combinatorial explosion that'd ruin everything, again.) Instead, someone needs to set a policy of how web apps and web servers interface, what their common interface is, and what files should be put where, for web apps to work out of the box, with minimal fuss for the users. That's part of the integration work that goes into a Linux distribution. For Debian, such decisions are recorded in the Policy Manual and its various sub-policies. Further, distributions provide quality assurance, particularly at the system level. It's not realistic to expect most upstream projects to do that. It's a whole different skillset and approach that is needed to develop a system, rather than just a single component. Distributions also provide user support, security support, longer term support than many upstreams, and port software to a much wider range of architectures and platforms than most upstreams actively care about, have access to, or even know about. In some cases, these are things that can and should be done in collaboration with upstreams; if nothing else, portability fixes should be given back to upstreams. So I do think distributions have a bright future, but the way they're working will need to change.

A month ago, I published an article on the compared performances of stunnel, nginx and stud as SSL terminators. The conclusion was to use stud on a 64-bit system, with session caching and AES. stunnel was unable to scale properly and nginx exhibited important latency issues. I got constructive comments on many aspects. Therefore, here is the second round. The protagonists are the same but both the conditions and the conclusions have changed a bit.

Benchmarks The hardware platform remains unchanged:

• HP DL 380 G7
• 6 GB RAM
• 2 L5630 @ 2.13GHz (8 cores), hyperthreading disabled
• 2 Intel 82576 NIC
• Spirent Avalanche 2900 appliance

We focus on the following environment:

• Ubuntu Lucid, featuring a patched version of OpenSSL 0.9.8k with AES-NI builtin and enabled by default
• 2.6.39 kernel (250 HZ)
• 64-bit
• 1 core for HAProxy, 2 cores for network cards¹, 4 cores for SSL, 1 core for the system
• system limit of 100,000 file descriptors per process

On the SSL side, the main components are:

• 2048-bit certificates
• AES128-SHA cipher suite
• 8 HTTP/1.0 requests per client
• HTTP response payload of 1024 bytes

If you are interested in different conditions (32-bit, number of cores, 1024-bit certificates, proportion of session reuses), have a look at the previous article whose results on the matter are still valid. I only consider 2048-bit certificates because most certificates issued today are 2048-bit. In SP800-57, NIST advises that 1024-bit RSA keys will no longer be viable after 2010 and advises moving to 2048-bit RSA keys:

If information is initially signed in 2009 and needs to remain secure for a maximum of ten years (i.e., from 2009 to 2019), a 1024-bit RSA key would not provide sufficient protection between 2011 and 2019 and, therefore, it is not recommended that 1024-bit RSA be used in this case.

And the protagonists are:

• stud, @e207dbc5a3, with session cache enabled for 20,000 sessions, listen backlog of 1000 connections,
• stunnel 4.44, configured with --disable-libwrap --with-threads=pthread (configuration),
• nginx 1.1.4, configured with --with-http_ssl_module (configuration).

UPDATED: Configuration of nginx has been updated to switch to off both multi_accept and accept_mutex thanks to the feedback of Maxim Dounin on latency issues that were present in the previous article (and still present in the early version of this article). HAProxy (configuration) is used for stunnel and stud to add load-balancing. For each test, we want to maximize the number of HTTP transactions per second (TPS) while minimizing latency and unsucessful transactions. The maximum number of allowed failed transactions is set at 1 transaction out of 1000. The maximum allowed average latency is 100 ms².

Control case Below is the first comparison between stud, nginx and stunnel on plain Ubuntu Lucid. Like in previous benchs, stunnel is not able to scale, with a maximum of 400 TPS. However, early latency issues of nginx has been solved, thanks to the change in configuration. stud and nginx gets similar performance but, as you can see on the bottom plot (whose scale is logarithmic) stud starts with an initial latency of 40 ms. This is pretty annoying. I have tried some micro optimization to solve this issue but without luck. If latency matters for you, stick with stunnel or nginx (or find where the problem lies in stud). UPDATED: After reporting this issue, a fix has been commited. The culprit was Nagle s algorithm which improves bandwidth at the expense of latency. With the fix, performances are mostly the same but there is no latency penalty any more.

OpenSSL 1.0.0e When I published the first round of benchmarks, Micha Trojnara was dubious about the bad results of stunnel. He told me on Twitter:

I solved it! You compiled stunnel against a 4 years old OpenSSL with MT-unsafe SSL_accept. Upgrade OpenSSL and rebuild stunnel. This bug was fixed in OpenSSL 1.0.0b. I implemented a (slow) workaround, so stunnel doesn t crash with earlier versions of OpenSSL.

Therefore, I backported Ubuntu Oneiric OpenSSL 1.0.0e to Ubuntu Lucid³ and, as you can see yourself on the plot below, there is a huge performance boost for stunnel: about 400%! stud features a slight improvement too (about 8%) but still starts with a latency of 40 ms while nginx does not improve.

Intel Accelerator Engine Most Linux distributions are still using OpenSSL 0.9.8. For example, Debian Squeeze ships with OpenSSL 0.9.8o, Ubuntu Lucid with 0.9.8k and RHEL 5.7 with 0.9.8e. Therefore, they are not eligible to recent performance improvements. To solve this, Intel released Intel Accelerator Engine, which also brings, among other things, AES-NI support:

This is essentially a collection of assembler modules from development OpenSSL branch, packed together as a standalone engine. Standalone means that it can be compiled outside OpenSSL source tree without patching the latter. Idea is to provide a way to utilize new code in already released OpenSSL versions constrained by support and release policies limiting their evolvement to genuine bug fixes.

Alas, in my tests, there was no improvement compared to OpenSSL 1.0.0e (already patched with AES-NI support). stunnel comes with out-of-the-box support for additional engines. Just add something like this to your stunnel.conf:

engine=dynamic
engineCtrl=SO_PATH:/WOO/intel-accel-1.4/libintel-accel.so
engineCtrl=ID:intel-accel
engineCtrl=LIST_ADD:1
engineCtrl=LOAD
engineCtrl=INIT

For stud, you should grab support for engine selection from GitHub and for nginx, just specify the engine with ssl_engine.

Intel Performance Primitives In his great article Accelerated SSL , Simon Kuhn shows huge performance improvements by using Intel s IPP library, a library aimed at leveraging power of multi-core processors and including some cryptographic primitives and a patch to make use of them in OpenSSL. Unfortunately, this library is not free, not even as in free beer . The patch provided by Intel only applies on OpenSSL 0.9.8j. Simon updated it for OpenSSL 1.0.0d. It also applies with a slight modification to OpenSSL 1.0.0e. Look at Simon s article for instructions on how to compile OpenSSL with IPP. Simon measured 220% improvement on RSA 1024-bit private key operations by using this library. Unfortunately, we don t get such a performance burst with stud which exhibits a 15% improvement (but remember that private key operations only happen once every 8 transactions thanks to session reuse). Worst, with stunnel, performances are reduced by one third and nginx just segfaults. I think such a library is risky to use in production.

Hyperthreading During the previous tests, only half the capacity of each core were used. For stud, adding more workers per core only slightly improves this situation (less than 10% with 4 workers per core). Therefore, I thought using hyperthreading could rectify the situation here. In this case, stud, stunnel and nginx are allocated 8 virtual cores instead of 4. stud only got a small improvement of 5%, nginx a moderate improvement of 11% while stunnel got a significant improvement of 26%.

Conclusion Here is a quick recap of the maximum TPS reached on each tests (while keeping average response time below 100 ms). All tests were run on 4 physical cores with 2048-bit certificates, AES128-SHA cipher suite. Each client makes 8 HTTP requests and reuse the first SSL session.

Context	nginx 1.1.4	stunnel 4.44	stud @e207dbc5a3
OpenSSL 0.9.8k	1489 TPS	400 TPS	1492 TPS
OpenSSL 1.0.0e	1501 TPS	1561 TPS	1609 TPS
OpenSSL 1.0.0e + IPP	-	1101 TPS	1857 TPS
OpenSSL 1.0.0e + HT	1718 TPS	1973 TPS	1694 TPS

I also would like to stress that below 1500 TPS, stunnel adds an overhead of a few milliseconds while stud adds 40 milliseconds. With OpenSSL 1.0.0e and hyperthreading enabled, stunnel has both good performance and low latency on low loads to moderate loads. You may just want to limit the number of allowed connections to avoid collapsing on high loads. While in the previous round, stud was the champion and stunnel lagged behind, the situation is now reversed : stunnel is on top! UPDATED: As said earlier, the latency issue with stud has been solved. Performances remain mostly the same.

---

1. HAProxy and network cards don t need dedicated core for this kind of load. However, if you also want to handle non SSL traffic, those dedicated cores can get busy.
2. The real condition is more complex. Each point is considered valid if the corresponding average latency is less than 100 ms or if the average on the next 8 points is less than 100 ms. Temporary latency bursts are therefore allowed. We are not so forgiveful with failed transactions.
3. This is pretty straighforward: I just dropped multiarch support. I chose to use Ubuntu version of OpenSSL instead of upstream because there are some interesting patches, like automatic support of AES-NI.

I just uploaded a snapshot of GNU Screen to Debian Experimental. The package (4.1.0~20110819git450e8f3-1) is based on upstream s HEAD whose most recent commit currently dates to the 19th of August 2011. While the upload fixes tons of bugs which accumulated over the past two years in Debian s, Ubuntu s and upstream s bug tracker, I don t yet regard it as suitable for the next stable release (and hence for Debian Unstable) since there s one not so nice issue about it:

• #644788: screen 4.1.0 can t attach to a running/detached screen 4.0.3 session

Nevertheless it fixes a lot of open issues (of which the oldest is a wishlist bug report dating back to 1998 :-) and I didn t want to withhold it from the rest of the Debian community so I uploaded it to Debian Experimental. Issues closed in Debian Experimental

• #25096: digraph table should be run-time configurable
• #152961: lacks tsl/fsl/dsl caps
• #176626: mini-curses type of interface for screen -r w/ multiple screens? (Fixed by suggesting iselect, screenie or byobu)
• #223320: does not switch mouse mode
• #344759: mishandles xterm control string to set window title
• #353090: please enable the built-in telnet
• #361274: cannot reattach to sessionname if there is another session with similar sessionname
• #450421: please raise MAXWIN to at least 100 (merged with #499273)
• #461107: Requires test -t 0 even when opening a new window on existing screen
• #481411: window created with -d -m silently ignores -X exec
• #488619: Session name string escape
• #496750: screen -d -m and -D -m segfault if setenv given with no value in a configuration file
• #532240: screen with caption SEGVs when resized to 1 line tall
• #541793: C-a h (mis)documented twice
• #558724: breaks altscreen
• #560231: Please remove restriction on user/login name length
• #578729: outputs spaces when refreshing/attaching a window with defbce on
• #591624: segfault when running screen -d -m with layout save default in .screenrc
• #603009: Updating the screen Uploaders list
• #612990: /etc/init.d/screen-cleanup: should check for existence of screen binary
• #621704: Fix slow scrolling in vertical splits
• #630535: manpage typo
• #641867: version bump (this bug report sparked the upload :-)

Issues which will be closed in Ubuntu

• #183849: update to git version of screen
• #315237: crashes with certain options and terminal sizes
• #582153: doesn t accept login names longer than 20 chars
• #588846: slow when using vertical split
• #702094: Copying and pasting from mutt includes many trailing spaces
• #786292: segfaults if using layout saving with -D -m
• #788670: segfault in screen/byobu in natty

Please test the version from Experimental If you are affected by one of the issues mentioned above, please try the version from Debian Experimental and check if they re resolved for you, too. Thanks to all who contributed! A lot of the fixes have been made or applied upstream by Sadrul Habib Chowdhury who also industriously tagged Debian bug reports as fixed-upstream . Thanks! Thanks also to Brian P Kroth who gave the initial spark to this upload by packaging Fedora 15 s git snapshot for Debian and filing bug although the upload is based on the current HEAD version of GNU Screen as this fixes some more important issues than the snapshot Fedora 15 includes. That way also two patches from Fedora/RedHat s screen package are included in this upload. (Co-) Maintainer wanted! Oh, and if you care about the state of GNU Screen in Debian, I d really appreciate if you d join in and contribute to our collab-maint git repository there are still a lot of issues unresolved and I know that I won t be able to fix all of them myself. And since Hessophanes unfortunately currently has not enough time for the package, we definitely need more people maintaining this package. P.S. Yes, I know about tmux and tried to get some of my setups working with it, too. But I still prefer screen over tmux. :-)

As I wrote in part 1, my family and I were in Mexico recently. Today I ll write about the places we stayed. We spent most of the time in a room we rented in a private home in Guadalajara. My friend Jonathan had found it for us, and it was not too far from his home. The owner was a grandmother, and across the courtyard was more family, including a granddaughter close in age to our boys. They enjoyed playing together. It was really a perfect arrangement for us. There s no better way to be a part of local life when traveling than to stay in someone s house. Here s our bedroom: Notice the glass slats in the window it s a nifty, though not airtight, alternative to our regular windows. More on that later. More of the inside: We had a bit of a language barrier while in Mexico, though never anything significant. My Spanish vocabulary started with almost nothing and I reached maybe a few dozen words by the time we left. Terah knew some Spanish from high school and college, and my friend was fluent. Our hostess also knew a little English. But we all communicated well enough. Terah or Jonathan would help translate when needed. As I d seen before, the children would say things to each other, but never seemed to be bothered that their playmates didn t understand what was being said. They just had a great time anyhow. On Sunday afternoon, when we came back from our activities, there was a buzz of activity. Children everywhere outside, running and playing. Adults too, chatting. We didn t know exactly what was happening, but sent Jacob and Oliver out to play anyhow (which they were eager to do). The yard was enclosed by a wall, so children could pretty much run around without lots of supervision. Eventually they were invited to have some birthday cake (ah ha!) it was one of the children s birthday. Jacob and Oliver actually were served the first two pieces of cake (as the amigos ). Everyone seemed so friendly, warm, and welcoming. Each morning started with breakfast at the house, followed by a scene like this: That s Jacob and Oliver, looking to see if Jonathan had arrived for the day yet. I often noticed in Mexico that I was unsure if I was inside or outside. Here in Kansas, we can have a string of summer days that each exceed 110F (44C) or a few weeks in winter that never get above 15F (-9C), even in daytime. And then we have some pleasant days like right now, too or rain blowing sideways at 60MPH. In general, we spend a lot of effort keeping the outside, well, out. It is quite clear that this isn t a problem in the Guadalajara area. Some restaurants could have been described as buildings with large, open windows so you feel a lot of breeze while inside. Or perhaps as a simple shade roof with a few supports on the edges, so you re never really inside at all (sort of like going under a small shade tent outdoors). To the extent that windows could close, many of them couldn t be made airtight. It was clear that in that area, people spend more energy finding ways to invite the outdoors in rather than to keep it out, thanks to the year-round moderate climate. Perhaps the best example of this surprised me one evening. We were arriving in Guanajuato, an old silver mining town in the mountains, and were going to spend the night at Hotel Socav n, which had been recommended to us by a local friend of Jonathan s. From the street, the hotel looked tiny. But walk in, and you get in this old-looking (and feeling) entry tunnel: That s the front desk, apparently cut out of it there on the right. We asked to see some of the rooms before buying apparently a normal request around there. The innkeeper agreed, and gave us keys and directions to find them on the third floor. It included going up 2 flights of stairs, passing through a courtyard, and going up another flight. It was after dark, and the hotel was dimly lit something I was fine with. I thought we were stepping out into a beautiful atrium with some potted plants in the center of the building something fairly common in some nicer hotels. Until I felt rain on my head. Then I realized that the courtyard, which began two floors up from the street, was open to the sky. Beautiful! Here was the view from out room door: And down the hall : After getting home, a Google happened to turn up some reviews of this hotel. I was so annoyed at what some people wrote! One person gave them only 3 stars because they didn t have air conditioning, had poor water pressure, and lots of steps . Someone else complained of the dark entry tunnel something I couldn t help but smiling about the moment I entered. My review, which should hopefully get posted soon, is certainly different. I gave them 5 stars, because if I wanted a Super 8 with generic fluorescent lighting and the same layout as thousands of other hotels, I would have gone to Nebraska instead of Mexico. Most homes and local hotels in the region don t have air conditioning because they don t need it, and that s just how water pressure is in Mexico (due to needing to pump it from municipal supplies to private storage tanks overhead). And who doesn t appreciate entering a hotel through a brick tunnel? Ah, sigh This should give you some idea of the kind of travel we like: part of the point of traveling is enjoying the differences from home, and I think it is a huge mistake to be annoyed at everything that is different. Enjoy the differences! Finally, here s a photo of the staircase in the home we stayed in, which I thought was fascinating:

A commentor on my previous post asked why I think MySQL is a toy. I've actually blogged about that a number of times, but when wanting to point that out, I found that most of those posts point out just one thing, rather than having one post that enumerates them all. So let's remedy that, shall we? There are many things wrong with MySQL, including, but not limited to:

• In my personal opinion, the whole storage engine thing is a bad idea. It leads to dilution of effort (the MySQL developers need to implement stuff more than once), and makes users choose between disjoint feature sets, which will almost always result in suboptimal results. It also neatly allows PR people to claim that MySQL is 'fast' and 'ACID compliant', without mentioning that you can't combine both.
• One of MySQL's default settings is to truncate string values if they are longer than the field in which they're supposed to fit. This eats data. Hint: it's a database. While truncating strings might be appropriate in extreme corner cases, making it the default is so wrong it's not even funny.
• One of MySQL's default settings is to use MyISAM as a storage engine, which doesn't do transactions. This means you don't have atomicity, which eats data. Hint: it's a database. While telling a user "if the system crashes, I might write your entire data set to disk, or I might write just half of it, or I might corrupt the data set and I'm not going to tell you until you try to read again" might be appropriate in extreme corner cases, making it the default is so wrong it's not even funny. Update: as many people pointed out to me in the comments, apparently one of the first things Oracle did when they took over MySQL is to change the default from MyISAM to InnoDB, making this point no longer valid. I still think the replaceable storage engine thing is a bad idea, but with InnoDB as default, it's not as much of a pain anymore.
• If those two weren't enough 'mysql eats your data' arguments, note that most distributions routinely run a data recovery tool at MySQL startup, because not doing so caused problems in the past.
• I've seen MySQL crash and burn and segfault reproducibly when it encountered a corrupt table. Now I'm not saying that the database should be able to read data as if nothing happened from a corrupt file, but it should not crash and burn and segfault; instead, it should produce an error message.
• Fetching a result set can be done in two ways: either you call mysql_use_result() before calling mysql_fetch_row() which tells MySQL you'll be fetching the result one row at a time, or you call mysql_store_result(), which will read the entire result set into memory. So far so good. The problem, however, is that if you use mysql_use_result(), you're locking the tables from which you're fetching data, and no other client will be able to update any data in those tables until you're finished. On the other hand, if you need to process a large amount of data that can't be processed on the server for some reason, you may need to run a query that returns more data than you have memory. In that case, running mysql_store_result() is plain impossible. This isn't just a theoretical thing; I've seen cases in data warehouse situations where a database client needed to process multi-gigabyte query results. Trying this on MySQL is a pain.
• When compared to PostgreSQL, the MySQL feature set is immature. For instance, here's a number of useful[1] features which PostgreSQL has but MySQL, to the best of my knowledge, does not (corrections are welcome):
  • table inheritance
  • asynchronous notification
  • full ACID compliance, in all cases. (MySQL only offers full ACID compliance if you pick a particular storage engine)
  • asynchronous command processing.
  • Very flexible authentication system, and real actual users. For instance, PostgreSQL supports Kerberos authentication, and understands that users may actually log in from different hosts (gasp!)
  • SELinux extensions, called SE-PostgreSQL.
  • Server-side languages are implemented using a plugin scheme, allowing stored procedures to be written bascially in any possible language. Someone wrote a PL/LOLCODE which, while not very useful, shows the flexibility; MySQL only supports one language for stored procedures and triggers if the storage engine supports triggers, which not all do.
  • Sequences (an AUTO_INCREMENT column is a reasonable workaround, but still not a sequence)
  • window functions
  • extensible data type system; of those, MySQL only supports enumerated types.
Against that list, MySQL can only pit "multi-master clustering". While I'm sure that's useful for some use cases, I remain unconvinced that it's a useful enough feature to have to deal with the administrative overhead that MySQL's multi-master clustering imposes upon you, or that it is worth losing all the above over.

So it's my opinion that any database which fails to store data correctly in its default settings can't be anything but a toy; or that a database which has a comparatively small feature set can't be anything but a toy. But maybe that's just me. [1] No, I haven't used all those features; but I have used asynchronous notification, sequences (other than for primary keys), kerberos auth, custom data types, and (obviously) I have enjoyed the extra peace of mind of knowing that my database is ACID compliant, meaning that it will either accept my transaction as a whole, or reject it as a whole (but usually the former). In addition, I've seen customers use the table inheritance feature.

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (170 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. This month passed by very quickly since I attended both the Libre Software Meeting / RMLL and the DebConf. Libre Software Meeting / RMLL I attended only 3 days out of the 6 but that was a deliberate choice since I was also attending DebConf for a full week later in the month. During those 3 days I helped with the Debian booth that was already well taken care of by Fr d ric Perrenot and Arnaud Gambonnet. Unfortunately we did not have any goodies to sell. We (as in Debian France) should do better in this regard next time. One of the talks I attended presented EnVenteLibre. This website started as an online shop for two French associations (Ubuntu-fr, Framasoft). They externalize all the logistic to a company and only have to care about ordering goodies and delivering to the warehouse of the logistic company. They can also take some goodies from the warehouse and ship them for a conference, etc. We discussed a bit to see how Debian France could join, they are even ready to study what can be done to operate at the international level (that would be interesting for Debian with all the local associations that we have throughout the world). Back to the LSM, while I had 3 good days in Strasbourg, it seems to mee that the event is slowly fading out it s far from being an international event and the number of talks doesn t make for a better quality. BTW, do you remember that Debconf 0 and Debconf 1 were associated to this event while it was in Bordeaux? dpkg-source improvements During my time in Strasbourg (and in particular the travel to go there and back!) I implemented some changes to 3.0 (quilt) source format. It will now fail to build the source package if there are upstream changes that are not properly recorded in a quilt patch:

dpkg-source: info: local changes detected, the modified files are:
 2ping-1.1/README
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/2ping_1.1-1.diff.cki8YB

As the error message hints, there s a new --commit command supported by dpkg-source that will generate the required quilt patch to fix this. In the processe you will have to submit a name and edit the patch header (pre-formatted with DEP3 compatible fields). You can get back the old behavior with the --auto-commit option. Build flags changes Ever since we adopted the Ubuntu changes to let dpkg-buildpackage set some build related environment variables (see #465282), many Debian people expressed their concerns with this approach both because it broke some packages and because those variables are not set if you execute debian/rules directly. In the end, the change was not quickly reverted and we fixed the package that this change broke. Despite this we later decided that the correct approach to inject build flags would be a new interface: dpkg-buildflags. Before changing dpkg-buildpackage to no longer set the compilation flags, I wanted to ensure dpkg-buildflags had some decent coverage in the archive (to avoid breaking too many packages again). My criteria was that CDBS and dh (of debhelper) should be using it. With the recent debhelper change (see #544844) this has been reached so I changed dpkg-buildpackage accordingly. Makefile snippets provided by dpkg At the same time, I also wanted an easy way for maintainers not using dh or CDBS to be able to fix their package easily and go back to injecting the compilation flags in the environment but doing it from the rules files. Starting with the next version of dpkg, this will be possible with something like this:

DPKG_EXPORT_BUILDFLAGS = 1
include /usr/share/dpkg/default.mk

Without DPKG_EXPORT_BUILDFLAGS the variables are not exported in the environment and have no effect unless you use them somewhere. More than build flags, this will also provide a bunch of other variables that can be useful in a rules files: all the variables provided by dpkg-architecture, vendor related variables/macro and some basic package information (mainly version related). dpkg-buildflags improvements Given the renewed importance that dpkg-buildflags will take now that dpkg-buildpackage no longer sets the corresponding environment variables, I thought that I could give it some love by fixing all the open issues and implementing some suggestions I got. I also had a chat with a few members of the technical committee to discuss how hardening build flags could be enabled in Debian and this also resulted in a few ideas of improvements. In the end, here are the main changes implemented:

• new prepend directive to inject flags at the start (see commit);
• new strip directive to strip flags from the result returned by dpkg-buildflags (see commit);
• new environment variables DEB_flag_MAINT_directive that can be set by the maintainer to adjust what dpkg-buildflags will return (see commit);
• new --export=configure command to inject build flags on the ./configure command line (see commit);
• new --dump command that is the default (see #603435).

Will all those changes, the complete set of compilation flags can be returned by dpkg-buildflags (before it would only return the default flags and it was expected that the Debian packaging would add whatever else is required afterwards). Now the maintainer just has to use the new environment variables to ensure the returned values correspond to what the package needs. DebConf: rolling and hardening build flags I spent a full week in DebConf (from Sunday 24th to Sunday 31th) and as usual, it s been a pleasure to meet again all my Debian friends. It s always difficult to find a good balance between attending talks, working in the hacklab and socializing but I m pretty happy with the result. I did not have any goal when I arrived, except managing the Rolling Bof (slides and video here) but all the discussions during talks always lead to a growing TODO list. This year was no exception. The technical committee BoF resulted in some discussions of some of the pending issues, in particular one that interests me: how to enable hardening build flags in Debian (see #552688). We scheduled another discussion on the topic for Tuesday and the outcome is that dpkg-buildflags is the proper interface to inject hardening build flags provided that it offers a mean to drop unwanted flags and a practical way to inject them in the ./configure command line. Given this I got to work and implemented those new features and worked with Kees Cook to prepare a patch that enables the hardening build flags by default. It s not ready to be merged but it s working already (see my last update in the bug log). A few words about the Rolling BoF too. The room was pretty crowded: as usual the topic generates lots of interest. My goal with the BoF was very limited, I wanted to weigh the importance of the various opinions expressed in the last gigantic discussion on debian-devel. It turns out a vast majority of attendants believe that testing is already usable. But when you ask them if we must advertise it more, answers are relatively mixed. When asked if we can sustain lots of testing/rolling users, few people feel qualified to reply but those that do tend to say yes. More dpkg work Lots of small things done:

• I did again some bug triaging on Launchpad. But Brian Murray did a lot of it and the result is impressive, we re down to 154 bugs (from more than 300 a month ago!).
• I updated my multiarch branch multiple times. I was hoping to meet Guillem during DebConf to make some progress on this front but alas he did not attend. I have been asked a status update multiple times during my time in DebConf.
• I fixed a regression in update-alternatives (#633627), a test-suite failure when run as root (#634961), a segfault in findbreakcycle. There have been a bunch of minor improvements too (#634510, #633539, #608260, #632937).

Package Tracking System and DEHS Christoph Berg recently wrote a replacement for DEHS because the latter was not really reliable and not under control of the QA team. This is a centralized system that uses the watch files to detect new upstream versions of the software available in Debian. I updated the Package Tracking System to use this new tool instead of DEHS. The new thing works well but we re still lacking the mail notifications that DEHS used to send out. If someone wants to contribute it, that would be great! Misc packaging work I did some preliminary work to update the WordPress package to the latest upstream version (3.2). I still have to test the resulting package, replacing upstream shipped copies of javascript/PHP libraries is always a risk and unfortunately all of them had some changes in the integration process. I also updated nautilus-dropbox to version 0.6.8 released upstream. I also uploaded the previous version (that was in testing at that time) to squeeze-backports. So there s now an official package in all the Debian distributions (Squeeze, Wheezy, Sid and Experimental)! Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

jupp 3.1.17 uploaded today, mostly thanks to user input suggesting I improve things, especially the syntax highlighting. (Maybe more to come.) I like users who don t complain but give helpful comments and send in patches even. Since the Debian FTP masters complain that the NEW queue is empty for the first time in ages, I also uploaded jupp to Debian proper (got requests, several, from actual users independent of each other). I originally thought I were the only user, it s not worth it, maybe too close to joe (which segfaults a lot more and has some ugly things, so I cherry-picked the better features of it instead of rebasing jupp), but it s had a package in mports (MidnightBSD ports) for ages, users submitted one to FreeBSD last year and keep it updated, there s even a WIP package in pkgsrc , and who knows where else or how many people are using my OpenSuSE Buildservice package or have had installed the previous DEB package I uploaded to my play repo. So now I feel it worth to upload. I even invested some major packaging rework, such as splitting the build-arch and build-indep parts from each other, and importing the upstream source into the packaging VCS, as I have learned in the packaging with git talk here at DebConf. (No guys, I will stick to CVS as git doesn t give me anything.)

After being dropped out of Debian, the m68k was stalled from some time now. There was no real upgrade path and so my machines still are running etch-m68k. Thanks to Thorsten Glaser the port is slowly keeping up with NPTL now ported to kernel and glibc for m68k. He took care to port and compile a lots of packages that are needed for upgrading from etch-m68k. Big thanks for that to Thorsten! Anyway, I'm in the progress of upgrading my m68k machines and buildds with the help and tips from Thorsten and this is the way I'm doing this:

1. Change your /etc/apt/sources.list to include this:
   
   deb http://ftp.debian-ports.org/debian/ unstable main contrib
   deb http://archive.debian.org/debian etch-m68k main contrib non-free
   
2. Get libuuid-perl from snapshot.debian.org:
   
   wget http://snapshot.debian.org/archive/debian/20070128T000000Z/pool/main/libu/libuuid-perl/libuuid-perl_0.02-1_m68k.deb
   dpkg -i libuuid-perl_0.02-1_m68k.deb
   
3. Get kernel & linux-base from unstable
   You need to install a recent kernel like linux-image-2.6.39-2-amiga in my case. Either download it by hand or use apt:
   
   apt-get -d install linux-image-2.6.39-2-amiga linux-base
   cd /var/cache/apt/archive
   dpkg --force-depends -i linux-image-2.6.39-2-amiga_2.6.39-3_m68k.deb linux-base_3.3_all.deb
   
4. If needed, remove linux-bases postinst, when you get this kind of error:
   
   syntax error at /var/lib/dpkg/info/linux-base.postinst line 1275, near "# UUIDs under /dev"
   Can't use global $_ in "my" at /var/lib/dpkg/info/linux-base.postinst line 1289, near " $_"
   rm /var/lib/dpkg/info/linux-base.postinst
   dpkg --configure --pending --force-depends
   
   
5. If everything installed fine you should be ready to boot into your new kernel.
   On Amigas you mostly likely need to edit your boot script and copy your kernel and System.map to your AmigaOS partition. This is how my boot looks like:
   
   amiboot-5.6 -k vmlinux-2.6.39 "debug=mem root=/dev/sda4 video=pal-lace devtmpfs.mount=1"
   
   You can omit the debug=mem. This is just for the case that the kernel crashes. You can then collect the dmesg output under AmigaOS with the dmesg tool. The other parameter devtmpfs.mount=1 is needed because we don't want udev. Using video=pal-lace is necessary because the 2.6.39 kernel crashes on initializing my PicassoII graphics card and I've unplugged the card for the time being necessary to solve the problem.
   
6. Kernel 2.6.39 runs fine, but you can't ssh into machine.
   Because we don't want udevd, there's now a problem when trying to login by SSH:
   
   pty allocation request failed on channel 0
   stdin is not a tty
   
   You can fix this either by installing udev, which most websites recommend when you're looking for this error, because on Xen this is the recommended solution, but as we are on m68k and not under Xen, it's better to run with a static /dev. So you need to create /dev/pts and add the following to your /etc/fstab:
   
   mkdir /dev/pts
   devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
   
7. After kernel boots into 2.6.39 you can dist-upgrade to unstable.
   When you successfully booted into your new kernel, you should be save to proceed with upgrading to unstable. For that you should first let the missing or broken depends from linux-image and linux-base being installed:
   
   apt-get -f install
   
   This should lead to install some dependencies that were missing, because using dpkg --force-depends from above. After that I upgraded dpkg, apt and apt-utils:
   
   apt-get install dpkg apt apt-utils
   
   When this succeeded, you should be save to fully dist-upgrade to unstable:
   
   apt-get -u dist-upgrade
   
   When you get errors during apt-get dist-upgrade, you might run dpkg --configure --pending or apt-get -f install, before proceeding with apt-get -u dist-upgrade. Another problem can occur with apt. When you see this error:
   
   E: Could not perform immediate configuration on 'perl-modules'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
   
   you should add "-o APT::Immediate-Configure=false" to your apt-get command, for example:
   
   apt-get -o APT::Immediate-Configure=false -f install
   
   Another pitfall might be exim4-daemon-heavy, which currently segfaults. Replace it by exim4-daemon-light in that case, which works.

As stated above, my PicassoII in my A3000 doesn't seem to work under 2.6.39, whereas the PicassoIV in my A4000T does not crash the kernel. Please don't hesitate to add additions, corrections or other kind of feedback by commenting below! P.S.:
Wouter and Thorsten are currently at Debconf in Banja Luka, working on the m68k port. Wouter just finished a first version of a new debian-installer image. He asks for testing it on real hardware. Please volunteer if you can! It's available at: http://people.debian.org/~wouter/di-m68k/

I had cause to try and get a core dump from a segfaulting process at work the other day, and I wanted to figure out if fiddling with /etc/security/limits.conf was going to do the trick (I didn't) or if I had to modify the initscript to include a ulimit -c unlimited call. Of course, on a modern system (>= 2.6.24), one would just take a look at /proc/PID/limits and get on with life, but unfortunately the system in question was running 2.6.18, so more fiddling around was required. I'd found something once before that told me how to do it with GDB, but all I could find this time around was a rather over-complicated Knol article, which made a bunch of assumptions (well mainly that the binary in question wasn't stripped). So with some help from the Knol article, I muddled through it. Disclaimer: I don't profess to be an expert on system internals like this, so if this eats your first-born, don't come crying to me. Firstly, you need to know that it's the getrlimit(2) system call that you want to be using, and then you need to figure out the number for the resource limit you want to retrieve. The man page for getrlimit(), tells you it's defined in /usr/include/sys/resource.h, but I've found that the actual useful bits end up being in /usr/include/bits/resource.h I wanted the resource limit for the maximum core dump size, which is RLIMIT_CORE and has a value of 4. Next, you need to know that the getrlimit() system call takes an integer and a pointer to an rlimit structure as arguments. We've just figured out the value for the integer, but we're also going to need to pass a pointer as the second parameter. A pointer to enough memory to hold an rlimit structure. Fortunately, the rlimit structure is pretty simple:

struct rlimit  
    rlim_t rlim_cur;  /* Soft limit */
    rlim_t rlim_max;  /* Hard limit (ceiling for rlim_cur) */
 ;

After a bit of grepping around in /usr/include, I determined that an rlim_t is essentially an unsigned long int, so we need to allocate a pointer big enough to hold two of them. Note that if we had an unstripped binary, we could have saved a lot of faffing around by just going

print sizeof(struct rlimit)

in GDB (assuming that the binary has the getrlimit symbol in it) The sure-fire way of figuring out how much memory we need for this pointer is to go

print sizeof(unsigned long int)

in GDB, and then double that (since we want two of them). On my system an unsigned long int is four bytes, so I'm going to want to allocate enough memory for 8 bytes. Now it's time to attach GDB to the offending process and see what the resource limit currently is. (gdb -p PID)

(gdb) print malloc(8)
$1 = 152186904
(gdb) print getrlimit(4, $1)
$2 = 0
(gdb) x/2xw $1
0x9123018:	0x00000000	0x7fffffff
(gdb) quit

So in this particular case, we've retrieved the soft and hard limits of the RLIMIT_CORE resource limit, and you can see that the soft limit is zero, and the hard limit is unlimited. Note that the getrlimit() function returns an integer as its return code, which is what the $2 = 0 is above. Now it's just a case of altering the the resource limits via the preferred mechanism, restarting the process and then repeating this GDB examination of the process to check they were changed successfully. Circumstances where this can all fall down would appear to be ones where the getrlimit symbol isn't present in the binary, or the binary is compiled as a position-independent executable. I'd think that in the latter case, the system is probably modern enough to be running a kernel that supports directly examining the resource limits via the /proc filesystem.

When soliciting for opinions, recently, I discovered that the python-based fabric tool was not dead, and was in fact perfect for my needs. During the process of getting acquainted with it I looked over the source code, it was mostly neat but there was a trivial (low-risk) symlink attack present. I reported that as #629003 & it is now identified more globally as CVE-2011-2185. I guess this goes to show that getting into the habit of looking over source code when you install a new package is a worthwhile thing to do; and probably easier than organising a distribution-wide security audit </irony>. In other news I'm struggling to diagnose a perl segfault, when running a search using the swish-a perl modules. Could it be security worthy? Possibly. Right now I just don't want my scripts to die when I attempt to search 20Gb of syslog data. Meh. ObQuote: "You're scared of mice and spiders, but oh-so-much greater is your fear that one day the two species will cross-breed to form an all-powerful race of mice-spiders who will immobilize human beings in giant webs in order to steal cheese. " - Spaced.

Well hello there. A couple of days ago my debian.org account was created, meaning that I'm one1 of the crop of current new Debian Developers. Actually the news was broken to me by Rhonda when I attached to irssi after arriving at work, a nice surprise

<Rhonda> All congratulate Laney on becoming a Debian Developer.  ;)
* Rhonda . o O ( http://db.debian.org/search.cgi?dosearch=1&uid=laney )
<Laney> Rhonda: I did?!?!?!

I'll quickly introduce myself by paraphrasing from the background section of the AM report before letting you go about your business. I apparently submitted my first thrilling patch to the alsa-tools package in Ubuntu on February 2nd, 2008. This was sponsored into Hardy by Daniel Chen. Thereafter followed a myriad of exciting patches to various packages that somehow managed to convince a bunch of people that I had enough skill to become an Ubuntu developer. Fast forward a while and I get sucked into the world of Debian packaging by the CLI/Mono strike force of Mirco Bauer and Jo Shields by way of the Mono 2.0 transition. This was where I got my first Debian upload, and it was in this team that I fully realised both the excellence and importance of Debian in the FOSS world2. At some point the Debian Haskell Group formed and I've been involved to some extent there all along too. What I've mainly learned from these two groups is that team maintenance is a really great way to look after a bunch of related packages. When I see people touting new packages about, I often recommend that they look at the list of teams to find a nice home. Perhaps one or two actually did. Thanks to everyone who's supported me so far. I hope to be able to do the same for others in the future.

What is CORBA? The wikipedia page says it all:

The Common Object Request Broker Architecture (CORBA) is a standard
defined by the Object Management Group (OMG) that enables software
components written in mutiple computer languages and running on
multiple computers to work together (i.e., it supports mutiple
platforms).

The standard was created twenty years ago, in 1991. The technology can be used in many programming languages: Python, Java, C++, C, etc. Hence you can write a software in your favorite programming language, exposing services that others will be able to call using a different language. (Nowadays you can do the same with WebServices. This article is about CORBA.) All CORBA specifications are available on [OMGSPEC]; latest version is [CORBA3.1]. [MCHALE] book is a very good way to learn CORBA.