It s been a while since I ve written about Moonshot. A lot has gone on; we ve been too busy doing to be busy blogging. However there s something that s happened recently that s so cool I had to take a moment to discuss it. Padl Software, the same people (well person) who brought us LDAP support to replace NIS and the first Active Directory clone, has now produced a GSS-EAP Security Service Provider. That s software that implements the Moonshot protocol and plugs it into the standard Windows security infrastructure. This is neat because it allows you to use GSS-EAP with unmodified Windows applications like Internet Explorer and Outlook/Exchange. Obviously, this will be great for Moonshot. However, I think the positive affects are more far-reaching than that. Luke has demonstrated that we can evolve the Windows security infrastructure without waiting for Microsoft to lead the way. For those of us working in the enterprise security space, that s huge. We can innovate and bring our innovation to Windows. In terms of getting acceptance in important user communities, getting funding for work, and making a practical difference, that s a big deal. This code is still in the early stages. Padl has not decided how the code will be made available. We don t know if it will be under an open-source license yet. Luke, naturally wants to get paid for his work. However if this code does get released under an open-source license, it will be very valuable. That will give all of us who are looking for a starting point for security innovations a starting point for bringing our innovations to Windows. Some in the open-source community will argue that we shouldn t work on improving Windows: if the open-source platforms have features Windows does not, then it may drive people to open-source. Especially for enterprise infrastructure, it tends not to work that way. You need broad cross-platform support to drive new technology. However, it does mean that we can take control of the evolution of our infrastructure; even for Windows there is no requirement that a single vendor controls what is possible.

Poor Sarah. Cat dramas always seem to befall us when she's in Australia. It looks as if Smudge has managed to escape the house. Being at work all day, it took me 24 hours to notice. Our friend Helen came over on Sunday night. Smudge was definitely around then, because she was climbing all over Liam. Helen used the downstairs bathroom before she went home, and I remember her having to shoo Smudge out because she was in there. I'm pretty sure she went home after that. If my memory serves, she saw herself out and I was in the kitchen cleaning up, so my current theory is that she escaped when Helen left. On Monday night, I had a lengthy video call with Sarah, who was at my parent's place setting up their new Mac Mini. I didn't realise until after that, when I was about to go to bed, that I hadn't been harassed by Smudge all evening. So I searched the house that night, and couldn't find her. Liam had already gone to bed for the night, so I hoped she was stuck in his room, and went to bed too. This morning, Liam and I both searched the house, and the immediate surrounds, without any luck. I called the Palo Alto Animal Services (I got the dispatch because the shelter hadn't opened yet), emailed the local neighbourhood association, and went to work. I called Palo Alto Animal Services again later in the day to report her missing to the shelter itself, and Betty-Ann there recognised my accent and name and asked me if I was Sarah's husband (Sarah used to volunteer there a lot, and we adopted Smudge from there), so they're now keeping a look out. Smudge is micro chipped, but wasn't wearing a collar. She's allowed in the back yard, because it's totally enclosed and can't escape, so she shouldn't be totally freaked out by being outdoors. Whenever she does escape out the front door, she usually makes a beeline to the right, which is mostly enclosed, but she can certainly get out of the complex. Unfortunately she often doesn't come when called, so for all I know she's in some obscure location 10 metres from the front door and I can't see her. My main concerns are that we're only two blocks from Highway 101, so if she does stray too far, she could get into serious trouble, and I'm also a bit worried about raccoons giving her grief. I spent some time tonight putting up posters in the immediate area and poking around bushes. I've left the two humane cat traps that we have out by the front door with some cat food in them, in the hope that she's roaming around and will walk into one of them. Oh and the other big concern I have is that Halloween is coming up. Shelters tend not to adopt out black cats around Halloween because bad things can happen to them. So now we wait. I'm really hoping she turns up safe and sound, soon.

I have recently been working on some code for NetSurf which builds up an output buffer by repeatedly calling snprintf(); No great shock there, well understood trivial pattern that has been used repeatedly for ages. Gosh I feel silly now, in fact I was so convinced I was right I wrote a program to "prove" it.

I have several backup servers that run the excellent rsnapshot software, which uses Secure Shell (SSH) for remote access. The SSH private key of the backup server can be a weak link in the overall security. To see how it can be a problem, consider if someone breaks into your backup server and manages to copy your SSH private key, they will now have the ability to login to all machines that you take backups off (and that should be all of your machines, right?). The traditional way to mitigate SSH private key theft is by password protecting the private key. This works poorly in an unattended server environment because either the decryption password needs to be stored in disk (where the attacker can read it) or the decrypted private key has to be available in decrypted form in memory (where attacker can read it). A better way to deal with the problem is to move the SSH private key to a smartcard. The idea is that the private key cannot be copied by an attacker who roots your backup server. (Careful readers may have spotted a flaw here, and I need to explain one weakness with my solution: an attacker will still be able to login to all your systems by going through your backup server, however it will require an open inbound network connection to your backup server and the attacker will never know what your private key is. What this does is to allow you to more easily do damage control by removing the smartcard from the backup server.) In this writeup, I ll explain how to accomplish all this on a Debian/Ubuntu-system using a OpenPGP smartcard, a Gemalto USB Shell Token v2 with gpg-agent/scdaemon from GnuPG together with OpenSSH.


First we need to install some packages. The goal is to configure OpenSSH to talk to the gpg-agent which will start and talk to scdaemon which in turn talks to pcscd which talks to the smart card reader. For some strange reason, the scdaemon binary is shipped with GnuPG s S/MIME interface in the gpgsm package.

# apt-get install pcscd gnupg-agent gpgsm

The above command should install and start pcscd and if all works well, you should be able to check the status of the smartcard using GnuPG.

# gpg --card-status

You need to initialize the smartcard and generate a private key on it, again using GnuPG. If you trust GnuPG more than the smartcard to generate a good private key, you may generate the private key using GnuPG and then move it onto the smartcard (hint: use the keytocard command). Make sure you don t leave a copy of the private key on the same machine!

# gpg --card-edit
gpg: detected reader  Gemalto GemPC Key 00 00'
...
gpg/card> admin
Admin commands are allowed
gpg/card> name
Cardholder's surname:
Cardholder's given name: host.example.org
gpg: 3 Admin PIN attempts remaining before card is permanently locked
Please enter the Admin PIN
gpg: gpg-agent is not available in this session
gpg/card> lang
Language preferences: en
gpg/card> generate
Make off-card backup of encryption key? (Y/n) n
Please enter the PIN
What keysize do you want for the Signature key? (2048)
What keysize do you want for the Encryption key? (2048)
What keysize do you want for the Authentication key? (2048)
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: host.example.org
Email address:
Comment:
You selected this USER-ID:
    "host.example.org"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (33 seconds)
gpg: signatures created so far: 0
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (18 seconds)
gpg: signatures created so far: 1
gpg: signatures created so far: 2
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (23 seconds)
gpg: signatures created so far: 3
gpg: signatures created so far: 4
gpg: key 12345678 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   2048R/12345678 2011-09-19
      Key fingerprint = 1234 5678 1234 5678 1234  5678 1234 5678 1234 5678
uid                  host.example.org
sub   2048R/23456789 2011-09-19
sub   2048R/34567890 2011-09-19
gpg/card> quit
#

Now for the interesting part. OpenSSH talks to an agent for private key handling, and GnuPG s gpg-agent supports this protocol when the --enable-ssh-support parameter is given. During startup, gpg-agent will print some environment variables that needs to be set when ssh is run. Normally gpg-agent is invoked by the Xsession.d login scripts, so that the environment variables are inherited by all your processes. However, for an unattended machine without any normal login process, we need to write a script to start gpg-agent. First do these manual steps, to confirm that everything works.

# gpg-agent --daemon --enable-ssh-support > /var/run/gpg-agent-info.env
# . /var/run/gpg-agent-info.env
# ssh-add -L
ssh-rsa AAAAB3N... cardno:000500000BD8
#

If the final step printed a SSH public id, the (sometimes) tricky part in getting the hardware to work is (hopefully) complete. What remains is to script things so that gpg-agent is started on boot and to make sure that your backup scripts has the proper environment variables before launching whatever processes will launch ssh. Further, since we will be running unattended, we need a mechanism to unlock the smartcard using a PIN interactively once on each boot of the machine. I prefer manually entering the PIN on every boot over having the PIN stored in a file on the disk. I will use the /etc/rc.local mechanism to start gpg-agent, like this:

# cat> /etc/rc.local
#!/bin/sh -e
exec gpg-agent --daemon --enable-ssh-support \
    --pinentry-program /usr/local/sbin/pinentry-unattended \
    --write-env-file /var/run/gpg-agent-info.env
^D

The astute reader will now ask what /usr/local/sbin/pinentry-unattended is and why it is needed. Now here is the situation. scdaemon will normally query the user for a PIN using a tool called pinentry which reads and write to the user s TTY directly. This won t work in unattended mode, so we want the scdaemon to signal failure here unless we are actually unlocking the smartcard manually. Here is the entire script:

#!/bin/sh
# /usr/local/sbin/pinentry-unattended -- by Simon Josefsson
if test x"$PINENTRY_USER_DATA" = xinteractive; then
    exec pinentry "$@"
fi
exit 1

What remains is a script to unlock the smartcard by providing the PIN. This is typically invoked manually if the server has restarted for some reason. Don t worry, any ssh sessions invoked by cron until you have managed to unlock the smartcard will fail with an authentication error it won t hang waiting for a PIN to be entered.

#!/bin/sh
# /usr/local/sbin/unlock-smartcard -- by Simon Josefsson.
. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID
gpg-connect-agent 'scd killscd' /bye > /dev/null
while ! gpg-connect-agent 'scd serialno' /bye   grep -q SERIALNO; do
    sleep 1
done
PINENTRY_USER_DATA=interactive
export PINENTRY_USER_DATA
checkpin

And the script checkpin is as follows:

#!/bin/sh
# /usr/local/sbin/checkpin -- by Simon Josefsson.
id= gpg-connect-agent 'scd serialno' /bye   head -1   cut -d\  -f3 
gpg-connect-agent "scd checkpin $id" /bye   grep -q OK

At this point, you should have everything configured and installed. Don t forget to chmod +x the scripts. The typical use-pattern is as follows. After the machine has been started, gpg-agent is running but the smartcard is not unlocked with the PIN. You need to manually login to the machine and run unlock-smartcard and enter the PIN. In the script that runs the backup jobs, invoked via cron, make sure that the first line of the scripts reads (assuming Bourne shell script syntax):

. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID

To avoid needlessly attempting ssh connections if the smartcard is not unlocked, your backup script can also call the checkpin code and abort if it doesn t return true.

checkpin   exit 1

Some final words about debugging. A basic command to run to check that the GnuPG side is working is gpg --card-status, it should print some information about the smartcard if successful. To check that the SSH agent part is working, use ssh-add -L. If you get error messages, try killing the scdaemon process by running killall -9 scdaemon and let gpg-agent respawn a new scdaemon process. That s it! If you like my writeup, please flattr it.

Wouter described how to get Akonadi (the back-end for KDE PIM) to use PostgreSQL [1]. I don t agree with his MySQL is a toy sentiment. But inspired by his post I decided to convert some of my systems to use a MySQL instance running on a server instead of one instance for each user. In the default configuration you have 140M of disk space and 200M of RAM used by each user for a private MySQL installation which has about 24K of data (at least at the moment on systems I run, maybe more in future). Here s some pseudo shell script to dump the database and get a new config: mysqldump socket=$HOME/.local/share/akonadi/socket-$HOSTNAME/mysql.socket akonadi > dump.sql
akonadictl stop
rm -rf .config/akonadi
rm -rf .local/share/akonadi
mkdir .config/akonadi
cat > .config/akonadi/akonadiserverrc <<EOF
[%General]
Driver=QMYSQL
SizeThreshold=4096
ExternalPayload=false

[QMYSQL]
Name=$USER_akonadi
Host=IP_OR_HOST
User=$USER
Password=$PASS
StartServer=false
Options=
ServerPath=/usr/sbin/mysqld

[Debug]
Tracer=null
EOF Then with DBA privs you need to run the following in the mysql client: create database $USER_akonadi;
GRANT ALL PRIVILEGES ON $USER_akonadi.* to $USER @'% IDENTIFIED BY $PASS ; Then run the following to import the SQL data: mysql $USER_akonadi < dump.sql Ideally that would be it, but on my test installation (Debian/Squeeze MySQL server and Debian/Unstable KDE workstations) I needed to run the following SQL commands to deal with some sort of case problem.
rename table schemaversiontable to SchemaVersionTable;
rename table resourcetable to ResourceTable;
rename table collectionattributetable to CollectionAttributeTable;
rename table collectionmimetyperelation to CollectionMimeTypeRelation;
rename table collectionpimitemrelation to CollectionPimItemRelation;
rename table collectiontable to CollectionTable;
rename table flagtable to FlagTable;
rename table mimetypetable to MimeTypeTable;
rename table parttable to PartTable;
rename table pimitemflagrelation to PimItemFlagRelation;
rename table pimitemtable to PimItemTable; I am not using any PIM features other than the AddressBook (which hasn t been working well for a while), so I m not sure that this is working correctly. But I would prefer that something I don t use and is probably broken take up disk space and RAM on a server instead of a workstation anyway.

• [1] http://grep.be/blog/en/computer/debian/no_more_mysql?show_comments=yes

A month ago, I published an article on the compared performances of stunnel, nginx and stud as SSL terminators. The conclusion was to use stud on a 64-bit system, with session caching and AES. stunnel was unable to scale properly and nginx exhibited important latency issues. I got constructive comments on many aspects. Therefore, here is the second round. The protagonists are the same but both the conditions and the conclusions have changed a bit.

Benchmarks The hardware platform remains unchanged:

• HP DL 380 G7
• 6 GB RAM
• 2 L5630 @ 2.13GHz (8 cores), hyperthreading disabled
• 2 Intel 82576 NIC
• Spirent Avalanche 2900 appliance

We focus on the following environment:

• Ubuntu Lucid, featuring a patched version of OpenSSL 0.9.8k with AES-NI builtin and enabled by default
• 2.6.39 kernel (250 HZ)
• 64-bit
• 1 core for HAProxy, 2 cores for network cards¹, 4 cores for SSL, 1 core for the system
• system limit of 100,000 file descriptors per process

On the SSL side, the main components are:

• 2048-bit certificates
• AES128-SHA cipher suite
• 8 HTTP/1.0 requests per client
• HTTP response payload of 1024 bytes

If you are interested in different conditions (32-bit, number of cores, 1024-bit certificates, proportion of session reuses), have a look at the previous article whose results on the matter are still valid. I only consider 2048-bit certificates because most certificates issued today are 2048-bit. In SP800-57, NIST advises that 1024-bit RSA keys will no longer be viable after 2010 and advises moving to 2048-bit RSA keys:

If information is initially signed in 2009 and needs to remain secure for a maximum of ten years (i.e., from 2009 to 2019), a 1024-bit RSA key would not provide sufficient protection between 2011 and 2019 and, therefore, it is not recommended that 1024-bit RSA be used in this case.

And the protagonists are:

• stud, @e207dbc5a3, with session cache enabled for 20,000 sessions, listen backlog of 1000 connections,
• stunnel 4.44, configured with --disable-libwrap --with-threads=pthread (configuration),
• nginx 1.1.4, configured with --with-http_ssl_module (configuration).

UPDATED: Configuration of nginx has been updated to switch to off both multi_accept and accept_mutex thanks to the feedback of Maxim Dounin on latency issues that were present in the previous article (and still present in the early version of this article). HAProxy (configuration) is used for stunnel and stud to add load-balancing. For each test, we want to maximize the number of HTTP transactions per second (TPS) while minimizing latency and unsucessful transactions. The maximum number of allowed failed transactions is set at 1 transaction out of 1000. The maximum allowed average latency is 100 ms².

Control case Below is the first comparison between stud, nginx and stunnel on plain Ubuntu Lucid. Like in previous benchs, stunnel is not able to scale, with a maximum of 400 TPS. However, early latency issues of nginx has been solved, thanks to the change in configuration. stud and nginx gets similar performance but, as you can see on the bottom plot (whose scale is logarithmic) stud starts with an initial latency of 40 ms. This is pretty annoying. I have tried some micro optimization to solve this issue but without luck. If latency matters for you, stick with stunnel or nginx (or find where the problem lies in stud). UPDATED: After reporting this issue, a fix has been commited. The culprit was Nagle s algorithm which improves bandwidth at the expense of latency. With the fix, performances are mostly the same but there is no latency penalty any more.

OpenSSL 1.0.0e When I published the first round of benchmarks, Micha Trojnara was dubious about the bad results of stunnel. He told me on Twitter:

I solved it! You compiled stunnel against a 4 years old OpenSSL with MT-unsafe SSL_accept. Upgrade OpenSSL and rebuild stunnel. This bug was fixed in OpenSSL 1.0.0b. I implemented a (slow) workaround, so stunnel doesn t crash with earlier versions of OpenSSL.

Therefore, I backported Ubuntu Oneiric OpenSSL 1.0.0e to Ubuntu Lucid³ and, as you can see yourself on the plot below, there is a huge performance boost for stunnel: about 400%! stud features a slight improvement too (about 8%) but still starts with a latency of 40 ms while nginx does not improve.

Intel Accelerator Engine Most Linux distributions are still using OpenSSL 0.9.8. For example, Debian Squeeze ships with OpenSSL 0.9.8o, Ubuntu Lucid with 0.9.8k and RHEL 5.7 with 0.9.8e. Therefore, they are not eligible to recent performance improvements. To solve this, Intel released Intel Accelerator Engine, which also brings, among other things, AES-NI support:

This is essentially a collection of assembler modules from development OpenSSL branch, packed together as a standalone engine. Standalone means that it can be compiled outside OpenSSL source tree without patching the latter. Idea is to provide a way to utilize new code in already released OpenSSL versions constrained by support and release policies limiting their evolvement to genuine bug fixes.

Alas, in my tests, there was no improvement compared to OpenSSL 1.0.0e (already patched with AES-NI support). stunnel comes with out-of-the-box support for additional engines. Just add something like this to your stunnel.conf:

engine=dynamic
engineCtrl=SO_PATH:/WOO/intel-accel-1.4/libintel-accel.so
engineCtrl=ID:intel-accel
engineCtrl=LIST_ADD:1
engineCtrl=LOAD
engineCtrl=INIT

For stud, you should grab support for engine selection from GitHub and for nginx, just specify the engine with ssl_engine.

Intel Performance Primitives In his great article Accelerated SSL , Simon Kuhn shows huge performance improvements by using Intel s IPP library, a library aimed at leveraging power of multi-core processors and including some cryptographic primitives and a patch to make use of them in OpenSSL. Unfortunately, this library is not free, not even as in free beer . The patch provided by Intel only applies on OpenSSL 0.9.8j. Simon updated it for OpenSSL 1.0.0d. It also applies with a slight modification to OpenSSL 1.0.0e. Look at Simon s article for instructions on how to compile OpenSSL with IPP. Simon measured 220% improvement on RSA 1024-bit private key operations by using this library. Unfortunately, we don t get such a performance burst with stud which exhibits a 15% improvement (but remember that private key operations only happen once every 8 transactions thanks to session reuse). Worst, with stunnel, performances are reduced by one third and nginx just segfaults. I think such a library is risky to use in production.

Hyperthreading During the previous tests, only half the capacity of each core were used. For stud, adding more workers per core only slightly improves this situation (less than 10% with 4 workers per core). Therefore, I thought using hyperthreading could rectify the situation here. In this case, stud, stunnel and nginx are allocated 8 virtual cores instead of 4. stud only got a small improvement of 5%, nginx a moderate improvement of 11% while stunnel got a significant improvement of 26%.

Conclusion Here is a quick recap of the maximum TPS reached on each tests (while keeping average response time below 100 ms). All tests were run on 4 physical cores with 2048-bit certificates, AES128-SHA cipher suite. Each client makes 8 HTTP requests and reuse the first SSL session.

Context	nginx 1.1.4	stunnel 4.44	stud @e207dbc5a3
OpenSSL 0.9.8k	1489 TPS	400 TPS	1492 TPS
OpenSSL 1.0.0e	1501 TPS	1561 TPS	1609 TPS
OpenSSL 1.0.0e + IPP	-	1101 TPS	1857 TPS
OpenSSL 1.0.0e + HT	1718 TPS	1973 TPS	1694 TPS

I also would like to stress that below 1500 TPS, stunnel adds an overhead of a few milliseconds while stud adds 40 milliseconds. With OpenSSL 1.0.0e and hyperthreading enabled, stunnel has both good performance and low latency on low loads to moderate loads. You may just want to limit the number of allowed connections to avoid collapsing on high loads. While in the previous round, stud was the champion and stunnel lagged behind, the situation is now reversed : stunnel is on top! UPDATED: As said earlier, the latency issue with stud has been solved. Performances remain mostly the same.

---

1. HAProxy and network cards don t need dedicated core for this kind of load. However, if you also want to handle non SSL traffic, those dedicated cores can get busy.
2. The real condition is more complex. Each point is considered valid if the corresponding average latency is less than 100 ms or if the average on the next 8 points is less than 100 ms. Temporary latency bursts are therefore allowed. We are not so forgiveful with failed transactions.
3. This is pretty straighforward: I just dropped multiarch support. I chose to use Ubuntu version of OpenSSL instead of upstream because there are some interesting patches, like automatic support of AES-NI.

As somebody who often watches movies and TV shows I have the recurring problem of finding the right English subtitles for the video I am about to enjoy. Now, that is really a tedious thing to do. You go on a website such as opensubtitles.org and type the title of what you are looking for. Then you usually have to choose from a plethora of results, download the archive, unzip it, fire up VLC, go to Video Subtitles Track Open File, and finally open your subtitles file. And what you typically find out at that point is that the subtitles are completely out of sync with the audio, but rather than doing the whole process again you prefer to just sync them with a bit of VLC-fu. Which always takes longer than you thought, but hey. Now, what if there was an extremely simple way to just choose the subtitles for a given movie directly from VLC? The good news is that somebody wrote a beautiful VLC extension that does exactly that. He also wrote an introductory article about VLC extensions. Thank you, Jean-Philippe Andr . Installing the thing is as simple as downloading the subtitles extension and putting it in the VLC scripts folder. On Linux machines, the extensions folder can be found at ~/.local/share/vlc/lua/extensions/, on Windows APPDATA\VLC\lua\extensions. But seriously, don t use Windows. Once the extension is installed you can start watching a movie and then click View Subtitles to open the subtitles search interface. This is how it looks like: Now that the practical problem is solved, please take some time to look at the source code. VLC uses Lua as the language for writing extensions. I certainly cannot say that the web is full of resources about how to extend VLC using Lua, but the piece of code I wrote about in this post is certainly a good way to start hacking, and maybe scratch yet another itch.

Our friends Andrew and Janice finally tied the knot last Sunday, in Langkawi, Malaysia. They're Aussies currently living in New York, and Janice is of Malaysian descent, so I think part of the reason was to accommodate some Malaysian relatives that wouldn't be able to travel, and the other reason was that it was going to mean a bunch of people had to travel anyway, so why not travel somewhere a bit exotic? As my sister is getting married later this month in Brisbane, we decided to spare Zoe two massive international flights in close succession, and Sarah and Zoe flew out a few days ahead of me, and went to Brisbane, and then Zoe, Sarah and her mum flew to Langkawi from there, and I flew there from San Francisco, with the plan being for Sarah and Zoe to hang out in Brisbane for the 4 weeks in between weddings. It was the longest amount of travel I think I've ever done. I hadn't sat down and calculated what the travel time was beforehand, and when I got on the flight from San Francisco to Hong Kong, and they said it was going to be nearly 14 hours just to Hong Kong, I nearly wept. The only good thing about it was I didn't have to wrangle a toddler as well. From there, I had to get to Singapore, with a several hour layover, and then another flight from Singapore to Langkawi. I left SFO at 1:20am on Monday morning, and finally arrived at about 7am (San Francisco time) the following day. I didn't even bother getting off the plane in Hong Kong, it was only an hour and a half layover. I think the layover in Singapore was 5 hours, and it was a separate flight. Singapore airport at least was pretty good to kill time in. It was funny, I was reading Wired while I was killing time in Singapore, and they had an article, which included a side bar on 10 Best Airports to Get Stuck In, and I observed that getting to Langkawi and back involved using 4 of them. The flight into Langkawi was more eventful than usual. I was dozing a bit as we were coming in to land, and I woke up to the sensation of taking off again. I wondered if we'd had an aborted landing? Sure enough, soon after, the captain got on the public address and said that they'd had to abort the landing due to bad weather in Langkawi. He had pretty bad Engrish, thanking us for our frustration and saying something about fuel, so I didn't get a lot of the details. We proceeded to circle for a while before finally landing. I think the flight ended up landing something like an hour or more later than scheduled. So that was mildly exciting. Langkawi airport was one of those little "walk across the tarmac" places. I think I stood the closest to an operating jet engine that I ever have, as another aeroplane taxied in to a stop between my plane and the terminal while I was walking to it. Loud. Anyway, that was all just me getting there. Sarah had a bit of fun getting to Brisbane on her own with Zoe. She's now too long to sleep in an in-flight bassinet, and doesn't seem to like sleeping vertically in the Ergo Baby Carrier (possibly she was too hot in it). I think she got a bit of sleep lying across Sarah's lap, but Sarah didn't get much sleep. I think there was similar fun for the Brisbane to Kuala Lumpur flight, but at least she had two laps to lie across for that one. Sarah, Zoe and Sarah's mum arrived in Langkawi earlier in the day on Tuesday, and I made it to the resort at about 10pm (local time) and pretty much immediately crashed for the night. The Westin Langkawi was just fabulous. I don't think we've stayed at a resort since our honeymoon, and this resort was a whole other level compared to Heron Island. The rooms were very comfortable, and the resort was very accommodating of a toddler. Their Kids Club menu was excellent. The breakfast buffet (included in our room rate) was the most comprehensive breakfast buffet I have ever seen in a hotel. The only thing missing was bacon, being a Muslim country, but I can live without that. It was pretty easy to just fill up on breakfast, skip lunch and just have a light dinner. The on-resort dining prices were also very reasonable. The off-resort prices were even cheaper. The resort facilities in general were very comprehensive. Three pools. A children's play room. A well equipped gym and recreational area including squash courts. An extremely nice day spa. The thing I liked best about the whole destination wedding thing was all of the activities leading up to the actual wedding. The wedding was on the Sunday, and we had the Hen's and Buck's parties on Friday. Apparently Australia is the only place that calls it a Buck's party. The rest of the world calls it a Stag party, how about that? There was some last-minute changes to the Buck's party, and we ended up chartering this huge $2 million yacht for 4 hours and going for a cruise and doing some jetskiing and water sports. The captain was Aussie, so we all had a good chat with him. Apparently it's a corporate vessel, with an exclusive chartering arrangement with the Westin. On Friday night, we all went out to a seafood buffet dinner at the Yellow Cafe over at Cenang Beach. Most of the guys and girls already knew each other courtesy of the day's activities, so dinner involved a lot of meeting the other halves. What I really liked about all of the pre-wedding stuff was that by the time the wedding rolled around, you really felt like you knew a bunch of Andrew and Janice's family and friends, which I think made the whole thing feel a lot more intimate. Saturday was a recovery day. Sarah's aunties Glenda (who was in Singapore on business) and Vicki (who lives in Thailand) had joined us at the resort by this stage, so we were all knocking around together checking out the island. We took the cable car to the top, but unfortunately the sky bridge was closed due to the inclement weather. There was a lot of touristy stuff at the bottom of where the cable car was, and Zoe got to have an encounter with a very large python. She even tried to give it a kiss. One of the things that was surprising was how popular Zoe was with all of the locals and other Asian tourists. We could probably have charged for photographs. There was a little girl, who I thought was possibly a similar age to Zoe (turned out she was two and a half) and I asked her parents if I could take a photo of the two girls together, and no sooner had they sat down together and a virtual paparazzi materialised out of nowhere taking photographs. Similarly, all of the staff at the Westin were totally taken by Zoe, and wanted to touch her. To her credit, Zoe handled all of the extra attention pretty well. The wedding wasn't until 6pm on Sunday, so we did a 4 hour island hopping boat tour in the morning. That was a bit of an experience. It was nothing short of sheer bedlam trying to board the right boat at the wharf (I use the term lightly) and then the boat's captain had to arrange the passengers appropriately so as to keep the boat on an even keel. It was a narrow motor boat with a canopy cover over the back two thirds of it. First stop was Dayang Bunting Island, home of Pregnant Maiden Lake, where we stopped for 45 minutes. More bedlam ensued getting off the boat at the wharf there (there was veritable flotilla of tour boats all trying to load and unload at the same time). Monkeys abounded. They seemed incredibly tame. We were practically tripping over them. Zoe didn't seem to be as taken by the monkeys as I thought she would. It was a short walk up a reasonable flight of stairs to get to the lake, though some incredibly humid jungle, and once we got there, there wasn't a huge amount to do in the time we had available. So we dunked Zoe's feet in the lake a few times and then made our way back to the chaos at the wharf and got back on our boat. We stopped for a similar amount of time at another island with a pretty nice beach, and Zoe had enough time to a quick swim in the ocean with Sarah, and then we made our way back to the boat again to another spot where we watched sea eagle feeding. Heaps of eagles circling overhead would take turns to swoop down and grab fish and fly off again. It was impressive. I think I managed to snap some good action shots with the long lens. Then we had the wedding itself. It was on the beach out the front of the Heavenly Spa on the resort. Let me say, the Westin knows how to put a wedding together. The ceremony was as lovely as the location. For the reception, the one 5 bedroom villa on the resort was booked for the night. Dinner was a buffet on the lawn out the front (briefly delayed thanks to a rain shower) and then speeches and general partying happened around the terrace out the back of the villa. There were some beautiful speeches. Janice's brought a tear to my eye. Part of the whole package included something like a 10 person video crew that were running around capturing everything. It felt a bit surreal. It was definitely the most high-end wedding I've ever been to (sorry Scott and Julie, your "Wedding of the Century" has been eclipsed by the "Wedding of the Millennium"). Monday was a wind down day. Zoe mercifully gave us a sleep in until 8am. Our flights didn't leave until the evening on Tuesday, so after we checked out we went for a drive around the island to try and get Zoe to take a nap in the car. We ended up at the fruit farm, which to the best of my understanding is run by the government. Zoe had fallen asleep by this stage, and Sarah said she'd stay behind in the car and have a nap too. So her mum and I went on the tour. It was a fun little tour. There were only four of us on it. They drive you around the farm on the back of a truck and show you the various crops they have. They stop at a few places and give you a bunch of fruit samples, attempt to answer your questions, and take funny pictures of you. The favourite seemed to be to dangle a fruit on a knife up close in front of the camera while having the subject(s) in the background look like they're carrying an enormous fruit. We got back to the car to an upset Sarah and small group of locals milling around. Sarah had managed to lock Zoe in the car for about 30 minutes when she got out to make her a sandwich after Zoe had woken up from her nap. Fortunately the engine was running and the air conditioning was on, so there was no risk of Zoe overheating (on the contrary, she was a bit on the cool side). Some kind locals had somehow figured out which car rental company to call (it's possible there's only one) and someone came out with a spare key, so the crisis was averted just before we got back to the car. It's the first time Zoe's been locked in the car (it's pretty much impossible in the Prius) and it was just our luck that it'd happen in a foreign country! One of the saddest things about Langkawi was the amount of garbage floating around in the ocean, and washing up on the beaches. I was swimming in the ocean off the yacht while we were out on the Buck's party, and it was disgusting. The jetski was temporarily disabled due to sucking in something. We visited Black Sand Beach, which was supposed to be something of a tourist attraction. Covered in trash. Ian Kiernan would be sad. The captain of the yacht we were on for the Buck's party said it was all from Thailand and got flushed out of all the rivers and streams by the recent rain. Just goes to show how connected everything is. The house is very quiet without Sarah and Zoe, but my friend Liam is visiting from Zurich for a couple of weeks, so that's helping break the silence. I'm using the time to get a heap of things done that I don't otherwise get time to get done. Unfortunately the photos from the trip are only available on Facebook at the moment, but they'll eventually get uploaded here.

I just uploaded a snapshot of GNU Screen to Debian Experimental. The package (4.1.0~20110819git450e8f3-1) is based on upstream s HEAD whose most recent commit currently dates to the 19th of August 2011. While the upload fixes tons of bugs which accumulated over the past two years in Debian s, Ubuntu s and upstream s bug tracker, I don t yet regard it as suitable for the next stable release (and hence for Debian Unstable) since there s one not so nice issue about it:

• #644788: screen 4.1.0 can t attach to a running/detached screen 4.0.3 session

Nevertheless it fixes a lot of open issues (of which the oldest is a wishlist bug report dating back to 1998 :-) and I didn t want to withhold it from the rest of the Debian community so I uploaded it to Debian Experimental. Issues closed in Debian Experimental

• #25096: digraph table should be run-time configurable
• #152961: lacks tsl/fsl/dsl caps
• #176626: mini-curses type of interface for screen -r w/ multiple screens? (Fixed by suggesting iselect, screenie or byobu)
• #223320: does not switch mouse mode
• #344759: mishandles xterm control string to set window title
• #353090: please enable the built-in telnet
• #361274: cannot reattach to sessionname if there is another session with similar sessionname
• #450421: please raise MAXWIN to at least 100 (merged with #499273)
• #461107: Requires test -t 0 even when opening a new window on existing screen
• #481411: window created with -d -m silently ignores -X exec
• #488619: Session name string escape
• #496750: screen -d -m and -D -m segfault if setenv given with no value in a configuration file
• #532240: screen with caption SEGVs when resized to 1 line tall
• #541793: C-a h (mis)documented twice
• #558724: breaks altscreen
• #560231: Please remove restriction on user/login name length
• #578729: outputs spaces when refreshing/attaching a window with defbce on
• #591624: segfault when running screen -d -m with layout save default in .screenrc
• #603009: Updating the screen Uploaders list
• #612990: /etc/init.d/screen-cleanup: should check for existence of screen binary
• #621704: Fix slow scrolling in vertical splits
• #630535: manpage typo
• #641867: version bump (this bug report sparked the upload :-)

Issues which will be closed in Ubuntu

• #183849: update to git version of screen
• #315237: crashes with certain options and terminal sizes
• #582153: doesn t accept login names longer than 20 chars
• #588846: slow when using vertical split
• #702094: Copying and pasting from mutt includes many trailing spaces
• #786292: segfaults if using layout saving with -D -m
• #788670: segfault in screen/byobu in natty

Please test the version from Experimental If you are affected by one of the issues mentioned above, please try the version from Debian Experimental and check if they re resolved for you, too. Thanks to all who contributed! A lot of the fixes have been made or applied upstream by Sadrul Habib Chowdhury who also industriously tagged Debian bug reports as fixed-upstream . Thanks! Thanks also to Brian P Kroth who gave the initial spark to this upload by packaging Fedora 15 s git snapshot for Debian and filing bug although the upload is based on the current HEAD version of GNU Screen as this fixes some more important issues than the snapshot Fedora 15 includes. That way also two patches from Fedora/RedHat s screen package are included in this upload. (Co-) Maintainer wanted! Oh, and if you care about the state of GNU Screen in Debian, I d really appreciate if you d join in and contribute to our collab-maint git repository there are still a lot of issues unresolved and I know that I won t be able to fix all of them myself. And since Hessophanes unfortunately currently has not enough time for the package, we definitely need more people maintaining this package. P.S. Yes, I know about tmux and tried to get some of my setups working with it, too. But I still prefer screen over tmux. :-)

Guus strikes back with a new Scientific tutorial in the Free OpenGL wikibook. It deals with tracing the Mexican Hat 3D function using surfaces, removing hidden ones, and also shows how to avoid visual artifacts when adding grid lines.

http://en.wikibooks.org/wiki/OpenGL_Programming/Scientific_OpenGL_Tutorial_05

It's become kind of a clich for me to claim that the reason I'm happy working on ACPI and UEFI and similarly arcane pieces of convoluted functionality is that no matter how bad things are there's at least some form of documentation and there's a well-understood language at the heart of them. My PhD was in biology, working on fruitflies. They're a poorly documented set of layering violations which only work because of side-effects at the quantum level, and they tend to die at inconvenient times. They're made up of 165 million bases of a byte code language that's almost impossible to bootstrap[1] and which passes through an intermediate representations before it does anything useful[2]. It's an awful field to try to do rigorous work in because your attempts to impose any kind of meaningful order on what you're looking at are pretty much guaranteed to be sufficiently naive that your results bear a resemblance to reality more by accident than design.

The field of bioinformatics is a fairly young one, and because of that it's very easy to be ignorant of its history. Crick and Watson (and those other people) determined the structure of DNA. Sanger worked out how to sequence proteins and nucleic acids. Some other people made all of these things faster and better and now we have huge sequence databases that mean we can get hold of an intractable quantity of data faster than we could ever plausibly need to, and what else is there to know?

Margaret Dayhoff graduated with a PhD in quantum chemistry from Columbia, where she'd performed computational analysis of various molecules to calculate their resonance energies[3]. The next few years involved plenty of worthwhile research that aren't relevant to the story, so we'll (entirely unfairly) skip forward to the early 60s and the problem of turning a set of sequence fragments into a single sequence. Dayhoff worked on a suite of applications called "Comprotein". The original paper can be downloaded here, and it's a charming look back at a rigorous analysis of a problem that anyone in the field would take for granted these days. Modern fragment assembly involves taking millions of DNA sequence reads and assembling them into an entire genome. In 1960, we were still at the point where it was only just getting impractical to do everything by hand.

This single piece of software was arguably the birth of modern bioinformatics, the creation of a computational method for taking sequence data and turning it into something more useful. But Dayhoff didn't stop there. The 60s brought a growing realisation that small sequence differences between the same protein in related species could give insight into their evolutionary past. In 1965 Dayhoff released the first edition of the Atlas of Protein Sequence and Structure, containing all 65 protein sequences that had been determined by then. Around the same time she developed computational methods for analysing the evolutionary relationship of these sequences, helping produce the first computationally generated phylogenetic tree. Her single-letter representation of amino acids was born of necessity[4] but remains the standard for protein sequences. And the atlas of 65 protein sequences developed into the Protein Information Resource, a dial-up database that allowed researchers to download the sequences they were interested in. It's now part of UniProt, the world's largest protein database.

Her contributions to the field were immense. Every aspect of her work on bioinformatics is present in the modern day larger, faster and more capable, but still very much tied to the techniques and concepts she pioneered. And so it still puzzles me that I only heard of her for the first time when I went back to write the introduction to my thesis. She's remembered today in the form of the Margaret Oakley Dayhoff award for women showing high promise in biophysics, having died of a heart attack at only 57.

I don't work on fruitflies any more, and to be honest I'm not terribly upset by that. But it's still somewhat disconcerting that I spent almost 10 years working in a field so defined by one person that I knew so little about. So my contribution to Ada Lovelace day is to highlight a pivotal woman in science who heavily influenced my life without me even knowing.

[1] You think it's difficult bringing up a compiler on a new architecture? Try bringing up a fruitfly from scratch.
[2] Except for the cases where the low-level language itself is functionally significant, and the cases where the intermediate representation is functionally significant.
[3] Something that seems to have involved a lot of putting punch cards through a set of machines, getting new cards out, and repeating. I'm glad I live in the future.
[4] The three-letter representation took up too much space on punch cards

comments

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (144.3 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg work While taking care of the last details for the hardening feature in dpkg 1.16.1, I have mailed debian-devel to find volunteers to handle a hardening release goal. The objective is to ensure a large number of packages have been converted/rebuilt to actually use the new hardening build flags. Then I prepared the draft of the announce of the dpkg 1.16.1 upload (aka Bits of dpkg maintainers sent to debian-devel-announce) which got expanded by Guillem to also cover new features since dpkg 1.15.7. update-alternatives got some refactoring by Guillem which resulted in a regression that has been fortunately discovered by Sven Joachim. I fixed that regression and did some further cleanup inspired by the root cause of this regression (see top 4 commits here). Note that Sven is one of the few persons who are running the git version of dpkg. Hopefully the number of tester will increase since I recently documented the APT repositories with autobuilt versions of dpkg in the wiki. At the end of the month, I started working on a bugfix release (what s going to be 1.16.1.1) by fixing some of the unavoidable problems discovered after an upload that accumulated more than 4 months worth of work (see top 4 commits here). The Debian Administrator s Handbook I spent countless hours finalizing the launch of the crowdfunding campaign for the Debian Administrator s Handbook and it went live on September 27th. So far it s on good track with more than 63% of the base funding already secured. But we still have a long way to go to reach the liberation goal (we re at 21%). It s still worth nothing that more than 55% of the money raised has been put in the liberation fund so there are many persons who care about getting the book freed. More than 250 persons are supporting the project currently with an average contribution of 38 EUR. I would have expected much less for the average contribution but many more supporters. I still hope we can get more people on board with the perspective of a good DFSG-free Debian ebook. Did you order your copy? If not, click here and fix this! By the way Paypal used to be required but it s no longer the case, you can support the project just with your usual credit card. Misc blog updates Over time, I have written many useful articles for Debian users and Debian contributors. But scattered in the history, they are somewhat difficult to find. To fix this I have created some index pages listing them. Check them out:

• Mastering Debian
• Contributing to Debian
• Debian Packaging Tutorials

Two new articles joined those pages this month: How to triage bugs in the Debian Bug Tracking System and Understand dpkg and don t get stuck with a maintainer script failure. While writing the first article, I noticed we lacked a good page showing the most buggy packages so I quickly created it (with the help of UDD): http://qa.debian.org/cgi-bin/bugs-by-source Misc packaging work I did a small update to the developer s reference. Luca Falavigna submitted a patch to clarify how one is supposed to deal with meta-packages (cf #569219), I improved it and integrated the result in the SVN repository. I upgraded nautilus-dropbox to version 0.6.9 and while doing this I discovered a bug in mergechanges (filed as #640782). I uploaded a new release of quilt mainly to add the Multi-Arch: foreign field so that it can satisfy dependencies of foreign packages (i.e. packages of a different architecture). Django released some security advisories (tracked in #641405) and since the maintainer did not deal with the issue, I stepped up to the task (I m a backup maintainer) and released the fixed version 1.3.1 to unstable. I took the opportunity to switch from python-support to dh_python2, and do some misc improvements to the packaging (see changelog). I wanted to update publican to a newer version but it turned out to be not possible because Debian doesn t have the latest version of docbook-xsl yet. I also discovered some bugs in the test suite and forwarded upstream the patch I created (see upstream bug). On top of this, fop was failing due to some java problem related to the introduction of multiarch. After having reported the bug, the java maintainers quickly released a fixed version. So now publican is ready in the git repository but it s waiting on the docbook-xsl update. I got in touch with the maintainer who said he would have the time to take care of it by mid-october. Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Would MTAs please stop sending hi-bit7 messages to other MTAs which do not advertise 8BITMIME! Recode it to QP or BASE64, damnit! The receiving MTA is entitled to strip the set bit7, which kinda makes things hard to read (while I know how to deal with blvde Stra_e, the advent of UTF-8 makes that blC6de StraC?e, introduces C0 control characters and makes typographic quotation marks into NUL-containing octet sequences (as their UTF-8 representation contains 0x80 octets) which let every sensible MDA terminate the line there). I even filed in the Debian BTS against the BTS (might be Drexim's fault, though). Would MUAs please default to Quoted-Printable! And mail hosters should use the same server when retrying delivery, to benefit greylisting. Or at least publish a list of outgoing IPv4 addresses they use for sending. Or use IPv6. Oh, and STARTTLS, while we are on my wishlist. It's a sad day when the percentage of correctly encoded eMail messages in my INBOX is smaller than that of my Spambox...

It doesn t look so old in hex; Zero, x, Two, Eight but I finally got there. So on this day, what other landmarks am I up to?

• I ve been programming, on and off, for 28 years (thanks for teaching me Chris anyone still use LOGO?)
• Writing Free Software for 17 years (thanks Terry for explaining the concepts and giving me my first Linux)
• A debian developer for 14 years (thanks for telling me and eventually adding me Bruce)

Some people get a little sad hitting this age, but it really is only a number, wether it is 0 28, \050 or even 40. As the saying goes: only the dead don t age.

Lintian uses some small scripts to collect data from packages. In daily talk, they are usually referred to as collection scripts. Lintian uses files to track the status of collection scripts between runs. Consider the following directory listing on lintian.d.o:

$ ls -al laboratory/binary/eclipse/.*-*
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.ar-info-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.bin-pkg-control-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.changelog-file-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.copyright-file-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.debian-readme-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.doc-base-files-1
-rw-r--r-- 1 user group  45 Jul 18 01:21 laboratory/binary/eclipse/.fields-1
[...]

This example shows that a bunch of collections scripts have been run for the eclipse binary package. Each of these files contain the version of Lintian that created them (or last wrote them) and a timestamp. Why the interest in these files? Let us go a bit back in time to the 15th of June 2011. That was the day where Lintian 2.5.1 was uploaded to unstable. That version of Lintian had 17 collection scripts[1] for binary packages. So every binary package would have 17 files and there are over 35 000 binary packages in the Debian archive. Ouch, so that makes about 595 000 files if we use this on all binary packages on the archive. The size of each of those files are about 45 bytes, so that is a total of 25.5 MB for all of these files[2]. So other than the inode abuse , this is not too bad. A little du -h should confirm this

$ du -h laboratory/binary/eclipse/.*-*
4.0K    laboratory/binary/eclipse/.ar-info-1
4.0K    laboratory/binary/eclipse/.bin-pkg-control-1
4.0K    laboratory/binary/eclipse/.changelog-file-1
4.0K    laboratory/binary/eclipse/.copyright-file-1
4.0K    laboratory/binary/eclipse/.debian-readme-1
4.0K    laboratory/binary/eclipse/.doc-base-files-1
4.0K    laboratory/binary/eclipse/.fields-1
[...]

Whaaa oh the file system uses a block-size of 4K bytes, so I guess we have to pay a full block for these files. Let s see what that gives, 595 000 times 4 kB is 2.27 GB Oops! That was in the (not too distant) past. About a month later (12th of July), the code creating these files are refactored into:

sub _mark_coll_finished  
    my ($self, $collname, $collver) = @_;
    # In the "old days" we would also write the Lintian version and the time
    # stamp in these files, but since we never read them it seems like overkill.
    #  - for the timestamp we could use the mtime of the file anyway
    return touch_file "$self-> base_dir /.$collname-$collver";
 

This turns out to be a very space-saving change if we ask du -h:

$ du -h laboratory/binary/lintian/.*-*
0       laboratory/binary/lintian/.ar-info-1
0       laboratory/binary/lintian/.bin-pkg-control-1
0       laboratory/binary/lintian/.changelog-file-1
0       laboratory/binary/lintian/.copyright-file-1
0       laboratory/binary/lintian/.debian-readme-1
0       laboratory/binary/lintian/.doc-base-files-1
[...]

Existing files are not emptied, so we still have some old non-empty files left on lintian.d.o. Nevertheless that was the "nice story" about "the side effect of being lazy sometimes reduces space waste"[3]. But we have another "number" problem. If you grep for "Too many links" in the lintian.log from lintian.d.o you should see:

$ grep -i "too many links" logs/lintian.log
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/aces3': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/boinc-cgi-stripchart': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/libbrailleutils-java': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/libbrailleutils-java-doc': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/libclippoly0': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/libclippoly-dev': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/xul-ext-cookie-monster': Too many links
mkdir: cannot create directory  /srv/lintian.debian.org/laboratory/binary/cucumber': Too many links
[...]
$ grep -i "too many links" logs/lintian.log   wc -l
3274

In the Lintian Laboratory every package is unpacked in a directory based on its type and name (as hinted in the output above). The problem is that ext3 has a limit of the amount sub-directories a directory can have[4]. This limit is just shy of 32 000 and (as a reminder) Debian has over 35 000 binary packages. So a lot of packages are currently not checked on lintian.d.o (#641468). The current feature branch for #641468 already solves the directory limit issue (by using a "mirror-like" pool layout). It also removes the need for "1 file per completed collection" by writing this information in the existing "per entry" status file.
I hope we can get the branch merged into master within 2-3 weeks, though there are still a few issues that needs to be worked out before then. Footnotes: [1] Actually it had 18 collections, but one of them are "auto-removed" so it is better not to include it in this case. [2] Content alone - metadata like permissions and such is ignored. [3] I admit it was more luck than intention! [4] The limit comes from the "32 000 hardlink per inode" limitation.

I was due for another Google interview mail it seems. I have to say I wasn't expecting it, but this week I had a follow up to my polite mail from 6 months ago that said "No thanks, I'm not looking" asking if it was still the case. Normally I welcome this little bit of ego stroking; it's always nice to be wanted. Except that's not really the case, is it? It's an invitation to interview for something, not any indication that you've done more than tick some initial boxes. Google mails inevitably ask me if I'd like to work in SRE. It's always SRE. No one ever emails and asks if I want to work on self-replicating nanobots that will roam Mars searching for the perfect spot to build a beach house. And that's where things fall down. If someone currently has a job, then emailing them out of the blue to ask if they want to come and interview for something vague is hoping that they're either looking, and just haven't come to you yet, or not looking but unhappy enough with their current role that all they needed to start was an email asking them to submit a recent CV. For the former, if you're Google, do you really think that person doesn't know where to find you? For the latter, you're being quite presumptuous, aren't you? The act of updating my CV my be some effort (actually it's usually not, because the stuff that's not on it is the stuff I can't talk about because it's not released yet, or stuff that's specific and thus wouldn't go on a CV for a vague job spec). Even if it's not the act of interviewing is potentially a waste of time for both of us, if the role isn't clear. One argument used is that people will be placed according to the skills they show during the interview process. That's fine from the employers point of view, but if you're actively trying to get some interest from people who are gainfully employed you really need to grab their attention somehow. I can't remember the last time I had an unsolicited email interview offer that actually wowed me, or indeed even showed more than a passing sign of tailoring a spec to my profile. When I was running Black Cat I made a point of always replying to unsolicited CVs. How polite I was depended on how the covering emails were worded (a Word document with nothing else was likely to get short shrift, something well targeted in a Linux friendly format would normally get some comment about how we weren't hiring and were unlikely to be, but if that changed it would be mentioned on the website), but I felt people deserved a reply - I have been disappointed by not receiving responses myself to what I considered well targeted job inquiries. So far I've so far taken the same approach with mails from corporate recruiters (less so with recruiters that are associated with recruitment firms, rather than directly with the companies they are hiring for), but I'm starting to feel like changing that stance. Candidates are told to tailor CVs to the role being applied for, provide a decent cover letter, and in general make companies want to talk to them. Companies who are sending out recruitment emails should be held to the same standard. Even assuming you do a basic phone screen first, I can probably expect to need to take a day off work assuming that goes well. You need to convince me I can justify that before I'm going to feel like engaging at all. (And if I'm honest, based on what I've seen so far, it's unlikely to happen. All of the things I've considered have come from conversations with people I know directly about companies they own or work for, never some random contact via email. I try hard not to think of recruitment mail as spam, but I can how that line of thought follows through.) I should apologize to Google here. They got mentioned as an example, but I don't think they're particularly bad. I did interview with them at one point, and made the decision not to continue that process after deciding a different, more certain, path was better for me. So I've displayed interest. And in response to my reply today of "I know where to find you, so please assume I'll do so if I change my mind." they've said they'll make a note on their records.

Some years ago I came across a really peculiar newsgroup post. It was not about technicalities of any sort. It was about history. A beautifully written history of computers. From the Turing machine to the Free Software world, the original author managed to capture all the important events of the computer revolution with a great deal of humor. Re-posting it here, it is just brilliant. The Gospel of Tux (v1.0) In the beginning Turing created the Machine. And the Machine was crufty and bodacious, existing in theory only. And von Neumann looked upon the Machine, and saw that it was crufty. He divided the Machine into two Abstractions, the Data and the Code, and yet the two were one Architecture. This is a great Mystery, and the beginning of wisdom. And von Neumann spoke unto the Architecture, and blessed it, saying, Go forth and replicate, freely exchanging data and code, and bring forth all manner of devices unto the earth. And it was so, and it was cool. The Architecture prospered and was implemented in hardware and software. And it brought forth many Systems unto the earth. The first Systems were mighty giants; many great works of renown did they accomplish. Among them were Colossus, the codebreaker; ENIAC, the targeter; EDSAC and MULTIVAC and all manner of froody creatures ending in AC, the experimenters; and SAGE, the defender of the sky and father of all networks. These were the mighty giants of old, the first children of Turing, and their works are written in the Books of the Ancients. This was the First Age, the age of Lore. Now the sons of Marketing looked upon the children of Turing, and saw that they were swift of mind and terse of name and had many great and baleful attributes. And they said unto themselves, Let us go now and make us Corporations, to bind the Systems to our own use that they may bring us great fortune. With sweet words did they lure their customers, and with many chains did they bind the Systems, to fashion them after their own image. And the sons of Marketing fashioned themselves Suits to wear, the better to lure their customers, and wrote grave and perilous Licenses, the better to bind the Systems. And the sons of Marketing thus became known as Suits, despising and being despised by the true Engineers, the children of von Neumann. And the Systems and their Corporations replicated and grew numerous upon the earth. In those days there were IBM and Digital, Burroughs and Honeywell, Unisys and Rand, and many others. And they each kept to their own System, hardware and software, and did not interchange, for their Licences forbade it. This was the Second Age, the age of Mainframes. Now it came to pass that the spirits of Turing and von Neumann looked upon the earth and were displeased. The Systems and their Corporations had grown large and bulky, and Suits ruled over true Engineers. And the Customers groaned and cried loudly unto heaven, saying, Oh that there would be created a System mighty in power, yet small in size, able to reach into the very home! And the Engineers groaned and cried likewise, saying, Oh, that a deliverer would arise to grant us freedom from these oppressing Suits and their grave and perilous Licences, and send us a System of our own, that we may hack therein! And the spirits of Turing and von Neumann heard the cries and were moved, and said unto each other, Let us go down and fabricate a Breakthrough, that these cries may be stilled. And that day the spirits of Turing and von Neumann spake unto Moore of Intel, granting him insight and wisdom to understand the future. And Moore was with chip, and he brought forth the chip and named it 4004. And Moore did bless the Chip, saying, Thou art a Breakthrough; with my own Corporation have I fabricated thee. Thou thou art yet as small as a dust mote, yet shall thou grow and replicate unto the size of a mountain, and conquer all before thee. This blessing I give unto thee: every eighteen months shall thou double in capacity, until the end of the age. This is Moore s Law, which endures unto this day. And the birth of 4004 was the beginning of the Third Age, the age of Microchips. And as the Mainframes and their Systems and Corporations had flourished, so did the Microchips and their Systems and Corporations. And their lineage was on this wise: Moore begat Intel. Intel begat Mostech, Zilog and Atari. Mostech begat 6502, and Zilog begat Z80. Intel also begat 8800, who begat Altair; and 8086, mother of all PCs. 6502 begat Commodore, who begat PET and 64; and Apple, who begat 2. (Apple is the great Mystery, the Fruit that was devoured, yet bloomed again.) Atari begat 800 and 1200, masters of the game, who were destroyed by Sega and Nintendo. Xerox begat PARC. Commodore and PARC begat Amiga, creator of fine arts; Apple and PARC begat Lisa, who begat Macintosh, who begat iMac. Atari and PARC begat ST, the music maker, who died and was no more. Z80 begat Sinclair the dwarf, TRS-80 and CP/M, who begat many machines, but soon passed from this world. Altair, Apple and Commodore together begat Microsoft, the Great Darkness which is called Abomination, Destroyer of the Earth, the Gates of Hell. Now it came to pass in the Age of Microchips that IBM, the greatest of the Mainframe Corporations, looked upon the young Microchip Systems and was greatly vexed. And in their vexation and wrath they smote the earth and created the IBM PC. The PC was without sound and colour, crufty and bodacious in great measure, and its likeness was a tramp, yet the Customers were greatly moved and did purchase the PC in great numbers. And IBM sought about for an Operating System Provider, for in their haste they had not created one, nor had they forged a suitably grave and perilous License, saying, First we will build the market, then we will create a new System, one in our own image, and bound by our Licence. But they reasoned thus out of pride and not wisdom, not forseeing the wrath which was to come. And IBM came unto Microsoft, who licensed unto them QDOS, the child of CP/M and 8086. (8086 was the daughter of Intel, the child of Moore). And QDOS grew, and was named MS-DOS. And MS-DOS and the PC together waxed mighty, and conquered all markets, replicating and taking possession thereof, in accordance with Moore s Law. And Intel grew terrible and devoured all her children, such that no chip could stand before her. And Microsoft grew proud and devoured IBM, and this was a great marvel in the land. All these things are written in the Books of the Deeds of Microsoft. In the fullness of time MS-DOS begat Windows. And this is the lineage of Windows: CP/M begat QDOS. QDOS begat DOS 1.0. DOS 1.0 begat DOS 2.0 by way of Unix. DOS 2.0 begat Windows 3.11 by way of PARC and Macintosh. IBM and Microsoft begat OS/2, who begat Windows NT and Warp, the lost OS of lore. Windows 3.11 begat Windows 95 after triumphing over Macintosh in a mighty Battle of Licences. Windows NT begat NT 4.0 by way of Windows 95. NT 4.0 begat NT 5.0, the OS also called Windows 2000, The Millenium Bug, Doomsday, Armageddon, The End Of All Things. Now it came to pass that Microsoft had waxed great and mighty among the Microchip Corporations; mighter than any of the Mainframe Corporations before it had it waxed. And Gates heart was hardened, and he swore unto his Customers and their Engineers the words of this curse: Children of von Neumann, hear me. IBM and the Mainframe Corporations bound thy forefathers with grave and perilous Licences, such that ye cried unto the spirits of Turing and von Neumann for deliverance. Now I say unto ye: I am greater than any Corporation before me. Will I loosen your Licences? Nay, I will bind thee with Licences twice as grave and ten times more perilous than my forefathers. I will engrave my Licence on thy heart and write my Serial Number upon thy frontal lobes. I will bind thee to the Windows Platform with cunning artifices and with devious schemes. I will bind thee to the Intel Chipset with crufty code and with gnarly APIs. I will capture and enslave thee as no generation has been enslaved before. And wherefore will ye cry then unto the spirits of Turing, and von Neumann, and Moore? They cannot hear ye. I am become a greater Power than they. Ye shall cry only unto me, and shall live by my mercy and my wrath. I am the Gates of Hell; I hold the portal to MSNBC and the keys to the Blue Screen of Death. Be ye afraid; be ye greatly afraid; serve only me, and live. And the people were cowed in terror and gave homage to Microsoft, and endured the many grave and perilous trials which the Windows platform and its greatly bodacious Licence forced upon them. And once again did they cry to Turing and von Neumann and Moore for a deliverer, but none was found equal to the task until the birth of Linux. These are the generations of Linux: SAGE begat ARPA, which begat TCP/IP, and Aloha, which begat Ethernet. Bell begat Multics, which begat C, which begat Unix. Unix and TCP/IP begat Internet, which begat the World Wide Web. Unix begat RMS, father of the great GNU, which begat the Libraries and Emacs, chief of the Utilities. In the days of the Web, Internet and Ethernet begat the Intranet LAN, which rose to renown among all Corporations and prepared the way for the Penguin. And Linus and the Web begat the Kernel through Unix. The Kernel, the Libraries and the Utilities together are the Distribution, the one Penguin in many forms, forever and ever praised. Now in those days there was in the land of Helsinki a young scholar named Linus the Torvald. Linus was a devout man, a disciple of RMS and mighty in the spirit of Turing, von Neumann and Moore. One day as he was meditating on the Architecture, Linus fell into a trance and was granted a vision. And in the vision he saw a great Penguin, serene and well-favoured, sitting upon an ice floe eating fish. And at the sight of the Penguin Linus was deeply afraid, and he cried unto the spirits of Turing, von Neumann and Moore for an interpretation of the dream. And in the dream the spirits of Turing, von Neumann and Moore answered and spoke unto him, saying, Fear not, Linus, most beloved hacker. You are exceedingly cool and froody. The great Penguin which you see is an Operating System which you shall create and deploy unto the earth. The ice-floe is the earth and all the systems thereof, upon which the Penguin shall rest and rejoice at the completion of its task. And the fish on which the Penguin feeds are the crufty Licensed codebases which swim beneath all the earth s systems. The Penguin shall hunt and devour all that is crufty, gnarly and bodacious; all code which wriggles like spaghetti, or is infested with blighting creatures, or is bound by grave and perilous Licences shall it capture. And in capturing shall it replicate, and in replicating shall it document, and in documentation shall it bring freedom, serenity and most cool froodiness to the earth and all who code therein. Linus rose from meditation and created a tiny Operating System Kernel as the dream had foreshewn him; in the manner of RMS, he released the Kernel unto the World Wide Web for all to take and behold. And in the fulness of Internet Time the Kernel grew and replicated, becoming most cool and exceedingly froody, until at last it was recognised as indeed a great and mighty Penguin, whose name was Tux. And the followers of Linus took refuge in the Kernel, the Libraries and the Utilities; they installed Distribution after Distribution, and made sacrifice unto the GNU and the Penguin, and gave thanks to the spirits of Turing, von Neumann and Moore, for their deliverance from the hand of Microsoft. And this was the beginning of the Fourth Age, the age of Open Source. Now there is much more to be said about the exceeding strange and wonderful events of those days; how some Suits of Microsoft plotted war upon the Penguin, but were discovered on a Halloween Eve; how Gates fell among lawyers and was betrayed and crucified by his former friends, the apostles of Media; how the mercenary Knights of the Red Hat brought the gospel of the Penguin into the halls of the Corporations; and even of the dispute between the brethren of Gnome and KDE over a trollish Licence. But all these things are recorded elsewhere, in the Books of the Deeds of the Penguin and the Chronicles of the Fourth Age, and I suppose if they were all narrated they would fill a stack of DVDs as deep and perilous as a Usenet Newsgroup. Now may you code in the power of the Source; may the Kernel, the Libraries and the Utilities be with you, throughout all Distributions, until the end of the Epoch. Amen.

Arcball is a conceptual sphere that users of your application can manipulate to rotate objects with the mouse. Read this new OpenGL wikibook tutorial!

http://en.wikibooks.org/wiki/OpenGL_Programming/Modern_OpenGL_Tutorial_Arcball