Wouter described how to get Akonadi (the back-end for KDE PIM) to use PostgreSQL [1]. I don t agree with his MySQL is a toy sentiment. But inspired by his post I decided to convert some of my systems to use a MySQL instance running on a server instead of one instance for each user. In the default configuration you have 140M of disk space and 200M of RAM used by each user for a private MySQL installation which has about 24K of data (at least at the moment on systems I run, maybe more in future). Here s some pseudo shell script to dump the database and get a new config: mysqldump socket=$HOME/.local/share/akonadi/socket-$HOSTNAME/mysql.socket akonadi > dump.sql
akonadictl stop
rm -rf .config/akonadi
rm -rf .local/share/akonadi
mkdir .config/akonadi
cat > .config/akonadi/akonadiserverrc <<EOF
[%General]
Driver=QMYSQL
SizeThreshold=4096
ExternalPayload=false

[QMYSQL]
Name=$USER_akonadi
Host=IP_OR_HOST
User=$USER
Password=$PASS
StartServer=false
Options=
ServerPath=/usr/sbin/mysqld

[Debug]
Tracer=null
EOF Then with DBA privs you need to run the following in the mysql client: create database $USER_akonadi;
GRANT ALL PRIVILEGES ON $USER_akonadi.* to $USER @'% IDENTIFIED BY $PASS ; Then run the following to import the SQL data: mysql $USER_akonadi < dump.sql Ideally that would be it, but on my test installation (Debian/Squeeze MySQL server and Debian/Unstable KDE workstations) I needed to run the following SQL commands to deal with some sort of case problem.
rename table schemaversiontable to SchemaVersionTable;
rename table resourcetable to ResourceTable;
rename table collectionattributetable to CollectionAttributeTable;
rename table collectionmimetyperelation to CollectionMimeTypeRelation;
rename table collectionpimitemrelation to CollectionPimItemRelation;
rename table collectiontable to CollectionTable;
rename table flagtable to FlagTable;
rename table mimetypetable to MimeTypeTable;
rename table parttable to PartTable;
rename table pimitemflagrelation to PimItemFlagRelation;
rename table pimitemtable to PimItemTable; I am not using any PIM features other than the AddressBook (which hasn t been working well for a while), so I m not sure that this is working correctly. But I would prefer that something I don t use and is probably broken take up disk space and RAM on a server instead of a workstation anyway.

• [1] http://grep.be/blog/en/computer/debian/no_more_mysql?show_comments=yes

This morning, when turning on TV, my ears have been yet another time hurt by the stupidity of a self-appointed economist, trying to intoxicate people with his fantasies about economy being able to regulate itself. Seriously. All neoliberals, please shut the fuck up. Now. The world has been running out of your ideas for more than 30 years. We lowered salaries and taxes, making the world run through credit. We gave everything to those who were already born with everything. Economy was almost brought to the verge of implosion thanks to your crazy ideas. For 4 years since the subprimes crisis, you psychopaths have kept on explaining it happened because we have not listened to you enough, and that the solution is to lower salaries even more. A quick look at the situation in Greece and UK should be enough to understand where this is going. It is time for something new. So shut up now, and let others fix the mess you have left behind.

I was expecting to write more about the 7th Rugby World Cup, but real life, Debian work and running activities prevented me to do so.. Still, I watched several games and I can share my feelings now. First of all, this is a tremendous organization from New Zealand. It seems that about the entire country is working on making this even a great success and I really appreciate to see a place I certainly have to visit some day receiving such wordlwide attention (OK, admitedly, mostly in the part of the world that understands rugby). First round already gave several surprises even if the 1/4 finals were after all kinda expected (at least, the list of countries).

• New Zealand All Blacks is by far the most shiny team. They crushed all their opponents, including our French team, and were indeed brilliant. So, really, the challenge is: who can beat them?
• Australia and South Africa were about the same. Australia has been surprised by Ireland (one of the most exciting games of first round) and is apparently no longer the incredible team that scored over 60 points in France back in November 2010. South Africa has been strong and powerful during 1st round... but, indeed, not that impressive (I'm not fond of south-african game, anyway).
• Argentina was again great. They had to qualify in a quite hard pool and made it well. Funny to see that, at many moments, the 15 players on the field were players....that usually play in French club teams. They were even sometimes talking in French with some referees (either the excellent Romain Poite, from France, and the also excellent M. Rolland, the most french Irish referee)
• Wales has been serious and strong enough to qualify in a not so easy pool. Their rear lines are the most dangerous among qualified teams, along with NZ
• Ireland made an excellent pool game. Their win against Australia was really a surprise and a nice illustration of what "Irish fighting spirit is". I'm not always fond of Irish rugby organisation (provinces only playing4 to 5 really serious games in a season), but it gives results
• Scotland is really reviving. OK, they didn't qualify, but this team is now far from the very bad team they had during the 2000 years. Rear players as well as their 2m-tall front player (whose name I can't remember now) are excellent
• England. Ah, England..:-). Seeing our "best enemy" in rugby being not so good was kinda a pleasure for me..:-). I just feel sad to see Mister Johnny Wilkinson in not so good shape, because that player is really someone I have respect for. Others...I have less respect, sometimes, I admit (particularly this Ashton guy who I expect to crash in the grass, some day, when arrogantly diving when scoring a try). OK, that's England. Always running love and hate story..:-)
• Fiji, Tonga and Samoa weren't as decisive and challenging as they used to be. Out of these three "Pacific Islands" countries, the Tonga team was IMHO the best and their match against France was a monument of courage. They would even have deserved to qualify and we have to thank Canada for beating them..:-)
• Italy was a deception for me. Their key players are aging and they have hard times renewing the team. And now, definitely, having a good scrum is not enough.
• Among the "have no chance to qualify" teams, I have a good impression of Canada. With so few players in the country, it is amazing to see that they can build a team able to beat Tonga and to play a great game against France. Japan also made at least one excellent game against France. I haven't seen many others, sorry for that.
• And how about France? I have to say they have been very French..:-). The pool round was nearly a disaster with very few good spirit in this team, not even a good scrum and way too many imprecisions and approximations. Still, they qualified.

Which lead us to the following 1/4 finals (in parenthesis are my bets):

• Wales-Ireland (Ireland)
• England-France (England)
• Australia-South Africa (South Africa)
• New Zealand-Argentina (New Zealand)

If you know about the results, you know I screwed it nearly completely..:-) The Welsh team played a great game against an uninspired Irish team. It has certainly been the best 1/4 final and they certainly deserve their win. Even though I usually tend to be supprotive of Ireland, I was very balanced here, and finally turned out to be in favor of the Welsh. We really have to fear them in semi-finals. Australia-South Africa was theoretically the most exciting 1/4 final but finally turned out to be quite boring. Both teams insisted on playing mostly to occupy their opponents part of the field, more than trying to score, then relying on penalties to score. That was apparently the good tactics for Australia ad they also deserve their win against a South-African team where forward players were not as decisive as they sometimes are. Argentina was again there and really there. It has been the only team up to now who lead score against New Zealand. And that was deserved. What a wonderful 1st halftime! Obviously, it was impossible for them to resist during second halftime and it slowly became obvious that the Blacks (saved during 1st half by the kicks of a very inspired Kiri Weepu) would finally manage to score tries. But, still, our argentinian friends, for instance the inoxydable "Super Mario" Ledesma, or the tireless Felipe Contepomi, were not here as a sacrificial victim. What to say about England-France? First half was astonishing for us, of course. This is what we love (and hate) with our beloved French team. Definitely the team that can make surprises and, imho, the only one that can beat New Zealand if that has to happen (but also the only one that can be entirely crushed by them). The defeat against Tonga and the week that followed completely transformed them. A stunning 3rd row, defending each and every single bit of England trying to invade "la patrie en danger". Rear lanes with the magicians of Toulouse (Clerc and M dard) as the ideal finishers of magic play by Parra, Trinh Duc, Mermoz, Palisson (the good surprise of this world cup, Alexis). And, during second half, a trilling resistance to assaults of the British White Knights, concluded by this delivering drop-goal by Trinh Duc. For sure, with games like this, they can beat everybody and by everybody, I mean everybody. Remember Millenium Stadium in 2007..:-) So, well. Australia-New Zealand and France-Wales. I know where my heart is balancing for both games. The Blacks and Les Bleus in final, thi is what we hope (and fear...), but both teams, particularly France, will have to first climb a quite big wall before reaching this.

A month ago, I published an article on the compared performances of stunnel, nginx and stud as SSL terminators. The conclusion was to use stud on a 64-bit system, with session caching and AES. stunnel was unable to scale properly and nginx exhibited important latency issues. I got constructive comments on many aspects. Therefore, here is the second round. The protagonists are the same but both the conditions and the conclusions have changed a bit.

Benchmarks The hardware platform remains unchanged:

• HP DL 380 G7
• 6 GB RAM
• 2 L5630 @ 2.13GHz (8 cores), hyperthreading disabled
• 2 Intel 82576 NIC
• Spirent Avalanche 2900 appliance

We focus on the following environment:

• Ubuntu Lucid, featuring a patched version of OpenSSL 0.9.8k with AES-NI builtin and enabled by default
• 2.6.39 kernel (250 HZ)
• 64-bit
• 1 core for HAProxy, 2 cores for network cards¹, 4 cores for SSL, 1 core for the system
• system limit of 100,000 file descriptors per process

On the SSL side, the main components are:

• 2048-bit certificates
• AES128-SHA cipher suite
• 8 HTTP/1.0 requests per client
• HTTP response payload of 1024 bytes

If you are interested in different conditions (32-bit, number of cores, 1024-bit certificates, proportion of session reuses), have a look at the previous article whose results on the matter are still valid. I only consider 2048-bit certificates because most certificates issued today are 2048-bit. In SP800-57, NIST advises that 1024-bit RSA keys will no longer be viable after 2010 and advises moving to 2048-bit RSA keys:

If information is initially signed in 2009 and needs to remain secure for a maximum of ten years (i.e., from 2009 to 2019), a 1024-bit RSA key would not provide sufficient protection between 2011 and 2019 and, therefore, it is not recommended that 1024-bit RSA be used in this case.

And the protagonists are:

• stud, @e207dbc5a3, with session cache enabled for 20,000 sessions, listen backlog of 1000 connections,
• stunnel 4.44, configured with --disable-libwrap --with-threads=pthread (configuration),
• nginx 1.1.4, configured with --with-http_ssl_module (configuration).

UPDATED: Configuration of nginx has been updated to switch to off both multi_accept and accept_mutex thanks to the feedback of Maxim Dounin on latency issues that were present in the previous article (and still present in the early version of this article). HAProxy (configuration) is used for stunnel and stud to add load-balancing. For each test, we want to maximize the number of HTTP transactions per second (TPS) while minimizing latency and unsucessful transactions. The maximum number of allowed failed transactions is set at 1 transaction out of 1000. The maximum allowed average latency is 100 ms².

Control case Below is the first comparison between stud, nginx and stunnel on plain Ubuntu Lucid. Like in previous benchs, stunnel is not able to scale, with a maximum of 400 TPS. However, early latency issues of nginx has been solved, thanks to the change in configuration. stud and nginx gets similar performance but, as you can see on the bottom plot (whose scale is logarithmic) stud starts with an initial latency of 40 ms. This is pretty annoying. I have tried some micro optimization to solve this issue but without luck. If latency matters for you, stick with stunnel or nginx (or find where the problem lies in stud). UPDATED: After reporting this issue, a fix has been commited. The culprit was Nagle s algorithm which improves bandwidth at the expense of latency. With the fix, performances are mostly the same but there is no latency penalty any more.

OpenSSL 1.0.0e When I published the first round of benchmarks, Micha Trojnara was dubious about the bad results of stunnel. He told me on Twitter:

I solved it! You compiled stunnel against a 4 years old OpenSSL with MT-unsafe SSL_accept. Upgrade OpenSSL and rebuild stunnel. This bug was fixed in OpenSSL 1.0.0b. I implemented a (slow) workaround, so stunnel doesn t crash with earlier versions of OpenSSL.

Therefore, I backported Ubuntu Oneiric OpenSSL 1.0.0e to Ubuntu Lucid³ and, as you can see yourself on the plot below, there is a huge performance boost for stunnel: about 400%! stud features a slight improvement too (about 8%) but still starts with a latency of 40 ms while nginx does not improve.

Intel Accelerator Engine Most Linux distributions are still using OpenSSL 0.9.8. For example, Debian Squeeze ships with OpenSSL 0.9.8o, Ubuntu Lucid with 0.9.8k and RHEL 5.7 with 0.9.8e. Therefore, they are not eligible to recent performance improvements. To solve this, Intel released Intel Accelerator Engine, which also brings, among other things, AES-NI support:

This is essentially a collection of assembler modules from development OpenSSL branch, packed together as a standalone engine. Standalone means that it can be compiled outside OpenSSL source tree without patching the latter. Idea is to provide a way to utilize new code in already released OpenSSL versions constrained by support and release policies limiting their evolvement to genuine bug fixes.

Alas, in my tests, there was no improvement compared to OpenSSL 1.0.0e (already patched with AES-NI support). stunnel comes with out-of-the-box support for additional engines. Just add something like this to your stunnel.conf:

engine=dynamic
engineCtrl=SO_PATH:/WOO/intel-accel-1.4/libintel-accel.so
engineCtrl=ID:intel-accel
engineCtrl=LIST_ADD:1
engineCtrl=LOAD
engineCtrl=INIT

For stud, you should grab support for engine selection from GitHub and for nginx, just specify the engine with ssl_engine.

Intel Performance Primitives In his great article Accelerated SSL , Simon Kuhn shows huge performance improvements by using Intel s IPP library, a library aimed at leveraging power of multi-core processors and including some cryptographic primitives and a patch to make use of them in OpenSSL. Unfortunately, this library is not free, not even as in free beer . The patch provided by Intel only applies on OpenSSL 0.9.8j. Simon updated it for OpenSSL 1.0.0d. It also applies with a slight modification to OpenSSL 1.0.0e. Look at Simon s article for instructions on how to compile OpenSSL with IPP. Simon measured 220% improvement on RSA 1024-bit private key operations by using this library. Unfortunately, we don t get such a performance burst with stud which exhibits a 15% improvement (but remember that private key operations only happen once every 8 transactions thanks to session reuse). Worst, with stunnel, performances are reduced by one third and nginx just segfaults. I think such a library is risky to use in production.

Hyperthreading During the previous tests, only half the capacity of each core were used. For stud, adding more workers per core only slightly improves this situation (less than 10% with 4 workers per core). Therefore, I thought using hyperthreading could rectify the situation here. In this case, stud, stunnel and nginx are allocated 8 virtual cores instead of 4. stud only got a small improvement of 5%, nginx a moderate improvement of 11% while stunnel got a significant improvement of 26%.

Conclusion Here is a quick recap of the maximum TPS reached on each tests (while keeping average response time below 100 ms). All tests were run on 4 physical cores with 2048-bit certificates, AES128-SHA cipher suite. Each client makes 8 HTTP requests and reuse the first SSL session.

Context	nginx 1.1.4	stunnel 4.44	stud @e207dbc5a3
OpenSSL 0.9.8k	1489 TPS	400 TPS	1492 TPS
OpenSSL 1.0.0e	1501 TPS	1561 TPS	1609 TPS
OpenSSL 1.0.0e + IPP	-	1101 TPS	1857 TPS
OpenSSL 1.0.0e + HT	1718 TPS	1973 TPS	1694 TPS

I also would like to stress that below 1500 TPS, stunnel adds an overhead of a few milliseconds while stud adds 40 milliseconds. With OpenSSL 1.0.0e and hyperthreading enabled, stunnel has both good performance and low latency on low loads to moderate loads. You may just want to limit the number of allowed connections to avoid collapsing on high loads. While in the previous round, stud was the champion and stunnel lagged behind, the situation is now reversed : stunnel is on top! UPDATED: As said earlier, the latency issue with stud has been solved. Performances remain mostly the same.

---

1. HAProxy and network cards don t need dedicated core for this kind of load. However, if you also want to handle non SSL traffic, those dedicated cores can get busy.
2. The real condition is more complex. Each point is considered valid if the corresponding average latency is less than 100 ms or if the average on the next 8 points is less than 100 ms. Temporary latency bursts are therefore allowed. We are not so forgiveful with failed transactions.
3. This is pretty straighforward: I just dropped multiarch support. I chose to use Ubuntu version of OpenSSL instead of upstream because there are some interesting patches, like automatic support of AES-NI.

As somebody who often watches movies and TV shows I have the recurring problem of finding the right English subtitles for the video I am about to enjoy. Now, that is really a tedious thing to do. You go on a website such as opensubtitles.org and type the title of what you are looking for. Then you usually have to choose from a plethora of results, download the archive, unzip it, fire up VLC, go to Video Subtitles Track Open File, and finally open your subtitles file. And what you typically find out at that point is that the subtitles are completely out of sync with the audio, but rather than doing the whole process again you prefer to just sync them with a bit of VLC-fu. Which always takes longer than you thought, but hey. Now, what if there was an extremely simple way to just choose the subtitles for a given movie directly from VLC? The good news is that somebody wrote a beautiful VLC extension that does exactly that. He also wrote an introductory article about VLC extensions. Thank you, Jean-Philippe Andr . Installing the thing is as simple as downloading the subtitles extension and putting it in the VLC scripts folder. On Linux machines, the extensions folder can be found at ~/.local/share/vlc/lua/extensions/, on Windows APPDATA\VLC\lua\extensions. But seriously, don t use Windows. Once the extension is installed you can start watching a movie and then click View Subtitles to open the subtitles search interface. This is how it looks like: Now that the practical problem is solved, please take some time to look at the source code. VLC uses Lua as the language for writing extensions. I certainly cannot say that the web is full of resources about how to extend VLC using Lua, but the piece of code I wrote about in this post is certainly a good way to start hacking, and maybe scratch yet another itch.

Our friends Andrew and Janice finally tied the knot last Sunday, in Langkawi, Malaysia. They're Aussies currently living in New York, and Janice is of Malaysian descent, so I think part of the reason was to accommodate some Malaysian relatives that wouldn't be able to travel, and the other reason was that it was going to mean a bunch of people had to travel anyway, so why not travel somewhere a bit exotic? As my sister is getting married later this month in Brisbane, we decided to spare Zoe two massive international flights in close succession, and Sarah and Zoe flew out a few days ahead of me, and went to Brisbane, and then Zoe, Sarah and her mum flew to Langkawi from there, and I flew there from San Francisco, with the plan being for Sarah and Zoe to hang out in Brisbane for the 4 weeks in between weddings. It was the longest amount of travel I think I've ever done. I hadn't sat down and calculated what the travel time was beforehand, and when I got on the flight from San Francisco to Hong Kong, and they said it was going to be nearly 14 hours just to Hong Kong, I nearly wept. The only good thing about it was I didn't have to wrangle a toddler as well. From there, I had to get to Singapore, with a several hour layover, and then another flight from Singapore to Langkawi. I left SFO at 1:20am on Monday morning, and finally arrived at about 7am (San Francisco time) the following day. I didn't even bother getting off the plane in Hong Kong, it was only an hour and a half layover. I think the layover in Singapore was 5 hours, and it was a separate flight. Singapore airport at least was pretty good to kill time in. It was funny, I was reading Wired while I was killing time in Singapore, and they had an article, which included a side bar on 10 Best Airports to Get Stuck In, and I observed that getting to Langkawi and back involved using 4 of them. The flight into Langkawi was more eventful than usual. I was dozing a bit as we were coming in to land, and I woke up to the sensation of taking off again. I wondered if we'd had an aborted landing? Sure enough, soon after, the captain got on the public address and said that they'd had to abort the landing due to bad weather in Langkawi. He had pretty bad Engrish, thanking us for our frustration and saying something about fuel, so I didn't get a lot of the details. We proceeded to circle for a while before finally landing. I think the flight ended up landing something like an hour or more later than scheduled. So that was mildly exciting. Langkawi airport was one of those little "walk across the tarmac" places. I think I stood the closest to an operating jet engine that I ever have, as another aeroplane taxied in to a stop between my plane and the terminal while I was walking to it. Loud. Anyway, that was all just me getting there. Sarah had a bit of fun getting to Brisbane on her own with Zoe. She's now too long to sleep in an in-flight bassinet, and doesn't seem to like sleeping vertically in the Ergo Baby Carrier (possibly she was too hot in it). I think she got a bit of sleep lying across Sarah's lap, but Sarah didn't get much sleep. I think there was similar fun for the Brisbane to Kuala Lumpur flight, but at least she had two laps to lie across for that one. Sarah, Zoe and Sarah's mum arrived in Langkawi earlier in the day on Tuesday, and I made it to the resort at about 10pm (local time) and pretty much immediately crashed for the night. The Westin Langkawi was just fabulous. I don't think we've stayed at a resort since our honeymoon, and this resort was a whole other level compared to Heron Island. The rooms were very comfortable, and the resort was very accommodating of a toddler. Their Kids Club menu was excellent. The breakfast buffet (included in our room rate) was the most comprehensive breakfast buffet I have ever seen in a hotel. The only thing missing was bacon, being a Muslim country, but I can live without that. It was pretty easy to just fill up on breakfast, skip lunch and just have a light dinner. The on-resort dining prices were also very reasonable. The off-resort prices were even cheaper. The resort facilities in general were very comprehensive. Three pools. A children's play room. A well equipped gym and recreational area including squash courts. An extremely nice day spa. The thing I liked best about the whole destination wedding thing was all of the activities leading up to the actual wedding. The wedding was on the Sunday, and we had the Hen's and Buck's parties on Friday. Apparently Australia is the only place that calls it a Buck's party. The rest of the world calls it a Stag party, how about that? There was some last-minute changes to the Buck's party, and we ended up chartering this huge $2 million yacht for 4 hours and going for a cruise and doing some jetskiing and water sports. The captain was Aussie, so we all had a good chat with him. Apparently it's a corporate vessel, with an exclusive chartering arrangement with the Westin. On Friday night, we all went out to a seafood buffet dinner at the Yellow Cafe over at Cenang Beach. Most of the guys and girls already knew each other courtesy of the day's activities, so dinner involved a lot of meeting the other halves. What I really liked about all of the pre-wedding stuff was that by the time the wedding rolled around, you really felt like you knew a bunch of Andrew and Janice's family and friends, which I think made the whole thing feel a lot more intimate. Saturday was a recovery day. Sarah's aunties Glenda (who was in Singapore on business) and Vicki (who lives in Thailand) had joined us at the resort by this stage, so we were all knocking around together checking out the island. We took the cable car to the top, but unfortunately the sky bridge was closed due to the inclement weather. There was a lot of touristy stuff at the bottom of where the cable car was, and Zoe got to have an encounter with a very large python. She even tried to give it a kiss. One of the things that was surprising was how popular Zoe was with all of the locals and other Asian tourists. We could probably have charged for photographs. There was a little girl, who I thought was possibly a similar age to Zoe (turned out she was two and a half) and I asked her parents if I could take a photo of the two girls together, and no sooner had they sat down together and a virtual paparazzi materialised out of nowhere taking photographs. Similarly, all of the staff at the Westin were totally taken by Zoe, and wanted to touch her. To her credit, Zoe handled all of the extra attention pretty well. The wedding wasn't until 6pm on Sunday, so we did a 4 hour island hopping boat tour in the morning. That was a bit of an experience. It was nothing short of sheer bedlam trying to board the right boat at the wharf (I use the term lightly) and then the boat's captain had to arrange the passengers appropriately so as to keep the boat on an even keel. It was a narrow motor boat with a canopy cover over the back two thirds of it. First stop was Dayang Bunting Island, home of Pregnant Maiden Lake, where we stopped for 45 minutes. More bedlam ensued getting off the boat at the wharf there (there was veritable flotilla of tour boats all trying to load and unload at the same time). Monkeys abounded. They seemed incredibly tame. We were practically tripping over them. Zoe didn't seem to be as taken by the monkeys as I thought she would. It was a short walk up a reasonable flight of stairs to get to the lake, though some incredibly humid jungle, and once we got there, there wasn't a huge amount to do in the time we had available. So we dunked Zoe's feet in the lake a few times and then made our way back to the chaos at the wharf and got back on our boat. We stopped for a similar amount of time at another island with a pretty nice beach, and Zoe had enough time to a quick swim in the ocean with Sarah, and then we made our way back to the boat again to another spot where we watched sea eagle feeding. Heaps of eagles circling overhead would take turns to swoop down and grab fish and fly off again. It was impressive. I think I managed to snap some good action shots with the long lens. Then we had the wedding itself. It was on the beach out the front of the Heavenly Spa on the resort. Let me say, the Westin knows how to put a wedding together. The ceremony was as lovely as the location. For the reception, the one 5 bedroom villa on the resort was booked for the night. Dinner was a buffet on the lawn out the front (briefly delayed thanks to a rain shower) and then speeches and general partying happened around the terrace out the back of the villa. There were some beautiful speeches. Janice's brought a tear to my eye. Part of the whole package included something like a 10 person video crew that were running around capturing everything. It felt a bit surreal. It was definitely the most high-end wedding I've ever been to (sorry Scott and Julie, your "Wedding of the Century" has been eclipsed by the "Wedding of the Millennium"). Monday was a wind down day. Zoe mercifully gave us a sleep in until 8am. Our flights didn't leave until the evening on Tuesday, so after we checked out we went for a drive around the island to try and get Zoe to take a nap in the car. We ended up at the fruit farm, which to the best of my understanding is run by the government. Zoe had fallen asleep by this stage, and Sarah said she'd stay behind in the car and have a nap too. So her mum and I went on the tour. It was a fun little tour. There were only four of us on it. They drive you around the farm on the back of a truck and show you the various crops they have. They stop at a few places and give you a bunch of fruit samples, attempt to answer your questions, and take funny pictures of you. The favourite seemed to be to dangle a fruit on a knife up close in front of the camera while having the subject(s) in the background look like they're carrying an enormous fruit. We got back to the car to an upset Sarah and small group of locals milling around. Sarah had managed to lock Zoe in the car for about 30 minutes when she got out to make her a sandwich after Zoe had woken up from her nap. Fortunately the engine was running and the air conditioning was on, so there was no risk of Zoe overheating (on the contrary, she was a bit on the cool side). Some kind locals had somehow figured out which car rental company to call (it's possible there's only one) and someone came out with a spare key, so the crisis was averted just before we got back to the car. It's the first time Zoe's been locked in the car (it's pretty much impossible in the Prius) and it was just our luck that it'd happen in a foreign country! One of the saddest things about Langkawi was the amount of garbage floating around in the ocean, and washing up on the beaches. I was swimming in the ocean off the yacht while we were out on the Buck's party, and it was disgusting. The jetski was temporarily disabled due to sucking in something. We visited Black Sand Beach, which was supposed to be something of a tourist attraction. Covered in trash. Ian Kiernan would be sad. The captain of the yacht we were on for the Buck's party said it was all from Thailand and got flushed out of all the rivers and streams by the recent rain. Just goes to show how connected everything is. The house is very quiet without Sarah and Zoe, but my friend Liam is visiting from Zurich for a couple of weeks, so that's helping break the silence. I'm using the time to get a heap of things done that I don't otherwise get time to get done. Unfortunately the photos from the trip are only available on Facebook at the moment, but they'll eventually get uploaded here.

another week, another list of fixed RC bugs, again most of them with patches from ubuntu. &, as zack said today, the next freeze is not really imminent but it's getter closer.

• #625002 src:ctemplate: "ctemplate: FTBFS: ./src/ctemplate/template_dictionary.h:73:11: error: 'ptrdiff_t' does not name a type"
  add patch from Ubuntu / Matthias Klose, upload to DELAYED/2
• #625005 src:cdrdao: "cdrdao: FTBFS: ScsiIf-linux.cc:287:37: error: no matching function for call to 'stat::stat(const char [22], stat*)'"
  add patch from brian m. carlson, upload to DELAYED/2
• #625064 src:zziplib: "zziplib: FTBFS: x86_64-linux-gnu-gcc: error: unrecognized option '--export-dynamic'"
  add patch from Ubuntu / Matthias Klose, upload to DELAYED/2
• #625068 src:liborigin: "liborigin: FTBFS: tree.hh:138:21: error: 'ptrdiff_t' does not name a type"
  add patch from Ubuntu / Colin Watson, upload to DELAYED/2
• #625080 src:fstrcmp: "fstrcmp: FTBFS: ./lib/diffseq.h:461:38: error: 'bxbest' may be used uninitialized in this function [-Werror=uninitialized]"
  add patch from Ubuntu / Colin Watson, upload to DELAYED/2
• #625096 src:zipios++: "zipios++: FTBFS: directory.h:85:12: error: 'ptrdiff_t' does not name a type"
  add patch from Ubuntu / Julian Taylor, upload to DELAYED/2
• #634394 src:gauche-c-wrapper: "gauche-c-wrapper: FTBFS: gcc: error: /usr/lib/libffi_pic.a: No such file or directory"
  add patch from Ubuntu / Matthias Klose, upload to DELAYED/2
• #644189 libcrypt-dsa-perl: "libcrypt-dsa-perl: signing leaks secret key on systems"
  upload package with patch prepared by Harlan (pkg-perl)
• #644423 libterm-readline-gnu-perl: "libterm-readline-gnu-perl: FTBFS: Could not find neither libtermcap.a, libncurses.a, or libcurses."
  add patch to find -ltinfo (pkg-perl)

Factis research has hired me to make sure their code gets contributed back to the Haskell community. As a first step, I have followed a request of someone in the audience of Stefan Wehr s talk at HaL 6 and prepared stm-stats seem my post at the factis research blog about that.
Flattr this post

I just uploaded a snapshot of GNU Screen to Debian Experimental. The package (4.1.0~20110819git450e8f3-1) is based on upstream s HEAD whose most recent commit currently dates to the 19th of August 2011. While the upload fixes tons of bugs which accumulated over the past two years in Debian s, Ubuntu s and upstream s bug tracker, I don t yet regard it as suitable for the next stable release (and hence for Debian Unstable) since there s one not so nice issue about it:

• #644788: screen 4.1.0 can t attach to a running/detached screen 4.0.3 session

Nevertheless it fixes a lot of open issues (of which the oldest is a wishlist bug report dating back to 1998 :-) and I didn t want to withhold it from the rest of the Debian community so I uploaded it to Debian Experimental. Issues closed in Debian Experimental

• #25096: digraph table should be run-time configurable
• #152961: lacks tsl/fsl/dsl caps
• #176626: mini-curses type of interface for screen -r w/ multiple screens? (Fixed by suggesting iselect, screenie or byobu)
• #223320: does not switch mouse mode
• #344759: mishandles xterm control string to set window title
• #353090: please enable the built-in telnet
• #361274: cannot reattach to sessionname if there is another session with similar sessionname
• #450421: please raise MAXWIN to at least 100 (merged with #499273)
• #461107: Requires test -t 0 even when opening a new window on existing screen
• #481411: window created with -d -m silently ignores -X exec
• #488619: Session name string escape
• #496750: screen -d -m and -D -m segfault if setenv given with no value in a configuration file
• #532240: screen with caption SEGVs when resized to 1 line tall
• #541793: C-a h (mis)documented twice
• #558724: breaks altscreen
• #560231: Please remove restriction on user/login name length
• #578729: outputs spaces when refreshing/attaching a window with defbce on
• #591624: segfault when running screen -d -m with layout save default in .screenrc
• #603009: Updating the screen Uploaders list
• #612990: /etc/init.d/screen-cleanup: should check for existence of screen binary
• #621704: Fix slow scrolling in vertical splits
• #630535: manpage typo
• #641867: version bump (this bug report sparked the upload :-)

Issues which will be closed in Ubuntu

• #183849: update to git version of screen
• #315237: crashes with certain options and terminal sizes
• #582153: doesn t accept login names longer than 20 chars
• #588846: slow when using vertical split
• #702094: Copying and pasting from mutt includes many trailing spaces
• #786292: segfaults if using layout saving with -D -m
• #788670: segfault in screen/byobu in natty

Please test the version from Experimental If you are affected by one of the issues mentioned above, please try the version from Debian Experimental and check if they re resolved for you, too. Thanks to all who contributed! A lot of the fixes have been made or applied upstream by Sadrul Habib Chowdhury who also industriously tagged Debian bug reports as fixed-upstream . Thanks! Thanks also to Brian P Kroth who gave the initial spark to this upload by packaging Fedora 15 s git snapshot for Debian and filing bug although the upload is based on the current HEAD version of GNU Screen as this fixes some more important issues than the snapshot Fedora 15 includes. That way also two patches from Fedora/RedHat s screen package are included in this upload. (Co-) Maintainer wanted! Oh, and if you care about the state of GNU Screen in Debian, I d really appreciate if you d join in and contribute to our collab-maint git repository there are still a lot of issues unresolved and I know that I won t be able to fix all of them myself. And since Hessophanes unfortunately currently has not enough time for the package, we definitely need more people maintaining this package. P.S. Yes, I know about tmux and tried to get some of my setups working with it, too. But I still prefer screen over tmux. :-)

Guus strikes back with a new Scientific tutorial in the Free OpenGL wikibook. It deals with tracing the Mexican Hat 3D function using surfaces, removing hidden ones, and also shows how to avoid visual artifacts when adding grid lines.

http://en.wikibooks.org/wiki/OpenGL_Programming/Scientific_OpenGL_Tutorial_05

exim 4.77 rc4 is available for testing in experimental. This release contains an incompatible change. Exim no longer performs string expansion on the second string of the match_* expansion conditions: "match_address", "match_domain", "match_ip" and "match_local_part". Named lists can still be used. See explanation and rationale on exim-announce mailing list.

Would people please stop using and/or deplying MTAs that are not 8-bit safe? I mean, it's the 2010s for crying out loud; not speaking 8BITMIME is very much frowned upon (page 16, paragraph 2, second sentence).

For Ada Lovelace Day this year, I want to share my appreciation for Dr. Marian C. Diamond. In years past, I ve saluted women in the field of computing, which is my field as well. Dr. Diamond, however, is a biologist. Her research includes neuroanatomy, environment, immune functions, and hormones. In particular, she is interested in studying the effects of the external environment, aging, and immune responses on the cerebral neocortex. She has, in her words, had a love affair with the brain for about 70 years. I know very little about biology. The content and methods of her research are, frankly, beyond me, though some of her results have garnered popular attention. She has inspired me by demonstrating that rare combination of gifts: a deep understanding of a technical subject, and the ability to explain it to other people in an accessible way. In her interviews, articles and lectures, many of which are available online, Dr. Diamond displays these gifts in abundance. Her skill and enthusiasm for both learning and teaching is unmistakable. After applying her gifts in the classroom for many years, digital distribution has now enabled many more people to see and hear her, through millions of YouTube views. In 1960, she became the first female graduate student in UC Berkeley s anatomy department, and was apparently given the job of sewing a cover for a magnifying machine. I can only imagine the persistence required to continue from there to become a recognized leader in her field. She has gone on to help many other students along their way, and was named an unsung, everyday hero for the support she provided to students outside of the classroom or lab. As if that weren t enough, she has also traveled to Cambodia to apply her expertise in helping children injured by land mines. She still teaches today, just across the bay from where I write this, and will turn 85 next month.

After a little pause, I thought I add another entry to my ongoing tips series. Today we'll get rid of the black bars you can most likely observe on a widescreen display, when playing a video DVD with MPlayer. This little post assumes that you've already inserted the DVD you'd like to watch, your MPlayer setup works with dvd:// style addresses, you know which title from the DVD you want to watch (the examples will use the first title) and you know your monitor's aspect ratio (the examples uses 16:9). Now all that is needed are the following two steps:

1. Detect the cropping parameters:

   mplayer -vf cropdetect -monitoraspect 16:9 dvd://1

   The previous line is followed by the usual MPlayer header and output. The film should start playing and after a few seconds you can press q (just wait until the detected values stop changing). Now there should be a few lines looking like:

   [CROP] Crop area: X: 0..719  Y: 76..499  (-vf crop=720:416:0:80).
   A:   1.5 V:   1.5 A-V: -0.000 ct: -0.018  38/ 38  5%  7%  1.0% 0 0
   [CROP] Crop area: X: 0..719  Y: 75..500  (-vf crop=720:416:0:80).
   A:   1.6 V:   1.6 A-V: -0.000 ct: -0.018  39/ 39  5%  7%  1.1% 0 0
   [CROP] Crop area: X: 0..719  Y: 75..500  (-vf crop=720:416:0:80).
   A:   1.6 V:   1.6 A-V: -0.000 ct: -0.018  40/ 40  5%  7%  1.1% 0 0

2. Play the film with the detected crop values and the correct aspect for the video material (4:3 for PAL and 3:2 for NTSC):

   mplayer -vf crop=720:416:0:80 -monitoraspect 16:9 -aspect 4:3 dvd://1

Of course you can add any other options to the invocation as usual, though you only need to add those to the second invocation (e.g. select the correct audio stream or start in fullscreen mode). Also, you might want to set the monitor aspect permantly in your MPlayer configuration (just add monitoraspect = "16:9"), as the monitor rarely changes for the average machine /etc/mplayer/mplayer.conf.local is probably the best place (make sure your /etc/mplayer/mplayer.conf includes that file at the end). Now you should be able to enjoy all video DVDs on your widescreen display without black bars at the top and bottom!

It's become kind of a clich for me to claim that the reason I'm happy working on ACPI and UEFI and similarly arcane pieces of convoluted functionality is that no matter how bad things are there's at least some form of documentation and there's a well-understood language at the heart of them. My PhD was in biology, working on fruitflies. They're a poorly documented set of layering violations which only work because of side-effects at the quantum level, and they tend to die at inconvenient times. They're made up of 165 million bases of a byte code language that's almost impossible to bootstrap[1] and which passes through an intermediate representations before it does anything useful[2]. It's an awful field to try to do rigorous work in because your attempts to impose any kind of meaningful order on what you're looking at are pretty much guaranteed to be sufficiently naive that your results bear a resemblance to reality more by accident than design.

The field of bioinformatics is a fairly young one, and because of that it's very easy to be ignorant of its history. Crick and Watson (and those other people) determined the structure of DNA. Sanger worked out how to sequence proteins and nucleic acids. Some other people made all of these things faster and better and now we have huge sequence databases that mean we can get hold of an intractable quantity of data faster than we could ever plausibly need to, and what else is there to know?

Margaret Dayhoff graduated with a PhD in quantum chemistry from Columbia, where she'd performed computational analysis of various molecules to calculate their resonance energies[3]. The next few years involved plenty of worthwhile research that aren't relevant to the story, so we'll (entirely unfairly) skip forward to the early 60s and the problem of turning a set of sequence fragments into a single sequence. Dayhoff worked on a suite of applications called "Comprotein". The original paper can be downloaded here, and it's a charming look back at a rigorous analysis of a problem that anyone in the field would take for granted these days. Modern fragment assembly involves taking millions of DNA sequence reads and assembling them into an entire genome. In 1960, we were still at the point where it was only just getting impractical to do everything by hand.

This single piece of software was arguably the birth of modern bioinformatics, the creation of a computational method for taking sequence data and turning it into something more useful. But Dayhoff didn't stop there. The 60s brought a growing realisation that small sequence differences between the same protein in related species could give insight into their evolutionary past. In 1965 Dayhoff released the first edition of the Atlas of Protein Sequence and Structure, containing all 65 protein sequences that had been determined by then. Around the same time she developed computational methods for analysing the evolutionary relationship of these sequences, helping produce the first computationally generated phylogenetic tree. Her single-letter representation of amino acids was born of necessity[4] but remains the standard for protein sequences. And the atlas of 65 protein sequences developed into the Protein Information Resource, a dial-up database that allowed researchers to download the sequences they were interested in. It's now part of UniProt, the world's largest protein database.

Her contributions to the field were immense. Every aspect of her work on bioinformatics is present in the modern day larger, faster and more capable, but still very much tied to the techniques and concepts she pioneered. And so it still puzzles me that I only heard of her for the first time when I went back to write the introduction to my thesis. She's remembered today in the form of the Margaret Oakley Dayhoff award for women showing high promise in biophysics, having died of a heart attack at only 57.

I don't work on fruitflies any more, and to be honest I'm not terribly upset by that. But it's still somewhat disconcerting that I spent almost 10 years working in a field so defined by one person that I knew so little about. So my contribution to Ada Lovelace day is to highlight a pivotal woman in science who heavily influenced my life without me even knowing.

[1] You think it's difficult bringing up a compiler on a new architecture? Try bringing up a fruitfly from scratch.
[2] Except for the cases where the low-level language itself is functionally significant, and the cases where the intermediate representation is functionally significant.
[3] Something that seems to have involved a lot of putting punch cards through a set of machines, getting new cards out, and repeating. I'm glad I live in the future.
[4] The three-letter representation took up too much space on punch cards

comments

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (144.3 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg work While taking care of the last details for the hardening feature in dpkg 1.16.1, I have mailed debian-devel to find volunteers to handle a hardening release goal. The objective is to ensure a large number of packages have been converted/rebuilt to actually use the new hardening build flags. Then I prepared the draft of the announce of the dpkg 1.16.1 upload (aka Bits of dpkg maintainers sent to debian-devel-announce) which got expanded by Guillem to also cover new features since dpkg 1.15.7. update-alternatives got some refactoring by Guillem which resulted in a regression that has been fortunately discovered by Sven Joachim. I fixed that regression and did some further cleanup inspired by the root cause of this regression (see top 4 commits here). Note that Sven is one of the few persons who are running the git version of dpkg. Hopefully the number of tester will increase since I recently documented the APT repositories with autobuilt versions of dpkg in the wiki. At the end of the month, I started working on a bugfix release (what s going to be 1.16.1.1) by fixing some of the unavoidable problems discovered after an upload that accumulated more than 4 months worth of work (see top 4 commits here). The Debian Administrator s Handbook I spent countless hours finalizing the launch of the crowdfunding campaign for the Debian Administrator s Handbook and it went live on September 27th. So far it s on good track with more than 63% of the base funding already secured. But we still have a long way to go to reach the liberation goal (we re at 21%). It s still worth nothing that more than 55% of the money raised has been put in the liberation fund so there are many persons who care about getting the book freed. More than 250 persons are supporting the project currently with an average contribution of 38 EUR. I would have expected much less for the average contribution but many more supporters. I still hope we can get more people on board with the perspective of a good DFSG-free Debian ebook. Did you order your copy? If not, click here and fix this! By the way Paypal used to be required but it s no longer the case, you can support the project just with your usual credit card. Misc blog updates Over time, I have written many useful articles for Debian users and Debian contributors. But scattered in the history, they are somewhat difficult to find. To fix this I have created some index pages listing them. Check them out:

• Mastering Debian
• Contributing to Debian
• Debian Packaging Tutorials

Two new articles joined those pages this month: How to triage bugs in the Debian Bug Tracking System and Understand dpkg and don t get stuck with a maintainer script failure. While writing the first article, I noticed we lacked a good page showing the most buggy packages so I quickly created it (with the help of UDD): http://qa.debian.org/cgi-bin/bugs-by-source Misc packaging work I did a small update to the developer s reference. Luca Falavigna submitted a patch to clarify how one is supposed to deal with meta-packages (cf #569219), I improved it and integrated the result in the SVN repository. I upgraded nautilus-dropbox to version 0.6.9 and while doing this I discovered a bug in mergechanges (filed as #640782). I uploaded a new release of quilt mainly to add the Multi-Arch: foreign field so that it can satisfy dependencies of foreign packages (i.e. packages of a different architecture). Django released some security advisories (tracked in #641405) and since the maintainer did not deal with the issue, I stepped up to the task (I m a backup maintainer) and released the fixed version 1.3.1 to unstable. I took the opportunity to switch from python-support to dh_python2, and do some misc improvements to the packaging (see changelog). I wanted to update publican to a newer version but it turned out to be not possible because Debian doesn t have the latest version of docbook-xsl yet. I also discovered some bugs in the test suite and forwarded upstream the patch I created (see upstream bug). On top of this, fop was failing due to some java problem related to the introduction of multiarch. After having reported the bug, the java maintainers quickly released a fixed version. So now publican is ready in the git repository but it s waiting on the docbook-xsl update. I got in touch with the maintainer who said he would have the time to take care of it by mid-october. Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Today I was wondering that I had almost no new mail in my inbox in the morning. After a while I decided to have a look into the server logfiles . so I learned that postfix wasn t able to deliver mails via lmtp cause of:

Oct 7 07:45:56 post cyrus/lmtpunix[307]: DBERROR db4: Logging region out of memory; you may need to increase its size
Oct 7 07:45:56 post cyrus/lmtpunix[307]: DBERROR: opening /var/lib/cyrus/deliver.db: Cannot allocate memory
Oct 7 07:45:56 post cyrus/lmtpunix[307]: DBERROR: opening /var/lib/cyrus/deliver.db: cyrusdb error
Oct 7 07:45:56 post cyrus/lmtpunix[307]: FATAL: lmtpd: unable to init duplicate delivery database
Oct 7 07:45:56 post cyrus/master[754]: service lmtpunix pid 307 in READY state: terminated abnormally Seems like this can be fixed with:

/etc/init.d/cyrus2.2 stop
cat< /var/lib/cyrus/db/DB_CONFIG
set_cachesize 0 2097152 1
set_lg_regionmax 1048576
EOM
/etc/init.d/cyrus2.2 start Looking more closer into the logs, it turned out that this trouble started last night when I connected with a client running the soon to be released Ubuntu Oneiric Ocelot using the new kmail2. So it looks like the KDE/Ubuntu folks broke again kmail (or any KDE subsystem), as it also has troubles when migrating over from kmail(1) and it looks like it s not able to access most of the imap subfolders. Well done!

Full stats are here Status for D-I level 1 (core D-I files):

• 15 languages 100% for all sublevels: bg cs de eo es fa fr it ja ko nl pt ru th zh_CN
• 3 languages 99%: pl si ta
• 1 language 98%: sv
• 2 languages 97%: eu gu
• 2 languages 96%: hi tr
• 5 languages 95%: be bn dz sk uk
• 4 langauges 94%: ast bs ga ro
• 3 languages 93%: hu lo nb
• 2 languages 92%: id sl
• 5 languages 91%: et mr pa vi zh_TW
• others are 90% or below

Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

• 8 languages 100%: de es fa fr pt ru si zh_CN
• 14 languages 99%: be cs da eo id it ja kk nl sk sl sv th uk
• 4 languages 98%: bg ca eu ro
• 1 language 97%: pt_BR
• 4 langauges 95%: el fi he nb
• 7 languages 94%: ar ast dz gl hr vi zh_TW
• 1 language 93%: is
• 7 languages 92%: bn bs hu ko ne pl tr
• 9 languages 91%: ga gu ka km lt mk mr sr ta
• others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

• 5 languages 100%: fa fr nl pt sv
• 20 languages 99%: be bg bs ca cs de el eo es fi ga gl he is ja ru sk sv th tr zh_CN
• 2 languages 97%: hu uk
• others are 90% or below

So, full 100% completeness (hall of fame) for: Persian (fa), French (fr), Portuguese (pt).

Would MTAs please stop sending hi-bit7 messages to other MTAs which do not advertise 8BITMIME! Recode it to QP or BASE64, damnit! The receiving MTA is entitled to strip the set bit7, which kinda makes things hard to read (while I know how to deal with blvde Stra_e, the advent of UTF-8 makes that blC6de StraC?e, introduces C0 control characters and makes typographic quotation marks into NUL-containing octet sequences (as their UTF-8 representation contains 0x80 octets) which let every sensible MDA terminate the line there). I even filed in the Debian BTS against the BTS (might be Drexim's fault, though). Would MUAs please default to Quoted-Printable! And mail hosters should use the same server when retrying delivery, to benefit greylisting. Or at least publish a list of outgoing IPv4 addresses they use for sending. Or use IPv6. Oh, and STARTTLS, while we are on my wishlist. It's a sad day when the percentage of correctly encoded eMail messages in my INBOX is smaller than that of my Spambox...