Today, Marc has released phpMyAdmin 3.4.0. This is huge step and introduces lot of new features. The most noteworthy are:

• User preferences
• Relation schema export to multiple formats
• ENUM/SET editor
• Simplified interface for export/import
• AJAXification of some parts
• Charts
• Visual query builder

As you can see, many of new features come from last GSoC, it took us almost half year to stabilize all new stuff and make it work nicely together. Thanks to last minute improvements in Japanese, it is fully translated to 15 languages, other translations are little behind, but you can still help improving them for future releases. The master branch is now open to development and we will start to merge patches accumulated in patch tracker for addition after feature freeze. PS: I forgot to mention that this release fixed largest number of feature requests and bugs ever - over 160 issues from sf.net tracker got closed. This can be nicely seen on tracker stats. PS2: Package with new version was just uploaded to Debian.

Filed under: English Phpmyadmin 8 comments Flattr this!

I started doing daily builds of the Debian Installer in February 2004. (Before me, Martin Sj gren did them for a while.) Seven years of building d-i every single day, twice a day in recent years, and also twice a day for armel. Well over ten thousand builds total. That would have been tedious, but thanks to cron it was instead seven years of keeping the machines running and upgraded, dealing with things when they broke, and getting highlighted on IRC whenever someone mentioned "people.debian.org/~joeyh/d-i". Still a bit tedious, but those bits were used to install a lot of machines. The rest of the d-i daily builds moved onto the autobuilders a while back, but I guess there was some reluctance to mess with my institution. I finally convinced them to take over my builds too. :) http://d-i.debian.org/daily-images/

Long time no blogging. Earlier, I wrote a blog post about transporting data to mobile phones. I have since then extracted the Data Matrix code I wrote there into a separate library, prison and added support for QR Code as well. So from KDE s next feature release, versioned 4.7 arriving around first of august, you have DataMatrix and QRCode support in Klipper. While on a barcode spree, I also added barcode support to the KDE Addressbooks for easy transfer of contacts. Also using libprison. Also available from 4.7 onwards. Few minutes ago, libprison 1.0 was tagged and hopefully soon, the KDE Sysadmins will publish my tarball. And about the weather. Some nice people wrote Debian Weather which in simple ways shows if it is unsafe to upgrade. Quite recently there have been thunderstorms and heavy rain in Debian Unstable. Perl rain.
I have recently added support for Debian Weather in the Plasma Dataengine for weather. Also available from 4.7 onwards. Last but not least, after spending a couple of hours on trying to understand Q3PtrList and how it was used in KHelpCenter, it is now, as the last component in KDE-Runtime, free of Qt3Support. All for now. Next up. Platform_11

I'm playing with btrfs to get a feel for what's coming up in linux filesystems. To be daring, i've configured a test machine using only btrfs for its on-disk filesystems. I really like some of the ideas put forward in the btrfs design. (i'm aware that btrfs is considered experimental-only at this point). I'm happy to report that despite several weeks of regular upgrade/churn from unstable and experimental, i have yet to see any data loss or other serious forms of failure. Unfortunately, i'm not impressed with the performance. The machine feels sluggish in this configuratiyon, compared to how i remember it running with previous non-btrfs installations. So i ran some benchmarks. The results don't look good for btrfs in its present incarnation. UPDATE: see the comments section for revised statistics from a quieter system, with the filesystems over the same partition (btrfs is still much slower). The simplified test system i'm running has Linux kernel 2.6.39-rc6-686-pae (from experimental), 1GiB of RAM (no swap), and a single 2GHz P4 CPU. It has one parallel ATA hard disk (WDC WD400EB-00CPF0), with two primary partitions (one btrfs and one ext3). The root filesystem is btrfs. The ext3 filesystem is mounted at /mnt I used bonnie++ to benchmark the ext3 filesystem against the btrfs filesystem as a non-privileged user. Here are the results on the test ext3 filesystem:

consoleuser@loki:~$ cat bonnie-stats.ext3 
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
loki          2264M   331  98 23464  11 10988   4  1174  85 39629   6 130.4   5
Latency             92041us    1128ms    1835ms     166ms     308ms    6549ms
Version  1.96       ------Sequential Create------ --------Random Create--------
loki                -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  9964  26 +++++ +++ 13035  26 11089  27 +++++ +++ 11888  24
Latency             17882us    1418us    1929us     489us      51us     650us
1.96,1.96,loki,1,1305039600,2264M,,331,98,23464,11,10988,4,1174,85,39629,6,130.4,5,16,,,,,9964,26,+++++,+++,13035,26,11089,27,+++++,+++,11888,24,92041us,1128ms,1835ms,166ms,308ms,6549ms,17882us,1418us,1929us,489us,51us,650us
consoleuser@loki:~$ 

And here are the results for btrfs (on the main filesystem):

consoleuser@loki:~$ cat bonnie-stats.btrfs 
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
loki          2264M    43  99 22682  17 10356   6  1038  79 28796   6  86.8  99
Latency               293ms     727ms    1222ms   46541us     504ms   13094ms
Version  1.96       ------Sequential Create------ --------Random Create--------
loki                -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1623  33 +++++ +++  2182  57  1974  27 +++++ +++  1907  44
Latency             78474us    6839us    8791us    1746us      66us   64034us
1.96,1.96,loki,1,1305040411,2264M,,43,99,22682,17,10356,6,1038,79,28796,6,86.8,99,16,,,,,1623,33,+++++,+++,2182,57,1974,27,+++++,+++,1907,44,293ms,727ms,1222ms,46541us,504ms,13094ms,78474us,6839us,8791us,1746us,66us,64034us
consoleuser@loki:~$ 

As you can see, btrfs is significantly slower in several categories:

• writing character-at-a-time is *much* slower: 43K/sec vs. 331K/sec
• reading block-at-a-time is slower: 28796K/sec vs. 39629K/sec
• all forms of file creation and deletion are nearly an order of magnitude slower
• Random seeks are almost as fast, but they swamp the CPU

I'm hoping that i just configured the test wrong somehow, or that i've done something grossly unfair in the system setup and configuration. (or maybe i'm mis-reading the bonnie++ output?) Maybe someone can point out my mistake, or give me pointers for what to do to try to speed up btrfs. I like the sound of the features we will eventually get from btrfs, but these performance figures seem like a pretty rough tradeoff.Tags: benchmarks, bonnie, btrfs, ext3

Almost a year ago I put together a simple picture sharing site, which was designed to accept uploaded images via OpenID logins and then share them. It was created because none of the existing image sharing sites allowed OpenID logins - and because frankly I rarely trust sites I don't control. (Especially when they're free sites, with no obvious revenue stream. We can reference recent compromises if we wish to be topical, but even without them it is all about trust.) I opened up the site to allow others to use it, but assumed it would remain Steve-only - After all why would somebody choose to use my site rather than imgur, or similar? Over the year I've spotted several minor problems with the site. The built-in support for comments was never used, primarily because I didn't have any other users, and some of the tagging aspects were hard to use. Since I've started using twitter I again found myself wanting a picture-hosting site, one with "short links". I started using twitpic.com, but wasn't happy with it. So the plan was formed; build another picture sharing site. But this time with short URLS. Sadly my first choice of http://picc.ie was denied to me - I couldn't register it as I don't live in Ireland. But my second choice http://linkti.me was free. As a challenge to myself this site uses Redis for storing all user-details, all image details, and all session data. It hasn't been difficult to implemenet a multi-user site, with logins, file uploads, and tagging. I guess I cheated as a lot of the code is borrowed from the previous site, but this time it looks prettier and there have been many cleanups due to my previous experience. This time round I was more focussed upon short links, so each image is allowed an incrementing number, which is then encoded using base 36 math - which is the alphabet you get if you use [0-9A-Z]. I designed three shareable links for each image to keep it simple:

• http://linkti.me/v/S - Show a HTML page with the image embedded.
• http://linkti.me/i/S - Show a raw image.
• http://linkti.me/r/S - Show a "thumbnail" of the image.

Why "/r" for the thumbnail? Well because it is a random thumbnail. To cope with difference sizes, and allow the recent image display to look regular, I wanted each thumbnail to be 150x150 pixels. My thumbnail generation simply picks a random X,Y co-ordinate from the image and then cuts out a 150x150 square block starting at that offset. The results of a random segment of an uploaded image are sometimes very cool, altough equally often they are misleading. If I upload naked people there is the possability that we'll just get a nipple or a cock - but I'm actually quite smug about the way it turned out. Simpler to code than to resize, scale, or similar and surprisingly random when you see 9x9 random images (as you do on the front page). Anyway that's all for now. I was tempted to talk about implementation or migration, but I've written too much for an early morning already. Some other time? ObQuote: "He's just trying to freak us out. " - 30 Days of Night.

For each and every bit of trite business advice, there is an equal and opposite bit of trite business advice "Fail fast" vs "be persistent" "The first person to mention a number in a negotiation loses" vs "by saying a number, you frame the negotiations" "Ideas don't matter" vs "the Number One startup killer ... making a product for which there is no interesting market" "Do something you're passionate about" vs "where there's muck, there's brass" "Business plans are a waste of time" vs "failure to plan is planning to fail" There's some valuable lesson behind all of these; I'm not saying these are bad advice. What I'm saying is that if it were so easy, more people would be doing it successfully. In reality, it's very difficult. What are some of your favorite equal and opposite bits of business advice?

(this post was going to be about something else, but after getting this far, I think it stands on its own as an introduction to dynamic linking) The shared library is an integral part of a modern system, but often the mechanisms behind the implementation are less well understood. There are, of course, many guides to this sort of thing. Hopefully this adds another perspective that resonates with someone. Let's start at the beginning - relocations are entries in binaries that are left to be filled in later -- at link time by the toolchain linker or at runtime by the dynamic linker. A relocation in a binary is a descriptor which essentially says "determine the value of X, and put that value into the binary at offset Y" each relocation has a specific type, defined in the ABI documentation, which describes exactly how "determine the value of" is actually determined. Here's the simplest example: The value of foo is not known at the time you make a.o, so the compiler leaves behind a relocation (of type R_386_32) which is saying "in the final binary, patch the value at offset 0x4 in this object file with the address of symbol foo". If you take a look at the output, you can see at offset 0x4 there are 4-bytes of zeros just waiting for a real address: That's link time; if you build another object file with a value of foo and build that into a final executable, the relocation can go away. But there is a whole bunch of stuff for a fully linked executable or shared-library that just can't be resolved until runtime. The major reason, as I shall try to explain, is position-independent code (PIC). When you look at an executable file, you'll notice it has a fixed load address This is not position-independent. The code section (with permissions R E; i.e. read and execute) must be loaded at virtual address 0x08048000, and the data section (RW) must be loaded above that at exactly 0x0805ff88. This is fine for an executable, because each time you start a new process (fork and exec) you have your own fresh address space. Thus it is a considerable time saving to pre-calculate addresses from and have them fixed in the final output (you can make position-independent executables, but that's another story). This is not fine for a shared library (.so). The whole point of a shared library is that applications pick-and-choose random permutations of libraries to achieve what they want. If your shared library is built to only work when loaded at one particular address everything may be fine until another library comes along that was built also using that address. The problem is actually somewhat tractable you can just enumerate every single shared library on the system and assign them all unique address ranges, ensuring that whatever combinations of library are loaded they never overlap. This is essentially what prelinking does (although that is a hint, rather than a fixed, required address base). Apart from being a maintenance nightmare, with 32-bit systems you rapidly start to run out of address-space if you try to give every possible library a unique location. Thus when you examine a shared library, they do not specify a particular base address to be loaded at: Shared libraries also have a second goal code sharing. If a hundred processes use a shared library, it makes no sense to have 100 copies of the code in memory taking up space. If the code is completely read-only, and hence never, ever, modified, then every process can share the same code. However, we have the constraint that the shared library must still have a unqiue data instance in each process. While it would be possible to put the library data anywhere we want at runtime, this would require leaving behind relocations to patch the code and inform it where to actually find the data destroying the always read-only property of the code and thus sharability. As you can see from the above headers, the solution is that the read-write data section is always put at a known offset from the code section of the library. This way, via the magic of virtual-memory, every process sees its own data section but can share the unmodified code. All that is needed to access data is some simple maths; address of thing I want = my current address + known fixed offset. Well, simple maths is all relative! "My current address" may or may not be easy to find. Consider the following: So foo will be in data, at a fixed offset from the code in function, and all we need to do is find it! On amd64, this is quite easy, check the disassembly: This says "put the value at offset 0x2002b2 from the current instruction pointer (%rip) into %eax. That's it we know the data is at that fixed offset so we're done. i386, on the other hand, doesn't have the ability to offset from the current instruction pointer. Some trickery is required there: The magic here is __i686.get_pc_thunk.cx. The architecture does not let us get the current instruction address, but we can get a known fixed address the value __i686.get_pc_thunk.cx pushes into cx is the return value from the call, i.e in this case 0x414. Then we can do the maths for the add instruction; 0x115c + 0x414 = 0x1570, the final move goes 0x18 bytes past that to 0x1588 ... checking the disassembly i.e., the value 100 in decimal, stored in the data section. We are getting closer, but there are still some issues to deal with. If a shared library can be loaded at any address, then how does an executable, or other shared library, know how to access data or call functions in it? We could, theoretically, load the library and patch up any data references or calls into that library; however as just described this would destroy code-sharability. As we know, all problems can be solved with a layer of indirection, in this case called global offset table or GOT. Consider the following library: Note this looks exactly like before, but in this case the foo is extern; presumably provided by some other library. Let's take a closer look at how this works, on amd64: The disassembly shows that the value to be returned is loaded from an offset of 0x200271 from the current %rip; i.e. 0x0200828. Looking at the section headers, we see that this is part of the .got section. When we examine the relocations, we see a R_X86_64_GLOB_DAT relocation that says "find the value of symbol foo and put it into address 0x200828. So, when this library is loaded, the dynamic loader will examine the relocation, go and find the value of foo and patch the .got entry as required. When it comes time for the code loads to load that value, it will point to the right place and everything just works; without having to modify any code values and thus destroy code sharability. This handles data, but what about function calls? The indirection used here is called a procedure linkage table or PLT. Code does not call an external function directly, but only via a PLT stub. Let's examine this: So, we see that function makes a call to code at 0x4d0. Disassembling this, we see an interesting call, we jump to the value stored in 0x200382 past the current %rip (i.e. 0x200858), which we can then see the relocation for the symbol foo. It is interesting to keep following this through; let's look at the initial value that is jumped to: Unscrambling 0x200858 we see its initial value is 0x4d6 i.e. the next instruction! Which then pushes the value 0 and jumps to 0x4c0. Looking at that code we can see it pushes a value from the GOT, and then jumps to a second value in the GOT: What's going on here? What's actually happening is lazy binding by convention when the dynamic linker loads a library, it will put an identifier and resolution function into known places in the GOT. Therefore, what happens is roughly this: on the first call of a function, it falls through to call the default stub, which loads the identifier and calls into the dynamic linker, which at that point has enough information to figure out "hey, this libtest.so is trying to find the function foo". It will go ahead and find it, and then patch the address into the GOT such that the next time the original PLT entry is called, it will load the actual address of the function, rather than the lookup stub. Ingenious! Out of this indirection falls another handy thing the ability to modify the symbol binding order. LD_PRELOAD, for example, simply tells the dynamic loader it should insert a library as first to be looked-up for symbols; therefore when the above binding happens if the preloaded library declares a foo, it will be chosen over any other one provided. In summary code should be read-only always, and to make it so that you can still access data from other libraries and call external functions these accesses are indirected through a GOT and PLT which live at compile-time known offsets. In a future post I'll discuss some of the security issues around this implementation, but that post won't make sense unless I can refer back to this one :)

I was amazed this morning that it's only been a month since I moved house. So much has happened since. For reasons that I won't go into, I've had to move house. Since I like living in Mechelen, not too far from work, however, I decided to search for something close by. And I did find something close by; I moved to "just around the corner". Which is a bit silly, but then blame the bureaucrats. At any rate, I'm starting to feel at home in the new spot. It's a bit smaller and a bit more expensive than the old place, but it looks a lot better, and that more than makes up for it. And it's not like I needed the extra space in the old place -- I kept having this big empty spot in the middle of my living room, anyway. To make matters more interesting, I've been on stage three times and out of the country once in this past month: One wedding, two concerts, and a work trip to Cork. I guess that's why it seemed like so much more time. But that's over now, and recently I finally managed to make some time working on other projects. Today it was incorporating some patches (which turned out not to work entirely) and doing a release for nbd. That release was overdue by quite a bit; there were a few fairly serious bugs outstanding that I had commits for in the git repository already, I just had to get them out there. Oh well. A few weeks back I've also managed to work a bit on ipcfg some more. The reason it has been so long since the last time I worked on it was that I had started to realize there were several things wrong with it, and I wasn't sure how to best move forward. But then not so long ago I had an epiphany, and I've now started to reimplement the broken bits which is, well, most of it. But hey. Moving isn't fun. Glad it's over. Almost.

While working on dictionaries for StarDict today, I've decided to finally bring them to openSUSE as well. First step was to fix my favorite tool to access them - sdcv. The fixed package is now in my home project and will be hopefully soon accepted to Education project so that it is more visible (see request 69901 if it is already the case). When I had the tool working, it was time to package dictionaries themselves. It was quite simple task and the rpms are available in my home project. To install packages on openSUSE 11.4 simply do:

zypper ar http://download.opensuse.org/repositories/home:/Nijel:/stardict/openSUSE_11.4/ stardict
zypper ref
zypper in  stardict-czech  stardict-english-czech  stardict-german-czech

Filed under: English Stardict Suse 0 comments Flattr this!

As announced on my French blog, I m attending Solutions Linux this week (10-12 May). I ll be on the Debian booth with a few other members of Debian France. If you re in Paris during those days, make sure to come by. I ll be pleased to meet you, and we ll be a bunch of Debian contributors ready to answer your questions. And if you feel like it, feel free to stop for a few hours and give us some help at the booth. It tends to be crowded and we re never enough to answer the questions. As usual, I ll come with a copy of the book Cahier de l Admin Debian (the one that I want to translate into English) to show, and I ll be glad to dedicate the book to whoever brings his copy with him/her. While speaking of conferences, this summer I m going to Banja Luka for DebConf 2011. I ve bought my plane tickets and I ll be there the whole week (24-31 july). Joining a DebConf is a completely different experience. DebConf is made by Debian people for Debian people. It s an opportunity to meet many of the people that you mainly know over IRC/email. And you don t need to be a hardcore Debian contributor to join. In fact, it s a great experience for anyone who just started contributing to Debian. Read this email of Asheesh Laroia as a proof. Note that the sponsored registration period has been extended up to the 17th of May. Register now! I m looking forward to DebConf11. On request of Stefano, I registered a Debian Rolling Bof. It will be a good occasion to see how far we are and to discuss future plans.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Today we had a very interesting demonstration. People who know me for some time know I like to be a part of demonstrations, adding my voice to the people. And even though I know most demonstrations, as massive as they might be, have no real and tangible effect... Well, I try to be there. Today we had a large demonstration in Mexico City of people angry with the government's tactics on the war on drugs Have you been to Mexico before 2007? You will remember we have always had our problems, but violence was basically limited to keeping an eye open, not flashing your money/jewels, and not looking too touristy. We were, all in all, a quiet country Known even as "the country where nothing happens, even if it happens". Five years later, a good portion of the country lives really in war-like conditions, and close to 45,000 people have been killed. At some point about a year ago (where the body count was at ~22,000), our oh-so-bright de-facto president said that "only" 10% of them were civilians. If that is still the case, and if proportions are held, we can now talk about ~4,500 people killed just because they were there Oh, but remember the country is not at war, and death penalty has very long been abolished, so we cannot give the other 90% a different status. We have had over 45,000 killed people. That means at least ~200,000 torn lifes. About six weeks ago, the cry from a father who lost his son started getting heard. And no, I do not want to talk about specific names or anything like that It is just the voice-bearer of many people who are fed up. Four days ago, he and a group of people left Cuernavaca city on a caravan and walked to Mexico city. Yesterday evening they arrived to the National University (UNAM), and were greeted by an amazing open-air concert: Mozart's Requiem. Today, our day started very early. My mother wanted to help, as many more anonymous citizens did, so we started making and packing sandwiches and apples at ~5:30AM, delivered them to the camp where they were staying and grouping with other people, and came back home for our breakfast. Later, at around 9AM, as the demonstration started going by our house (we live a block from UNAM), my girlfriend, father and me started walking with them. Why do I say this was a different demonstration than others I have been to?

1. Much longer. We walked for close to 16Km, for about five hours. Most demonstrations I have been part of are 3-5Km long
2. A different kind of support. I was surprised, even laughed at first when I saw a lady standing by us with a sign that read, I cannot join you today, but I'm there in spirit . But then I saw more, more people that just went out of their houses to a point close to where we walked on, and just stayed there on the streets. Not walking, yes, but being part of it.
3. The march was silent And yes, this is the first time I see it for true. At the very beginning we heard some people chanting, cheering, but the cheerers were requested not to be noisy. People wearing political parties' logos were requested not to wear them (as political parties are at an all-time low prestige). No, we were not silent for the whole 5 hours; we were talking with each other, but it felt... just very different from any other times.
4. Not only were people supporting, but this is the first time I saw this amount of people wanting to materially help. As I said, my mother delivered a little breakfast for 50 people, for those that slept at the camp, for those that had already walked ~70Km. But all along the road we were offered oranges, cookies, water, bread... Whatever people were able to offer, to somehow help. Very nice feeling of solidarity.
5. Contrary to all other times, we were really spread out, it was a very low density walk, os it was quite pleasant. Of course, I have not even seen an estimation of the number of participants... And I doubt anyboy can be decently close to it

Will this change anything? I doubt it. I am happy that for the first time, the de-facto president (remember a large percentage of Mexicans still believe he lost the presidential election, and it was only due to his lack of legitimacy that he pushed the army to the streets to start this war on drugs) is recognizing that this might not be the best strategy but it is the only one he has... People are still, despite what I saw many felt, very far from organized. And, yes, no short-term concrete proposals are made on how to stop the killing I feel the population agrees that drugs should be legalized and regulated, that would at least shift the problem, reduce the direct violence (and money) related to it... But, of course, it is plainly not feasible with our current reality. Anyway, after a 5hr walk today (and even after a nice nap and shower), I don't want to dig into hypotheses anymore. I just wanted to share about a very interesting and different experience I had the opportunity to be part of.

This should probably be an official FAQ, but a) I wanted to rant a bit more than is probably acceptable for something "official" and b) the sort of person this information is directed at never bloody reads keyring.debian.org, which is the logical place for it. Who are keyring-maint? Currently Gunnar Wolf (good cop) and Jonathan McDowell (bad cop). Previous keyring maintainers include Igor Grobman & James Troup. I'd like to be a DM/DD. Do I send you my key? No. You go through the DebianMaintainer or NM processes. Then the DM team or DAM tell us to add your key to the appropriate keyring. I'd like to replace my DM/DD key in the Debian keyring. What should I do? Read the instructions at http://keyring.debian.org/replacing_keys.html I have a new key that isn't signed by anyone else, will you accept it? No. Did you read http://keyring.debian.org/replacing_keys.html ? I've got a single DD signature on my new key. That's enough, right? Not unless your old key has been lost and you're getting a different DD to request the replacement for you (and if they're prepared to ask for a key replacement we'll wonder why they're not prepared to sign the new key too). Did you read http://keyring.debian.org/replacing_keys.html ? I'm still really confused about how I should request a key replacement. Help? Try reading https://rt.debian.org/Ticket/Display.html?id=3141 (which just happens to be a recent decent example). Clear subject line (I'd have added a real name too, but it's still fairly clear), full fingerprint of the old and new keys, inline signed so RT doesn't mangle it. New key signed by old key and 3 other DDs. Request signed by old key. That RT link needs a login. I don't have one. Have you tried reading up on the Debian RT system? There's a generic read only login that'll get you access. That's too hard. Can't you just give me the details? Damnit. It appears the read-only login details are currently disabled due to misuse (one wonders how). Try reading http://wiki.debian.org/rt.debian.org Why are you using RT? Isn't bugs.debian.org more appropriate? We need the ability to for people to contact us is in a private fashion, for example if they need to us to remove a key because it's been lost or compromised. We could only use RT for that purpose and use bugs.d.o for things that can be public, but this way all the information is in one place and we get to make the call about when it becomes a publicly viewable ticket. What's with jetring? Should I send you a jetring changeset? jetring is a tool written by Joey Hess that used to be used to manage the Debian Maintainers keyring. keyring-maint borrowed a number of good ideas from jetring but don't use it at all. We ignore jetring changesets. So you just want key fingerprints, not attached keys? Yes. Of course you have to make sure your key is actually on a public keyserver so we can get it. the.earth.li is a good choice (because Jonathan runs it and thus pays more attention to it), but subkeys.pgp.net or pool.sks-keyservers.net are also commonly used. My key has expired and I want to update the key expiry date. I should email RT asking for this to be done, right? No, you should send the updated key via HKP to keyring.debian.org. You can do this with "gpg --keyserver keyring.debian.org --send-key <keyid>" Obviously replace <keyid> with your own key ID. I tried to send an entirely new key via HKP to keyring.debian.org, but I can't see it there. What gives? keyring.debian.org only accepts updates to keys it already knows about. That means you can send updated expiry dates, new uids and new signatures to your existing key, but not an entirely new key. I sent my updated key via HKP to keyring.debian.org and can see it's updated there, but the Debian archive processing tools (eg dak) don't seem to recognize the update. Why not? The updates sent via HKP are folded back into the HKP server automatically every 15 minutes or so. They are folded into the live Debian keyrings on a manual basis, at least once a month. This means if your key has an expiry date then you probably want to update your key at least a month before it expires. Where can I find these live Debian keyrings? They're what's available via rsync from keyring.debian.org::keyrings/keyrings/ This is canonical location for the current Debian Developers and Debian Maintainers keyrings. What about the debian-keyring package? This is a convenience package of the keyrings. It's usually the most out of date. We update it sporadically and try to ensure that the version shipped with a stable Debian release is current at the point of release. It is not used by any of the official Debian infrastructure. Why don't you automatically update my key in the live keyring when I send an update via HKP? We think that automatic updates of keys that allow uploads to Debian are a bad thing and that invoking a human eye at some step of the process is a useful sanity check. Paranoid much? Never enough. How are updates to the keyring tracked? We use bzr to maintain the keyring, with a separate file per key that can then be easily combined into the various keyrings. You can see the repository at: http://bzr.debian.org/scm/loggerhead/keyring/debian-keyring/changes Note that this is only updated when a keyring is pushed to live; the working tree may contain details of compromised keys and thus isn't public. What's with the whole replacement of 1024 bit keys? 2 things. Firstly 1024 bit keys tend to use SHA1 as a hash algorithm, which has been shown to be weaker than expected. While we're not aware of active exploits against this updating all of the keys Debian uses is not a trivial process and it's wiser to get it done /before/ there's a known issue. Secondly computing power has moved on and we feel that upgrading to larger key sizes is prudent. Elliptic curve cryptography (ECC) keys look like the future. Can I use one for Debian? No, not at present. When there are tools that are part of a Debian stable release that support them we'll look into it, after discussion with the major users of the keyring (DSA, ftpmaster, the secretary).

GBrowse, a genome browser, is packaged for Debian since three weeks. It is the result of a work started four years ago, and I am especially grateful to Olivier Sallou for having unblocked and finished the project while I was starting to go round in circles, and to Aaron M. Ucko for repairing the new package, that was not compatible with our build farm. GBrowse and similar software, like IGV, are used for graphical representation of a chromosome and its annotations, such as the gene positions and markers of their activity, and in particular the result of experiments using high throughput DNA sequencing. A Debian Med task is actually dedicated to gather programs for that topic. In a couple of years, it may be possible that any person who wants can sequence his chromosomes, that is, his genome. This will revolution medicine and perhaps break some founding myths. Even if software for the general public will look different, packages like gbrowse are a first step towards a better access of the public to his medical data. Work continues on GBrowse, with the goal of being able to install it with a reference version of the human genome with a command like apt-get install human-genome. The next step will be to update BioPerl, used by GBrowse, to version 1.6.9.

DebConf11 program, the FOSS way I'm going to DebConf11 and, once more, I'm thrilled at idea of living for about 2 weeks immersed in a very very very, very Debian-ic life. DebConf10, DebConf edition of last year, has been for me one of the most intense DebConfs ever, it would be hard to match it. But one of the nice things about DebConf, no matter the edition, is that it's very "open source" and do-ocratic in they way it is organized: everyone could contribute into making it even better than the previous edition. For one thing, there is no "event organizer" company behind DebConf. Rather, most of the hard work that goes into making a DebConf possible is done by the DebConf team, a volunteer team formed by a mixture of Debian contributors, local team members, and DebConf enthusiasts. But while they take care of most of the work and hence deserve the thanks of all DebConf attendees for that they do not account for all that makes a DebConf a great DebConf. For instance, having a great program is up to DebConf participants and the event proposals they submit: talks, BoF, panel discussions, film projections, contests, you name it. Today, in deadline frenzy mode (read on), I've submitted for review the following events for DebConf11:

• bits from the DPL, the traditional "state of the union"-like address
• Test Driven Development BoF, about DEP-8 and autopkgtest (co-authored with Tom Marble)
• sprint and money BoF, to review one year of the Debian sprint program and further discuss how we use donations to the Debian project
• Debian dErivatives eXchange BoF, about DEX and other initiatives to improve collaboration with Debian derivatives
• DebConf in Debian BoF: one year later the "DebConf vs Debian" BoF, what's the state of integration of DebConf in Debian? (spoiler: good!) how can we improve it further? (co-authored with Gunnar Wolf, Holger Levsen, and Moray Allan)
• DPL wannabe BoF: inspired by a discussion during this year DPL campaign, an attempt to explain what the "DPL job" is about and, who knows, collect "training" material for wannabe DPLs
• rolling BoF: let's summarize the monster thread about Debian rolling for those we lost half-way through and present the advancements of more recent and consensual proposals (this event is actually not by me: I've just prodded some of the rolling proponents on IRC and Raphael Hertzog has kindly agreed to submit and organize the BoF)

That's it from me, how about you?
Are you going to DebConf and willing to attend a conference with a great program? Great, then start submitting a great event yourself! Anything you are working on in Debian or you are excited about qualifies. And don't worry if you've never attended DebConf either, video archives from past DebConfs can give you more than a feeling of typical DebConf events. While the first deadline for events submission expires today, submissions will be open all the way to the end of the conference, but you should better hurry up as the early birds get the best scheduling slots (and missing the deadline, you'll miss the round of the review for the main program ... but you still have a few hours left :)). Good luck and see you in Banja Luka.

One of my last blog suggested a way to systematically clean up versioned dependencies in packages. Given the comments posted thereafter, it s clear that the one-size-fits-all strategy is ill-advised. Sorry about that. So, should these versioned dependencies be cleaned up or not ? Well It depends. Cleaning up versioned dependencies will hurt people when the package you re working on may be back-ported to old version of Debian. In corporate world, upgrading a machine may cost a lot, so you can still find production machine running etch. I would not be surprised to hear about older systems still used in production. In consumer world (mostly desktop or multimedia), people will mostly run more recent versions of Debian. All in all, the following elements must be considered before removing old versioned dependencies:

• Is the package likely to be installed on server ? or is it more targeted to consumer world ?
• What is the policy of the maintenance group of this package ? For instance, Debian perl group policy is to clean up versioned dependencies older than old-stable.

Unfortunately, there s no universal strategy. In case you want to clean up old dependencies, the latest version of Dpkg::Control model for Config::Model will be helpful. The last version of this models provides a bunch of meta Dpkg parameters. These parameters are used to customize the behavior of the Dpkg editor tool to suit your needs (For more details, see Config::Model::models::Debian::Dpkg::Meta). The dependency meta parameters will enable you to tune the behavior of config-edit to your needs. There are 3 dependency parameters, from the most global to the most specific ones:

• dependency-filter: your personal general policy
• group-dependency-filter: the policy of the maintainer group of your package
• package-dependency-filter: filter policy specified per package

Before applying a cleanup, config-edit will check the policies in the above order. The most specific one will be applied. Ie. the group policy will supersede your personal policy and the package specific policy will override the group policy. Here s a screenshot of config-edit GUI invoked on a package: From the parameters set in the GUI above, you can see that a warning will be raised for :

• perl package with versioned dependencies older or equals than lenny
• any other package with versioned dependencies older or equals than etch

Last but not least, if you want to use config-edit without any dependency filtering, just make sure that all these filter parameters are empty. You will still benefits from other features, like warning if a package dependency is unknown, which is useful to catch typos. All the best

I was invited to talk at the Mukthi 11.04 event hosted at the M. S. Ramaiah Institute of Technology. I talked about Free Software/Debian - Philosophy, Design, Merits. The focus was more on the philosophical aspects of why Free Software/Debian. The slides are available here.

CyanogenMod does list Samsung Galaxy S in its list of devices, but given that it is marked experimental I was not very confident of trying it out. But yesterday, I took the plunge. Even though experimental, the ROM is pretty good. And the Cyanogen goodness just makes it awesome. This ROM (also being AOSP GingerBread) is Super Fast. So in a day of use, my impressions:

• Battery performance has turned out to be better than what I was used to.
• Cyanogen's enhancements are pretty impressive.
• The CyanogenGalaxyS team's work is of very good quality, even though they've marked it experimental. (Thanks codeworkx, coolya and the team)

The CyanogenMod experience is also good because it makes me feel like home, just like our Debian project. read more

Date for end of sponsored registrations for this years DebConf11 in Banja Luka was set for May 8th, today that date has officially been extended to May 19th. Now to answer your question on why are you extending sponsored registrations 12 hours before it ends here s the answer. Registrations are going fine, we have more then 300 attendees who registered so far, it is due to change in our monetary resources as sponsorship level has changed from one to another, and we ve be granted to welcome more people. But it is also to give some possible attendees more time and information, as I ve got a feeling a lot of people from this region and elsewhere have been lacking some even general information, and thus I d like to bring everything closer to everybody. Before anything, heading to our website might be a good start as we have immense amount of information up there. Since government of Republika Srpska is our greatest and main financial sponsor, local team has made a document for them in which we tried to explain most basic and crucial things from what DebCamp and DebConf is over logistical/statistical data to what we have planned for Fun and Free time, so checking out that document might be a good idea as well DebConf11 Plan and Program. Since that might be too much for somebody as the documents stretches to some ~30 pages, I ll try to give answers to most common/asked questions. Registrations Who should register? In plain English, everybody. Only people that don t need to register are people who will attend Debian day, please see Plan and Program section 5.3 for more information on Debian Day. If you want to attend any of the DebConf lectures/presentations/whatever you should register. Registrations will play great deal in this years DebConf as if you register you ll get accreditation with your name on it and what you re allowed to do. Whether you have sponsored food/accommodation and thus allowed to eat for free or that you can sleep in certain hotel, or whether you re allowed to the pools for free; also as we re planning to have tight security in our Venue, in order for you to even enter Venue building you ll need to have accreditation on you. So, if you re planning to do anything with DebConf please do register. What is DebCamp/should I attend DebCamp? Please refer to Plan and Program document it s all said at the end of page 15. Sponsorship After registrations this is next thing that is brought up and that is most unclear to most of you. You re able to request travel, accommodation and food sponsorship, what each of those means is pretty self elaborate. But the real question here might be, who is eligible to request sponsorship? Again, everybody. What are is the criterion for getting sponsorship? There really isn t a written rule who gets the sponsorship and who doesn t but if you have ever been involved or contributed in any sort of way to Debian, or even have contributed or have been involved in any open source projects you re more then welcome to apply for sponsorship. Furthermore, due to the funds that we have been granted from Government of Republika Srpska, I even encourage you to ask sponsorship, don t be intimidated and just do it Professional/Corporate This is category for those who are sent by their companies, and whose participation/attendance is sponsored by the same. Professional covers the price of actual costs of their visit, while the corporate is double that price, which is for those who besides paying the actual costs also want to contribute money and help Debian. Organized transport and getting to Banja Luka Now, for those who are flying to Banja Luka I personally told you to come to via Zagreb. Why did I do this, if Banja Luka has its own airport? I told you this because Zagreb is much larger airport and thus has many budget companies, meaning cheaper flights. Once the registrations are closed, based on your arrival/departure dates we ll be able to statistically predict how many of you will be coming to Zagreb on each date and each time as we re planning to have organized transport. This means that we ll send buses/shuttles to pick people up in Zagreb. Of course to have this organized we ll need to set up arrival/departure pages with all the information that s need for us to have this organized. Now, for those who want to come directly to Banja Luka airport this might be bit pricer then going via Zagreb, but not by much. I personally value my time and I even fly from Zagreb to Sarajevo Phil for example booked his flight just this morning from London via Ljubljanja to Banja Luka and it wasn t that much more expensive then Zagreb. Also worth noting is that Banja Luka might be limited for some destinations, but I for a fact know that there are flights coming from Zurich, Ljubljanja, Belgrade and so on. Maybe someone from Banja Luka could follow up on this one with more information. Questions? With this post I hope I have brought some of these topics closer to you and that I made some things clearer. If you have any questions, please feel free to leave your comments or email whatever you re concerned about to our mailing list on: debconf11-localteam@lists.debconf.org Further reading:
Gunnar Wolf Posts explaining DebConf

One of the nice features of a distributed version control system (DVCS) like git is the ability to tag specific states of your project, and to cryptographically sign your tags. Many projects use simple tag names with the version string like "0.35". This is a plea to ensure that the tags you make explicitly reference the project you're working on. For example, if you are releasing version 0.35 of project Foo, please make your tag "foo_0.35" for security and for future disambiguation. There is more than one reason to care about unambiguous tags. I'll give two reasons below, but they come from the same fundamental observation: All git repositories are, in some sense, the same git repository; some just store different commits than others. it's entirely possible to merge two disjoint git repositories, and to have two unassociated threads of development in the same repo. you can also merge threads of developement later if the projects converge.

Avoid tag replay attacks

Let's assume Alice works on two projects, Foo and Bar. She wraps up work on a new version of Foo, and creates and signs a simple tag "0.32". She publishes this tag to the Foo project's public git repo. Bob is trying to attack the Bar project, which is currently at version 0.31. Bob can actually merge Alice's work on Foo into the Bar repository, including Alice's new tag. Now looks like there is a tag for version 0.32 of project Bar, and it has been cryptographically-signed by Alice, a known active developer! If she had named her tag "foo_0.32" (and if all Bar tags were of the form "bar_X.XX"), it would be clear that this tag did not belong to the Bar project.

Be able to merge projects and keep full history

Consider two related projects with separate development histories that later decide to merge (e.g. a library and its most important downstream application). If they merge their git repos, but both projects have a tag "0.1", then one tag must be removed to make room for the other. If all tags were unambiguously named, the two repos could be merged cleanly without discarding or rewriting history.

I noticed this because of a general principle i try to keep in mind: when making a cryptographic signature, ensure that the thing you are signing is context-independent -- that is, that it cannot be easily misinterpreted when placed in a different context. For example, do not sign e-mails that just say "I approve" -- say specifically what you are approving of in the body of the signed mail. Otherwise, someone could re-send your "I approve" e-mail In-Reply-To a topic that you do not really approve of. By extension, signing a simple tag is like saying "the source tree in this specific state (and with this specific history) is version 0.3". A close inspection of the state and the history by a sensitive/intelligent human skilled in the art of looking at source code can probably figure out what the project is from its contents. But it's much less ambiguous to say "the source tree in this specific state (and with this specific history) is version 0.3 of project Foo". Once you start using unambiguous tag names, you make it safe for people to set up simple tools that do automated scans of your repository and can take action when a new signed tag appears. And you respect (and help preserve) the history of any future project that gets merged with yours.Tags: git, security

Steve Langasek has been contributing to Debian for more than a decade. He was a release manager for sarge and etch, and like many former release managers, he s still involved in the Debian release team although as a release wizard (i.e. more of an advisory role than a day-to-day contributor). Oh, and he did the same with Ubuntu: on the picture on the left, he just announced the release of Ubuntu 10.04 from his Debian-branded laptop. He has also been maintaining PAM in Debian for as long as can I remember and does a great job at that. He s very knowledgeable and fully deserves his place within the Debian Technical Committee. I m glad he still has the time to participate on several important Debian mailing lists because his contributions are always very useful. I m sure you ll notice this just by reading his answers below. My questions are in bold, the rest is by Steve. Who are you? I m 32 years old, have been running Linux since my first year in college back in 96, and have been a Debian developer now for ten years. Along the way I ve been involved in maintaining a variety of server packages, worked on the Alpha port for a while, did a stint as a release manager for a couple of years, and serve on the technical committee. This year I m also celebrating my ten year anniversary with my lovely wife Patty, who many know as an erstwhile front-desk volunteer at DebConf. God only knows why she puts up with my late-night hacking! These days in my day job I m a manager on the Ubuntu Platform team at Canonical, working to help make Ubuntu a daughter distribution that the Debian community can be proud of. What s your biggest achievement within Debian or Ubuntu? There s no doubt that my biggest achievement in Debian has been overseeing the release of two Debian releases as release manager. On the other hand, the scope of a release is so huge, and it represents the output of so many developers working together, that it would be arrogant to claim the release itself as an achievement of my own. Also, sarge and etch have long since been rotated off of the mirrors so no one cares about them anymore. For a more personal and lasting contribution in the distro itself, I m very proud of writing pam-auth-update. It s a small piece of code, but one that Debian was missing for a long time it s made a big difference to PAM module integration in packages! What are your plans for Debian Wheezy? My top priority for this cycle is to see multiarch through. We re still not far enough along in Debian for most developers to see any difference and once we are, the first thing people are going to see is a fair bit of breakage when we start breaking a lot of assumptions about paths that have been hard-coded upstream. But I m still excited by the progress that is being made here. We should be able to ship wheezy without any ia32-libs package. We might even be able to get rid of all the biarch library packages, including those used by the toolchain itself. 54 packages in testing build-depend on gcc-multilib right now, in order to build 32-bit code to ship in the amd64 package; a bunch of those should go away with absolutely no reduction in functionality, saving us a bit of space in the archive and saving the maintainers a lot of complexity in their packages, while at the same time giving us much better support for cross-compilation than we ve ever had before. It s a tall order, certainly, but the pieces are falling into place one by one. My second priority is to get a policy in Debian around packages integrating upstart jobs. It would of course benefit Ubuntu to have many packages back in sync with Debian, but if all we wanted was to sync with Debian, we could mostly just make debhelper ignore upstart jobs in Debian, prefer them in Ubuntu, and call it good. I m interested in making sure Debian also gets the benefits of being able to use upstart, because as Linux has become increasingly asynchronous (doing more in parallel at start up), the traditional sysvinit has not been able to keep up. There are all kinds of bugs now related to network startup, for instance, that we don t have a good answer for in a sysvinit model but that we can fix with an event-based system. Upstart has been around for a while now, but we ve been slow to integrate it into Debian because it only works on Linux. It would be a shame if right after the first Debian GNU/kFreeBSD technology preview, packages all stopped working on kFreeBSD because they started to assume the availability of upstart! Unfortunately, having been so cautious we now have systemd on the scene, which not only doesn t support non-Linux but seems to be in the process of trying to gobble up other, non-Linux-specific components of the desktop stack. So I have to wonder what the future holds for the free desktop on non-Linux kernels. If you could spend all your time on Debian, what would you work on? Well, based on my previous experiences when I did spend all my time on Debian, I think the answer here is QA / release work. Otherwise, I don t know. My hands are full enough now with multiarch that it s hard for me to see what the Next Thing would be. You re a member of the technical committee. In the interview of Bdale Garbee, I have argued that it s not working well. What s your point of view on this topic? Well, I feel a constant low level guilt about my own poor level of activity in the TC; but that doesn t translate into a belief that the system is broken. This is, after all, the decision making body of last resort for technical disputes in Debian, and as such it should really be used sparingly. And if a reputation for glacial deliberation means more developers work out their disputes on their own rather than asking the TC to step in, I think that s actually a healthy thing! I do still wish we were more effective at resolving those issues that do come our way, but there s no silver bullet for this. Though the funny thing is, I ve noticed that the majority of issues that get referred to the TC nowadays never even need us to make a decision; a short conversation with the disputants is often enough to get them to come to an agreement. What s the biggest problem of Debian? By and large, I think Debian is still doing a great job at what it s best at delivering a rock-solid distribution that users can rely on. If I would highlight one problem in Debian, though, it would be that I think we re becoming less innovative as time goes on. Part of that comes from being such a large project that we re bound to be more conservative as an institution; but even though the three pet Debian projects of mine that I mentioned above are fairly innovative (multiarch, pam-auth-update, upstart), each of these has landed first in Ubuntu rather than in Debian. Always with a clear intent of pushing back up into Debian, of course, but it just wasn t possible to do this work within Debian for the first cut without much longer delays. I worry that if Debian is no longer the place to try new things, that we re going to miss out on attracting contributions from the folks who are inspired to make Free Software better and not simply to make it stable. I m not sure how to address this, though. Maybe improved conversations with derivatives such as (but not limited to) Ubuntu, about what crack of the day is being tried where and how that can be integrated into Debian once it s proven to work? I don t think that team-based maintenance or low-threshold NMUs do anything to address this, though, as the kinds of innovation that matter most are ones that require discussion and consensus-finding not just routing around inactive maintainers. Do you have wishes for Debian Wheezy? Well, I d like to see the armhf port get on its feet and become an official port. Over the lifetime of the arm and armel ports, the state of the art on ARM has changed quite a bit; it would be great to see Debian taking advantage of this richer platform, to let people make better use of their hardware via Debian. As a former release manager, you re now a release wizard . I guess you have seen it on debian-devel, there are proposals to not freeze testing and to use another distribution starting as a snapshot of testing to finalize the new stable release. According to your experience, what needs to happen to make this possible? Frankly, I ve stayed out of that discussion because I don t think what s being asked for is possible. I think proponents of a freezeless release have seriously underestimated the amount of work required on the part of the release team to wrangle testing into a releasable product, and that anything that makes propagation of fixes into the pending release more time consuming will make Debian worse on the whole, not better. If people really want to avoid long freezes for the Debian release, the best way they can help this happen is by making Debian more releasable on an ongoing basis, by helping to hold our packages to our shared standards for quality (i.e., by fixing RC bugs!). The biggest factor in long freezes for Debian is the slow rate at which we bring the RC bug count down during the freeze. Back in the sarge, etch days we used to have really great bug squashing parties that would get people together on weekends to hack through RC bugs by the dozens. I don t see that happening as much anymore. I d really like us to get back to that, but my few attempts at this so far since retiring as release manager have led me to think I m really terrible at organizing parties of any kind. On the other side, as seen at http://bugs.debian.org/release-critical/, the RC bug count for testing at the beginning of the release cycle keeps getting higher and higher. I d love to know why that is so we can address it. I know we ve gotten better at detecting some classes of RC bugs; that s part of it, but I don t think it explains the whole trend. Is there someone in Debian that you admire for their contributions? Wow, what kind of arrogant jerk would I be if I didn t admire anyone in Debian for their contributions? Debian is and always has been an amazing community of top-notch developers; there are certainly too many I admire to list them all here. Joey Hess certainly makes the list, for his longstanding example of code speaking louder than words and for his ability to get to the heart of common problems and come up with elegant solutions. So does Russ Allbery, who by all accounts had his ability to feel anger in response to email burned out of him at a young age in a flame-related accident on Usenet. The list goes on, but here I think I have to follow Joey s example and cut the words short.

---

Thank you to Steve for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.