I have announced the Debian Community Poll and published first results in former blog posts. I'll publish my analysis of the remaining questions about changes to Debian now.
Should Debian remove its non-free component?

• 80.8% answered no
• 10.7% answered yes
• 6.3% answered: I don't know or don't care
• 2.1% did not answer

Should Debian spend more money?

• 28.9% choose answer #5: I don't know or don't care.
• 22.8% choose answer #3: Debian should pay people having important positions in Debian and doing important work.
• 21.2% choose answer #1: Debian should spend more money on organizing developer conferences and team meetings.
• 11.4% choose answer #4: Debian should not spend more money.
• 8.0% choose other (see below) or didn't answer
• 7.7% choose answer #2: Debian should spend more money on free merchandizing, free DVDs, having a sexy web site, and being present on IT events.

There were quite a number of other answers. One participant missed the information about how the money is currently spend. Several participants didn't want to choose one of the provided answers. They either wanted to choose multiple answers or various combinations of them. Most other answers fall into one of the following categories:

• various marketing suggestions with different focus than answer #2
• Debian hosted hardware, infrastructure, services
• funding upstream development
• QA and work on release-blocking issues
• partners and commercial support
• developing important features
• security support for oldstable
• education of prospective developers and contributors
• documentation for users
• improving usability and accessibility
• certifications like LPIC
• getting supported by hardware and non-free software vendors
• beer
• promoting debian in developing countries
• help contributors running a business on Debian
• Bounty system
• updating stable to avoid becoming stale
• developing multimedia codecs
• getting compliant with FSF guidelines for a free system distribution
• developing free replacements to non-free software
• maintaining a database of debian-friendly hardware
• lobbying and politics
• visibility to wider society, even non-IT
• hardware for driver developers
• a more sexy DVD/CD set with graphics (like Fedora, Ubuntu)
• membership to boards of W3C, TEI Consortium, OASIS, etc.

Do you prefer time based releases instead of the "it's ready when it's ready" releases?

• 73.1% answered no
• 19.8% answered yes
• 5.1% anwered: I don't know or don't care
• 2.0% didn't answer

Which release interval do you prefer?

• 38.7% choose answer #2: about 12 months
• 36.9% choose answer #3: 18 - 24 months
• 10.0% choose answer #5: I don't know or don't care.
• 5.9% choose answer #1: about 6 months
• 5.5% choose answer #4: more than 2 years
• 3.0% didn't answer

First there was mr, then etckeeper. Now git-annex completes my triptych of software tools that use git in ways that were never intended. Git's original tagline was "the stupid content tracker". Git-annex inverts that, by managing files with git, without checking their contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, checksumming time, or disk space. I've been working on git-annex for two weeks, and have been trusting my data to it for a week. Not a little data, but terabytes spread amoung multiple archival drives, file servers, laptops, and portable USB drives. In situations too complex for unison or rsync to be of much use. Applying git to this problem in an unconventional way has simplified things enormously. At this point it should be very usable, and solid. A package will be uploaded to Debian shortly; or git clone it.

First there was mr, then etckeeper. Now git-annex completes my triptych of software tools that use git in ways that were never intended. Git's original tagline was "the stupid content tracker". Git-annex inverts that, by managing files with git, without checking their contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, checksumming time, or disk space. I've been working on git-annex for two weeks, and have been trusting my data to it for a week. Not a little data, but terabytes spread amoung multiple archival drives, file servers, laptops, and portable USB drives. In situations too complex for unison or rsync to be of much use. Applying git to this problem in an unconventional way has simplified things enormously. At this point it should be very usable, and solid. A package will be uploaded to Debian shortly; or git clone it.

A couple of month ago, my step-father-in-law encountered quite some problems with the Windows installation on his Notebook and therefore showed a great interest in dem Linux da. So we installed him a Debian in a dual-boot setting. It was interesting to see how a basic school teacher (who is technical interested, but has no deeper technical knowledge of computers) worked with Linux, what problems he found, and how he solved them. But a big obstacle for him to work more or less entirely with Linux has always been power management problems with his notebook. To be more precise: While it was always possible to send his notebook into suspend or hibernate, it was also always impossible to wake the notebook up again. It seems that the (non-free but needed) NVidia driver was a part of the problem. Last weekend we again tried to solve the problem by installing the latest backported kernel and updating the NVidia driver, but failed again. So we decided, that with the squeeze being frozen for quite some time, it would be safe to upgrade his Notebook to that system. And believe it or not: It works! We can suspend the notebook without any problems and it will wake up again! And to his (and our) special delight waking the notebook up is a lot faster, than it used to be with his Windows (Vista?) installation. Not to mention that for some reason a suspended Linux-System seems to consume less power (he lend some kind of energy consumption measuring device from his power company to track energy saving potential, and while playing around, we also measured his notebook), than the suspended Windows on the very same system! I don't know why, but finally the installation of dem Linux da was a complete success :) So kudos to those who worked on that area :)

The bug webinterface of the Universal Debian Database currently knows about the following release critical bugs:

In Total:	700
Affecting Squeeze:	291
Squeeze only:	72
Remaining to be fixed in Squeeze:	219

Of these 219 bugs, the following tags are set:

Pending in Squeeze:	16
Patched in Squeeze:	44
Duplicates in Squeeze:	24
Can be fixed in a security Update:	15
Contrib or non-free in Squeeze:	8
Otherwise fixed in Squeeze:	27

Ignoring all the above (multiple tags possible), 125 bugs need to be fixed by Debian Contributors to get Debian 6.0 Squeeze released. However, with the view of the Release Managers, 173 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers. Connection to alioth.debian.org closed. PS: Due to an unscheduled downtime, last weeks statistics have not been published. But better late than never, here are the current statistics.

In the past, I ve helped customers run native Windows applications using emulation (wine) and/or virtualization, and in few cases, native support for C# by open source projects such as Mono. But, as it s been said, there are great benefits of having source code, and I recently worked around an issue in Mono by recompiling a whole .NET project in Debian. So here s a brief guide reviewing the issues that I found and how I solved each one of them leading to a natively-compiled C# application that runs seamlessly in Debian and Windows, and please remember I m not a C#/.NET/Mono developer:

• Missing assemblies: sometimes gmcs (which is the Mono compiler) will complain about missing assemblies, that is, libraries that are requested by the C# source code (files that end with .cs) hopefully because some methods are being used in that specific file. I know they re, but I still haven t found a .NET class that is not supported in Mono. So, first and foremost, you need to actually have that library installed in your system, if you re using a Debian based system run aptitude search libmono.* or the packages.debian.org site to find which package has the library file (ends in .dll) and then, when you re calling gmcs, remember to use the r: flag, for example r:System.Drawing.dll
• Missing resource indicators: apparently .NET projects consume stuff from resource files which also seem to be XML files. Since I don t know how to handle this situation, I resorted to the fine MonoDevelop IDE, since I had a .csproj file (a Visual Studio Project file) in place which hopefully could join the dots. So just use MonoDevelop (you need to install it separately) and then open the .csproj file and rebuild the entire project (F8) which in my case just left 2 errors to go
• Issues with configuration management: it seems that the .NET Framework has some classes that help with application configuration via generic XML configuration files, and the project that I was trying to compile used that indeed, so there were some tags in the configuration file that weren t properly handled by Mono, e.g., the default system proxy configuration parameter, so I just deleted it from the XML. Be also advised that you do need to have the .config XML file in place together with the resulting .exe file if your application access configuration parameters in runtime.
• SSL- and TLS certificate-related issues: finally, if your application uses an SSL or TLS Web Service (or any other SSL/TLS application FWIW) you might get some errors like authorization or decryption failed which are basically related to non-existent certificate handling in the application. The easiest way to solve it is to download and import the CA Certificate Files from Mozilla and also download the certificate for the site you re trying to access; you can run (as the user that will run the application):
• mozroots import ask-remove
• certmgr ssl https://your.site/

That said, I had a great time fiddling with this C# application and Mono on my Debian Sid laptop. I was able to properly compile without errors nor framework-related warnings, using an IDE such as MonoDevelop, and have a full-featured native binary that s interoperable.

I'm thinking about getting a Kindle, and I thought I'd write down some thoughts, even if they're a bit jumbled, and invite commentary. Good: One thing I really like, living here in Italy, is that I would be able to get English language books instantaniously, and not pay shipping or import duties on them. Whatever disadvantages there may be, that is a huge advantage. Closely tied to that: if we ever move back to the US, shipping all our books will be very expensive. Having some of them in an electronic format would lighten that load considerably. It's fairly cheap. At $180, it's not something I'd want to treat carelessly, but it's significantly less expensive than various existing tables. Long battery life means it's not something you have to worry much about. E-ink looks like it's easier on the eyes than LCD screens. I'm not sure, but also its slow update speed might be a "feature not a bug" in the sense that you're not so easily distracted by the internet being just one click away. Bad: I love having real, physical books in my house. I grew up in a house full of books, and I want the same thing for my daughter. I like being able ot look through them, pick one up, and read about something. Real books are very convenient, as long as you're not lugging around hundreds of them. I like not having to worry too much about dropping them, about getting a bit of water on one or that sort of thing. You can cart them around in a backpack with little worry, take them to the beach, and so on. As far as I can tell, Knuth's books are not available for, nor would appear like they do on paper. That's the golden standard for me: once "The Art of Computer Programming" looks good on an e-reader, it will be really and truly ready. We're not there yet. It's not general purpose. Like I said above, maybe this is good for reading, but there are also times when it'd be nice to have a general purpose device, with email, maps, applications, and so on. A while back, Amazon announced some sort of beta program for a kindle SDK. I haven't heard much of anything since. So it looks like it's not really possible to code for it, which is a bit disappointing. I like to have that option available, even if I don't always exercise it. Unknown: The pricing: sometimes Kindle versions are cheaper (especially, in my case, considering shipping, and, potentially, import duties), but sometimes they aren't, which seems a bit unfair, given that you cannot sell the book if you don't like it. I'm not sure how it would work out for me in the aggregate. Digital Rights Management, or DRM, means you can't, say, sell someone a "used" ebook. This is, in some ways, understandeable, because it's not like an e-book degrades with time as a physical one might. Were it possible to sell on your old books, that might radically change the market. I could see that having both good consequences (cheaper books), but also negative ones for authors (less money). I suppose this topic merits a post or two, or even an entire book of its own, because it's a complicated subject without, in my opinion, "easy" answers. Suffice it to say, though, that there are some drawbacks, but I suppose it's also nice that Amazon "knows" which books I have, so even if I switch to a different reader, I still have my books. And it's good that authors can get paid to work on a book. Not having actually bought it yet, I have no idea how it actually 'feels' to read a book with one. I love curling up with a good book, something I certainly can't do with my laptop. The Kindle looks light and small enough that it could work, but maybe it's just too high tech and sleek to provide that same comfortable coziness. Or maybe I'm just a luddite where books are concerned?
Do you have one? Do you use it? For all kinds of books or just certain kinds (technical books, cheap 'fun' reads, whatever...)? What's the experience like?

I use ratpoison as window manager on my ASUS EeePC netbook nemo for more than two years now. But although I m very happy with ratpoison in the EeePC, there are two feature wishes which have been refused by upstream: One is more flexibel window name matching for the unmanage command. The other one is focus follows mouse between ratpoison frames. Well, I always guessed that it was possible, but it took until now to find outhow to implement focus follows mouse for ratpoison. There s an ancient but still useful tool called Not a Window Manager (nawm) which is a small awk-like interpreter offering mostly window handling functions. The following .nawmrc implements focus follows mouse in nawm:

window newwin;  # stores window to raise
window lastwin; # stores previous window to prevent race conditions
leave  
    lastwin = currentwindow;
 
enter  
    newwin = pointerwindow();
    if (name(newwin) != "" && newwin != lastwin)  
        raise newwin;
        sync;
     
 

The leave hook is necessary to prevent flapping between two windows if switched between them via ratpoison s commands. I also had to add the following hook to my .ratpoisonrc to work around some cases where ratpoison s own window switching didn t work anymore. Only happened with more than one frame with one frame banishing the mouse cursor was annoying, so I filtered that case:

addhook switchwin exec if [  ratpoison -c fdump fgrep -o frame wc -l  -gt 1 ]; then ratpoison -c banish; fi

Unfortunately nawm has been removed from Debian Sid about a year ago due to being buggy and orphaned. There was not upstream development for seven years or so either. So for the moment you can get nawm either from Debian Lenny or from snapshot.debian.org. But I had to fix a segfault in nawm when calling name() on a window without name to be able to use it at all, so you will probably have to rebuild it anyway with the following patch:

diff -u nawm-0.0.20030130/builtins.c nawm-0.0.20030130-patched/builtins.c
--- nawm-0.0.20030130/builtins.c        2010-10-25 06:00:02.000000000 +0200
+++ nawm-0.0.20030130-patched/builtins.c        2010-10-25 04:15:25.000000000 +0200
@@ -546,8 +546,12 @@
     *name = gcstrdup("");
   else
      
-      *name = gcstrdup((char *)nm);
-      XFree(nm);
+      if ((char *)nm)  
+        *name = gcstrdup((char *)nm);
+        XFree(nm);
+        else  
+        *name = gcstrdup("");
+       
      
  

And yes, I m thinking about adopting and reintroducing the nawm package into Debian Sid. But I d prefer if anyone could give me a hint how to do this with more current and still maintained tools (or a patch against ratpoison :-). I looked into suckless-tools, but I haven t found anything in there which provides hooks on X events. And the Perl module Tk seems to be able to set X event hooks, but only within the application being written itself.

I launched raphaelhertzog.com this summer (taking over the English content of my former multi-lingual blog), when I decided that I would be more serious about blogging on Debian/Ubuntu related topics. On September, I decided to write 2 articles per week and up to now I managed to keep the schedule. Two of my articles were published by Linux Weekly News, those are much more researched than the average blog article (they are tagged with [LWN] in the list below). The most popular articles Most people read my blog through the RSS feed which happens to be syndicated on Planet Debian and Planet Ubuntu. According to the feedburner s statistics, the top-5 articles are:

1. 5 reasons why I still contribute to Debian after 12 years (32700 views)
2. [LWN] Understanding Membership Structures in Debian and Ubuntu (31700 views)
3. Social Micropayment Can Foster Free Software, Discover Flattr (30100 views)
4. Everything you need to know about conffiles: configuration files managed by dpkg (29900 views)
5. How to make 110.28 EUR in one month with free software and Flattr (29400 views)

But I also have occasional readers visiting my blog because my articles are announced on Identi.ca, Twitter and Facebook (and they circulate on social networks, thanks to those who are sharing them!). The top-5 articles according to the statistics of my website are:

1. 5 reasons why I still contribute to Debian after 12 years (6000 views)
2. [LWN] Can Debian offer a Constantly Usable Testing distribution? (5000 views)
3. Understanding Debian s release process (1500 views)
4. Flattr FOSS (1400 views, not an article but I regularly blog about this project)
5. Can Debian achieve world domination without being on Facebook? (1100 views)

The most flattered Since I am using Flattr on my blog, it can be interesting to see the articles which generated lots of flattr micro-donations. The top-3 articles are my articles about Flattr (1, 2, 3). Excluding articles related to Flattr, the top-5 is:

1. 5 reasons why I still contribute to Debian after 12 years (12 flattr)
2. The secret plan behind the 3.0 (quilt) Debian source package format (10 flattr)
3. How to use multiple upstream tarballs in Debian source packages? (5 flattr)
4. [LWN] Understanding Membership Structures in Debian and Ubuntu (4 flattr)
5. Do You Want a Free Debian Book? Read on. (4 flattr)

Most articles get 2 to 3 flattr clicks. The most commented I usually get 4-5 comments on most articles but some generate much more feedback:

1. [LWN] Can Debian offer a Constantly Usable Testing distribution? (40 comments)
2. 5 reasons why I still contribute to Debian after 12 years (22 comments)
3. Can Debian achieve world domination without being on Facebook? (15 comments)
4. How to generate different dependencies on Debian and Ubuntu with a common source package (14 comments)
5. [LWN] Understanding Membership Structures in Debian and Ubuntu (12 comments)

Factoids Here are my conclusions based on the above figures:

• Writing about your Debian/Ubuntu work and your long term involvement makes for highly popular content that spreads well.
• In-depth and well researched articles (like those written for LWN) do not generate more flattr revenues than the average article even if they take 4 to 8 times as long to write.
• People are more likely to flattr you for your free software contribution than for the value they get out of your article.
• People care a lot about the Debian release process, and like to discuss the topic.

If you also appreciate the above-linked articles, you should click here to subscribe to my email newsletter.

4 comments Support my work

gregoa writes he squashed only 5 bugs in two weeks, and I think he still deserves a huge kudos too, as he squashes them for a long time now :)
And while at it, kudos to everyone who does not post their stats to planet, actually find those bugs, etc etc. Thanks that we have such a community!Now to the usual list of (pseudo) squashed bugs:

• #600782 pyca is NOT compatible with the default python version 2.6
  actually pyca IS compatible with python2.6, it just throws some DeprecationWarnings
  after discussion with maintainer, upstream and submitter, bug downgraded to wishlist, as pyca should be ported to the new libraries to be compatible with later python versions (2.7 is still fine according to upstream, though)
• #600670 libhpmud0 installs /usr/lib/libhpmud.so.0 /usr/lib/libhpmud.so.0.0.6 but sane needs /usr/lib/libhpmud.so that is in libhpmud-dev, not needed to scan
  shipping the .so in libhpmud0 instead of libhpmud-dev seems wrong, patch sent to make the sane backend look for libhpmud.so AND libhpmud.so.0, submitter confirms patch works as expected. NMU not uploaded yet.
• #600530 madplay needs rebuild, currently uninstallable due to dependency
  madplay installs just fine in sid, as it depends libesd-alsa0 libesd0, closed as invalid by the submitter after short discussion
• #601070 python-django: FTBFS in squeeze: test_correct_url_value_passes failed
  some tests need network connection patch that disables these sent
• #599523 update-manager-gnome: wants to downgrade packages without any notice
  this one is old, got updates from the submitter, reproduced the bug in a VM. This is an old known bug in apt :(
• #601186 horde3: only shows php deprecations and a warning no actual operational parts
  why does php again pollute the output with useless warnings? my approach to disable the warnings wasn t succesfull, but a patch created by sed was. However I m not confident with it (OpenSSL anyone?) and want some more eyes to look over it.

Kanzepharaz of the one thousand and eight spires. Kanzepharaz capital of the Empire of K'thorv and since the succesful end to the war that led to the annihilation of the Pleiadic Oligarchy in K'threl, the richest and most splendid city in the Middle World. Kanzepheraz whose real estate prices have soared e'en beyond the calculation of the rainbow abacus of Thobis Himself. Anathomelis, a princeling of the Imperial Clan needs a location for his new and expanded orgy dome. Yet the only available and affordable on a seventh rank civil allowance place left in the city of one thousand and eight spires is the tomb of Pharaz the First and Subsequent, lich-emperor of the Jerelite dynasty and his eldritch retainers. Before the construction-serfs can move in, the tomb must be cleared of its unshriven and non-rent-paying denizens. As His Principalities' chief swordbearer it has fallen to you to make the tomb of Pharaz safe for gentrification. Treasures and honors await you if you succeed. Aeons of torment in the Planes of Unlife await you if you fail. WILL YOU RISE TO THE CHALLENGE? rec.games.roguelike.development is having another challenge, this time to complete a roguelike game in four days. My previous attempt to write one in seven days was a failure but as I can build upon my previous work, I think I can make it this time. This time I did not waste lots of time thinking of a name. S.Y.T.Y.C.D...OMG? and The Real Housewives of Mordor were the only ones under serious consideration. With that crucial task out of the way, I spent the morning ripping code out of A.L.F. wherever possible. In the afternoon, I looked into dungeon generation. My previous experiments with this sort of thing were pretty convoluted but now I think I have a simpler algorithm which still provides a random yet plausible-looking map. It basically goes like this:

1. Divide the map into a grid of sectors.As the S.Y.T.Y.C.D...OMG? map is 60 80, my sectors are 20 16 giving 3 5 grid.
2. Mark each space on the map as impassable.
3. In each sector, pick a random square as its "center" not too near the edge of the sector.
4. Join each pair of centers first W -> E and then N -> S. with passable spaces. These will be the corridors and doing it this way ensures all sectors will be connected. Last time, my mistake was making rooms first and then corridors which is much more complicated to get right.
5. For each sector, there is a 2/3 chance it contains a room. If there is, randomly a number of spaces to each side of the center square. These will be the dimensions of the room. Make all the spaces in the resulting rectangle passable.
6. Randomly take a quarter chunk out of some rooms making them "L" shaped for a bit of variety. (at some point I can add round, polygonal or other shapes.)
7. Create "walls" by making all the outer boundary spaces of the room impassable again. This also ensures rooms do not touch or overlap.
8. If a wall meets a corridor, turn it into a "door." Doors don't do anything yet but eventually will be locked, trapped etc.
9. Finally, place monsters, treasure, and items in the room.

Two weeks ago, nekohayo posted a blog entry on Recipe Manager, a (you guessed it) cooking recipe manager for GNOME. Looking good, I fetched the bzr tree from Launchpad and played a bit with it, and soon discovered it had no internationalisation support. I've tried to add i18n properly, but I've not had enough time to do it. Before tackling that, the authors need to give it some bootstrapping love so the app can actually install, look for its files in /usr/share, etc. My fugly, unpostable current patch does allow for a preview of how Recipe Manager will teach the world about the best rice dish ever, arr s a banda. Yum!

Recipe Manager, showing off the zenith of Valencian culture

Dear Gnome people. Is there a reliable way to right-attach panel applets? May laptop screen and my external monitor have different resolutions, and I havn't found a reliable way (except making two panels) of getting some of my applets to stick to the right. I usually keep my launchers on the left, while vairous system status icons on the right part. But when switching to a higher resolution, at most one will move to the right end of the panel at the new resolition, while the others just stay in place (i.e. left-attached at a given pixel position). There are so many smart layout managers, GTK pretty much always uses flexible layout managers, why is the Gnome panel relying pixel offsets? It has been suggested to me a couple of times that I could achieve this via gconf. However, since the gconf changes are not reflected at runtime but frequently overwritten I did not get this to work reliably even just for two applets. Thank you. If you have any comments, please email me via erich () debian org. On a side note, this is also the first true post to my new blog. :-)

So I got a SuperMicro A+ Server 1012G-MTF today (seems to be a very nice unit for a decent price) and am preparing it for taking over fortytwo.ch and related services. Now this thing has got IPMI / BMC with remote management and KVM (both serial console and full graphical console with virtual CD-ROM etc.); works very nice. Basically the only thing I miss is the ability to disable services I don't need and/or restrict access to certain IP addresses. (No, I don't have the BMC on a public IP, but still...) So the question is: has anybody worked out how to hack / what kind of file system the IPMI Firmware for the H8SGL-F mainboard is? Or how one could drop from the BMC commandline to a /bin/sh prompt on the urnning system? A blog entry at Serverfault suggests it's been done but doesn't say how. (Running strings on the firmware binary shows the string Photoshop ICC profile near the end. I'm not sure if I want to know the story ... ;-)

Some time ago, I asked what are the common solutions for services remote logging. It was suggested scribe, an application log server developed at Facebook and later released as an open source project, and we finally managed to get this ready for Debian squeeze.

While I have spent countless hours working on the new source format known as 3.0 (quilt) , I ve just realized that I have never blogged about its features and the reasons that lead me to work on it. Let s fix this. The good old 1.0 format Up to 2008, dpkg-source was only able to cope with a single source format (now named 1.0 ). That format was used since the inception of the project. While it worked fine for most cases, it suffered from a number of limitations mainly because it stored the Debian packaging files as a patch to apply on top of the upstream source tarball. This patch can have two functions: creating the required files in the debian sub-directory and applying changes to the upstream sources. Over time, if the maintainer made several modifications to the upstream source code, they would end up entangled (and undocumented) in this single patch. In order to solve this problem, patch systems were created (dpatch, quilt, simple-patchsys, dbs, ) and many maintainers started using them. Each implementation is slightly different but the basic principle is always the same: store the upstream changes as multiple patches in the debian/patches/ directory and apply them at build-time (and remove them during cleanup). Design goals for the new formats When I started working on the new source package format, I set out to get rid of all the known limitations and to integrate a patch system in dpkg-source. I wanted to clear up the situation so that learning packaging only requires to learn one patch system and would not require modifying debian/rules to use it. I picked quilt because it was popular, came with a large set of features, and was not suffering from NIH syndrome. This lead to the 3.0 (quilt) source format. I also created 3.0 (native) as a distinct format. 1.0 was able to generate two types of source packages (native and non-native) but I did not want to continue with this mistake of mixing both in a single format. The KISS principle dictated that the user should pick the format of his choice, put it in debian/source/format and be done with it. Now the build can rightfully fail when the requirements are not met instead of doing something unexpected as a fallback. Features of 3.0 (quilt) This is the format that replaces the non-native variant of the 1.0 source format. The features below are specific to the new format and differentiate it from its ancestor:

• Supports compression formats other than gzip: bzip2, lzma, xz.
• Can use multiple upstream tarballs.
• Can include binary files in the debian packaging.
• Automatically replaces the debian directory present in the upstream tarball (no repacking required).
• Creates a new quilt-managed patch in debian/patches/ when it finds changes to the upstream files.

Features of 3.0 (native) This format is very similar to the native variant of the 1.0 source format except for two things:

• it supports compression formats other than gzip: bzip2, lzma, xz.
• it excludes by default a bunch of files that should usually not be part of the tarball (VCS specific files, vim backup files, etc.)

Timeline Looking back at the history is interesting. This project already spans multiple years and is not really over until a majority of packages have switched to the new formats.

• January 2008: the discussion how to cope with patches sanely rages on debian-devel@lists.debian.org. My initial decisions are the result of this discussion.
• March 2008: I have implemented the new formats and I request feedback. dpkg 1.14.17 (uploaded to experimental) is the first release supporting them.
• April 2008: I ask ftpmasters to support the new source packages in #457345.
• June 2008: Lenny freeze. dpkg is not supposed to change anymore. Several changes concerning the new source formats are still accepted in the following months given that this code is not yet used in production and must only be present so that lenny can cope with new source packages once squeeze starts using them.
• February 2009: Lenny release.
• March 2009: Work on squeeze has started, ftpmasters have done nothing to support new source formats, I submit a patch in #457345 to speed things up. I start a wiki page to track the project s progress and to answer common questions of maintainers.
• November 2009: After an ftpmaster sprint, it s now possible to upload new source packages in unstable. This draws massive attention to the new format and some people start complaining about some design decisions. The implementation of 3.0 (quilt) changes a lot during this month. dpkg in lenny is even updated to keep up with those changes.
• March 2010: Up to now, I was planning to let dpkg-source build new source packages by default at some point in the future. After several rounds of discussions, I agree that it s not the best course of action and decides instead to make debian/source/format mandatory. The maintainer must be explicit about the source format that s/he wants to use.
• October 2010: The new source formats are relatively popular, a third of the source packages have already switched: see the graph. The squeeze freeze in August clearly stopped the trend, hopefully it will continue once squeeze is released.
• June 2013: Project is finished?

As you can see this project is not over yet, although the most difficult part is already behind me. For my part, the biggest lesson is that you won t ever get enough review until your work is used within unstable. So if you have a Debian project that impacts a lot of people, make sure to organize an official review process from the start. And specifying your project through a Debian Enhancement Proposal is probably the best way to achieve this. If you appreciate the work that I put into this project, feel free to join Flattr and to flattr dpkg from time to time. Or check out my page Support my work .

12 comments Support my work

It was for a very long time (I think back from PyconIT 2) that I wanted to contribute to python. About a month and a half ago I started reading all the documentation I could about development processes & workflows, mercurial stuff (I don't have commit right, so I prefer working on a DVCS than on SVN), bugs management and so.

In today s Linux distributions, there are usually two to three levels of package management. In this blog post, I will explain the three levels of package management. 1. Local (dpkg, rpm) The first level of package management is the local level. This level consists of package management tools that install and/or remove packages via package archives such as .deb or .rpm files and a database of the local s system state. 2. Connected (APT, YUM, Smart) The second level of package management is the connected level. In this level, tools are fetching packages and information about them from (remote) locations known as repositories or sources . A level 2 package manager is usually built on top of a level 1 package manager, for example, APT uses dpkg but there are also cases where one tool is both, such as opkg. 3. Abstract (PackageKit, Smart) A level 3 package manager is a tool that supports different level 1 package managers via one generic interface. It may be implemented on top of a level 2 package manager (PackageKit), or may be implemented directly in level 2 (Smart). Conclusion: Merge Level 2 and 3 Most level 2 package managers share a great deal of tasks, such as solving dependency problems. Merging level 2 package management and level 3 package management would enable us to reduce the number of dependency problem solvers and combine our efforts into a few package managers. Such an implementation would need to be written in C and support all common level 1 package managers in order to be successful. As some might have guessed, this is what I plan to do with the project codenamed APT2, although work is not progressing very fast. PS: Back from vacation I spent the more than the complete last week in (mostly rainy) Corfu, Greece and am now getting back to my usual development stuff. I have already processed my news & blogs backlog (1000+ messages) and will now start with my email backlog, so don t be angry if the answer to your email takes some time.
Filed under: APT2, General

If you re running a 64bit system, and you ve got users with access to a video device (/dev/video*), then be sure you update your kernels for CVE-2010-2963. I ve been slowly making my way through auditing the many uses in the Linux kernel of the copy_from_user() function, and ran into this vulnerability. Here s the kernel code from drivers/media/video/v4l2-compat-ioctl32.c:

static int get_microcode32(struct video_code *kp, struct video_code32 __user *up)
 
        if (!access_ok(VERIFY_READ, up, sizeof(struct video_code32))  
                copy_from_user(kp->loadwhat, up->loadwhat, sizeof(up->loadwhat))  
                get_user(kp->datasize, &up->datasize)  
                copy_from_user(kp->data, up->data, up->datasize))
                        return -EFAULT;
        return 0;
 

Note that kp->data is being used as the target for up->data in the final copy_from_user() without actually verifying that kp->data is pointing anywhere safe. Here s the caller of get_microcode32:

static long do_video_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 
        union  
                struct video_tuner vt;
                struct video_code vc;
...
          karg;
        void __user *up = compat_ptr(arg);
...
        switch (cmd)  
...
        case VIDIOCSMICROCODE:
                err = get_microcode32(&karg.vc, up);
...

So, the contents of up are totally under control of the caller, and the contents of karg (in our case, the video_code structure) are not initialized at all. So, it seems like a call for VIDIOCSMICROCODE would write video_code->datasize bytes from video_code->data into some random kernel address, just causing an Oops, since we don t control what is on the kernel s stack. But wait, who says we can t control the contents of the kernel s stack? In fact, this compat function makes it extremely easy. Let s look back at the union. Notice the struct video_tuner? That gets populated from the caller s up memory via this case of the switch (cmd) statement:

...
        case VIDIOCSTUNER:
        case VIDIOCGTUNER:
                err = get_video_tuner32(&karg.vt, up);
...

So, to control the kernel stack, we just need to call this ioctl twice in a row: once to populate the stack via VIDIOCSTUNER with the contents we want (including the future address for video_code->data, which starts at the same location as video_tuner->name[20]), and then again with VIDIOCSMICROCODE. Tricks involved here are: the definition of the VIDIOCSMICROCODE case in the kernel is wrong, and calling the ioctls without any preparation can trigger other kernel work (memory faults, etc) that may destroy the stack contents. First, we need the real value for the desired case statement. This turns out to be 0 4020761b. Next, we just repeatedly call the setup ioctl in an attempt to get incidental kernel work out of the way so that our last ioctl doing the stack preparation will stick, and then we call the buggy ioctl to trigger the vulnerability. Since the ioctl already does a multi-byte copy, we can now copy arbitrary lengths of bytes into kernel memory. One method of turning an arbitrary kernel memory write into a privilege escalation is to overwrite a kernel function pointer, and trigger that function. Based on the exploit for CVE-2010-3081, I opted to overwrite the security_ops function pointer table. Their use of msg_queue_msgctl wasn t very good for the general case since it s near the end of the table and its offset would depend on kernel versions. Initially I opted for getcap, but in the end used ptrace_traceme, both of which are very near the top the security_ops structure. (Though I need share credit here with Dan Rosenberg as we were working together on improving the reliability of the security_ops overwrite method. He used the same approach for his excellent RDS exploit.) Here are the steps for one way of taking an arbitrary kernel memory write and turning it into a root escalation:

• overwrite security_ops with default_security_ops, which will revert the LSM back to the capabilities-only security operations. This, however, means we can calculate where cap_ptrace_traceme is.
• overwrite default_security_ops->ptrace_traceme to point to our supplied function that will actually perform the privilege escalation (thanks to Brad Spengler for his code from Enlightenment).
• trigger the function (in this case, call ptrace(PTRACE_TRACEME, 0, NULL, NULL)).
• restore default_security_ops->ptrace_traceme to point to cap_ptrace_traceme so the next caller doesn t Oops the system (since userspace memory will be remapped).

Here s the source for Vyakarana as seen running in Enlightenment using cap_getcap (which is pretty unstable, so you might want to switch it to use ptrace_traceme), and as a stand-alone memory writer. Conclusions: Keep auditing the kernel for more arbitrary writes; I think there are still many left. Reduce the exploitation surface within the kernel itself (which PaX and grsecurity have been doing for a while now), specifically:

• Block userspace memory access while in kernel mode. This would stop the ability to make the kernel start executing functions that live in userspace a clear privilege violation. This protection would stop the current exploit above, but the exploit could be adjusted to use kernel memory instead.
• Keep function pointers read-only. There is no reason for these function pointer tables (fops, IDT, security_ops, etc) to be writable. These should all be marked correctly, with inline code exceptions being made for updating the global pointers to those tables, leaving the pointer read-only after it gets set. This would stop this particular exploit above, but there are still plenty more targets.
• Randomize the kernel stack location on a per-syscall basis. This will stop exploits that depend on a stable kernel stack location (as this exploit does).

I finally bought myself an eBook reader. To be precise, I got the Sony PRS-600 Reader Touch Edition. Two reasons:

• I bought a lot of technical literature in digital (PDF) format and want to read it without looking at a CRT while on a train.
• The idea of having a lot of reference material with me appeals me.
• Many of my books are read only once (especially Star Trek etc.) and I don t want to waste paper and the energy for shipping.

First of all, the display is only really great with a powerful light source. The reflection is a bit of a nuisance but I can live with it. What I don t like is the PDF support of the device. Viewing a normal page, the font faces are so tiny that it is not readable in any way. The zoom feature works around that, but you can t switch pages from there. Instead you have to unzoom, go to the next page and zoom again. Crappy! There is also the reflow option, which does not mix with source code figures in my books. I am glad that most of my electronic books are from The Pragmatic Bookshelf, which means that I can download them in Epub as well, all without DRM. Thanks, guys! For the PDFs I would like to carry, I googled a while and stumbled across this blog post. The example to work with source documents worked fine for The Little Book of Semaphores at least. But even with BRISS I did not (yet) manage to crop books I bought from APress. Most likely I will try doing it via a bit of python scripting and the poppler bindings for python. I also bought my first DRM content: Star Trek Vanguard: Harbinger. I really enjoyed the book, but the download was a PITA. After buying, you only get a link to a ASCM file, which is not useful by itself. One needs Adobe Digital Editions to download it. So I installed it into a virtual machine which has XP installed. After that, I was able to read it on the PC, which wasn t the goal of the exercise. To put it on the reader, one needs to authorize it from the same Windows software and upload the file. All that stuff took almost an hour and of course the digital edition of the book is more expensive than getting it from amazon germany. But at least I did not have to wait for delivery. Having finished the book I wanted to buy the second part, but of course it is only available for canadian residents on ebooks.com. I researched a number of ebook sites which mostly sell it only in the US. Funny enough, the first hits when searching where pirated versions of the book in full text. Why do they make it so hard to get stuff legally? *sigh*