I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it. PS: do MIND THE GAP, please. :-)

I've been working (part-time) on an existing Rails application for the last month or so, getting comfortable with it, modifying some sections to support needed functionality, and beginning an all-out, carpet-bombing "shock and awe" assault on it's test suite. However, a significant portion of the system is (or rather was, since I've already gutted a lot of it) a chunk of rather messy and unpleasant auto-generated code. I've always been a bit iffy about code generation tools, ever since my first encounters with them whilst working on some $DEITY-awful access application years ago. (First encounters with anything Microsoft usually have that effect on me, though). Since then, I've never found any reason to change my mind, and dealing with this particular chunk of auto-generated code has pushed me over the edge into "wild-eyed extremist" territory. Please note that when I say "code generator", I'm talking here about programs or wizards or whatever that create a block of code that you're then expected to tweak to suit your conditions. If it's just a mechanical transformation of an input format to an output format, where a human is never expected to maintain the output by hand, that's not code generation in my book -- that's just a compiler being used for it's intended purpose. I like compilers. I'm fairly sure you do too. Code generators, though? EVIL. Consider what you get out of your generator: A big blob of code in your application that you then have to understand and maintain. It's like you gave a commit bit to some random person who dropped a 5000 line change into your tree and then ran away into the night, cackling. If there's bugs in the generated code (and remember, there's always bugs in software), when they get fixed upstream you're up the creek unless you haven't modified the code at all (and thus can regenerate), or you're willing to apply the necessary patches to your modified version, often by hand. Hand-patching is error-prone, and if you've modified the code at the same place that has to be patched, things can get unpleasant. In the case of the generator I'm dealing with at the moment, I'm not even blessed with it being decent code. It's ugly code, in every possible sense of the word. There's inconsistent formatting, poor commenting, it uses some totally whack coding constructs[1], the hard-coded text strings are ungrammatical, it's got very few (and poorly covering) test cases, the methods are huge and unwieldy, and it's got security bugs (not so hot for something that's supposed to be guarding your application). In short, it looks like some quick-and-dirty internal hack that escaped one night and is now wreaking havoc on the world. There's probably some really good code generators out there. I've been poking around a bit, and there's some written for Rails by people who I know produce good code. However, no matter how good the author of the code generator is, the very best you can hope for is solid, clean code that is the equivalent of some portion of your application having been coded by some member of your team who you (and nobody else) has ever met, who didn't follow your team's coding conventions, and who has never looked at any other part of your application to make sure that everything hung together in a consistent manner. Which you now have to maintain. I've read a lot of books, articles, blog posts, and mailing list messages, and participated in no shortage of "why software sucks" discussions with other programmers, but I've never heard anyone advocate that the way to clean code, minimal bugs, and happy programmers was to drop a big chunk of unknown code right in the middle of things. Before you start charging up the e-mail-delivered tasers, ready to fry my delicate braincells for the twin crimes of Xenocodophobia and a vicious case of Not Invented Here syndrome, though, please understand that I'm not against code reuse in the right place. That right place is a library, component, or other abstract, documented, fairly static, and presumably-replaceable chunk of code. By using a library, you get a relatively simple interface to somebody else's complexity. This is a good thing, for all the reasons that everyone should already be able to recite. Items such as a comprehensible interface, minimal learning curve, and so on. Things you don't get with generated code because, pretty much by definition, the entire chunk of code is you "interface". There are some times, though, when what you want to do can't be done in a library. Either the thing has too many tentacles to be able to easily encapsulate behind an abstracted interface, or you need to modify it's behaviour in too many places to be able to get away with a few simple callbacks to get the job done. Hence, it might be tempting to fire up the code generator to get yourself out of a hole quickly. Don't do it though. Don't succumb to the temptation of saving yourself a little bit of time by firing up a code generator. What you're doing there is just making pain for yourself, by inheriting a big chunk of code that you don't fully understand, probably doesn't solve your problem particularly well, and which you (or whichever poor slob gets the job after you) will have to maintain into perpetuity -- most likely by gradually rewriting large pieces of it so that it fits your application better. The output of a code generator is likely to be the least maintainable code in your tree -- either by virtue of the fact that it's crap code, or at the very least because you just don't understand it. The general principle of good codebase hygiene is that you get rid of unmaintainable code quickly, so it doesn't infect everything else (a la the "broken window theory"). With that in mind, why would you ever consider deliberately introducing a huge chunk of unmaintainable code into your application? The closest I can come to a defence of code generators is something like Rails' scaffolding, where the controller and views for a set of simple CRUD operations on a table are auto-generated. The key word there, though, is scaffolding. It's very name suggests that it's temporary, to be replaced with something more robust and useful as soon as is reasonable. The default scaffolding is ugly as sin, besides, which encourages you to replace it, and -- as of Rails 1.1 (I think), there's a 'scaffold' method which creates the scaffolding output at runtime, so you never, ever need to generate scaffolding code any more. Anyone planning on producing any significant piece of their application through the use of code generators is, in my book, committing a crime against software decency.

---

1. My favourite whack code construct thus far has been this gem:

 case somevar
   when 17
     do_something_here()
 end

Because, you know, if statements are for people who know how to program, or something.

Yeah, we are almost two weeks back now, but until now I can feel we are done with lots of pending issues.

To summarize, I left DebConf on Saturday morning, arrived NYC on the afternoon where Raquel was waiting for me. By Sunday, we had the very nice chance to meet with abrotman, rcox and stew. We stayed at stew and rcox’s place until Tuesday, when we got back. We also had the chance to meet with people that we have been working for remotely. We were walking around in NYC, and suddenly, the 5th avenue became multicolor. It was the gay pride parade. You can take a look at the pics on the gallery, lots of gay people. Our visit to NYC was pretty gayish actually, since we spent our first night in Chelsea, presenced an electronic music festival or similar for lesbians, and the next day, first row on the parade

And that’s it, since we came back until like yesterday, life was a mess; lots of work pending to deliver and many other issues. Finally, we are right now settled up again, enjoying a cold rainy nice weather in Mexico City with very cheap beer (compared with UK pints!) and loved people Thanks to all who made such a wonderful DebConf, even that, in my case, I arrived late and left early, it was just great (thanks also to MJ Ray, specially; and also to all people who was worried about my arrival, you know who you are, thanks). Thanks also to Mike and Rachel who hosted us, they are a very nice couple. And also, thanks to Alex who joined us in NYC, coming from Philadelphia, spent a nice day with us, took us to Chinatown and Little Italy and gave me a Phillies baseball cap (just as exchange of the worldwide famous umbrella hat).

Sometimes wandering through the netsphere you suddenly find small pearls you inmediately fall in love with. This is the case of Hex-a-hop, a small little cute puzzle game in which you move a girl through an hexagonal map, trying to break all the green tiles. The game has no time limit of any kind, and it does not involve any kind of real-time issues either. The visual appearance of the game is really great, a lot of care has been taken in the details, the graphics are really sweet, the water moves slowly reflecting the world, it really feels nice to play. For the moment, I’ve put a package available for Debian SID, until it goes into Debian’s repositories, in case someone wants to try it. Tom Beaumont, the author of the game, has really done a nice work

I (as root) have a directory hierarchy that I want a particular group to always have write access to. The files and folders inside the hierarchy are owned and manipulated by a wide variety of diffrent users. Essentially I want to delegate ‘root’ access for a portion of the filesystem to a particular group. My first attempt at implementing this was to use the standard POSIX ACLs that are available for almost every filesystem Linux supports. I recursively set an ACL on the top-level directory to give the group write access to all files and directories that currently exist and then I recursively set a default ACL to give the group write access on all the directories. This default ACL should be inherited by any new files that are created ensuring that the group keeps write access to everything. Problem solved? Unfortunately not. The intricacies of complying with POSIX means that ACLs are implemented as an ACL plus a mask. To gain access to a particular file or directory the user or group must match an appropriate ACL granting the access and the mask for that file or directory must also allow the requested permission to be granted. When you add an ACL to a file or directory, the ‘group’ bits of the standard Unix permissions magically switch from controlling group access to controlling the mask portion of the ACL, effectively providing an upper bound on the permissions that an ACL entry can grant. This prevents legacy POSIX applications that do not understand ACLs from unintentionally granting excessive permissions - arguably a good thing. Unfortunately this also makes it very hard to preserve the ACL granting write access to the ‘root’ group which I legitimately intended to have in place on this portion of the filesystem. Newly created files under the hierarchy generally inherit the ACL as intended, as most applications attempt to create files with as many permissions as possible, leaving it up to the umask to remove undesired permissions. However any file that is copied into the hierarchy without the ‘group’ write bit set, or any file that has the ‘group’ write bit removed via chmod will actually remove the write bit from the ACL mask invalidating the ACL and leaving me back at square one! After a bit of Googling I thought that NFSv4 ACLs might be the answer to this problem, as they are marketed as “very similar to Windows ACLs” and I’m sure that I vaugely recall Windows being able to properly inherit ACLs from parent directories. Unfortunately after downloading the NFSv4 ACL patches and trying all the various mount options I cannot find any combination that will offer the functionality I need. The implementation conforms to POSIX, so it still has a mask parameter and the same problems as the standard POSIX ACLs. The only benefit from using NFSv4 ACLs that I can see is that you have more permissions to grant. So once again, I’m back to square one. I’m hoping that there is some fundamental point that I’m missing as this seems like a very common use-case that I would have thought would be well supported. If a command-line example is clearer to you look at:
http://www.mattb.net.nz/blog/dump/acl-inheritance-problems.txt My current solution is to run a cronjob every X minutes to recursively ‘chmod -R g+w /dir’, however that’s far from optimal as it exposes all sorts of race conditions and just seems ugly! Any suggestions or solutions will be gratefully received.

I got the idea reading this question on the webkit-dev list, and from my recollection of the git documentation. Well, for the webkit case, the basic script I put up would be of no help because the tarballs don’t contain all the files (plus, they use subversion, not git, so it would also require a long importing process). Anyways… Considering how SHA-1 hashes of objects are created with git, it is actually pretty easy to generate a SHA-1 hash from a random (non-git) tree, and then, the corresponding commit. First, you have git-hash-object that helps you creating a hash for a particular object (though it’s also trivial to do with sha1sum). For regular files, git-hash-object -t blob $filename is enough. For symbolic links, you have to read the link destination, and give it without a trailing character (be it the NULL character or a carriage return) to git-hash-object -t blob --stdin. For directories, you have to generate a tree “structure” by yourself and pass it to git-hash-object -t tree --stdin. I haven’t bothered looking at other file types. The tree structure can be guessed by either looking at mktree.c or at the output of git-cat-file tree $sha1 where $sha1 is the SHA-1 hash for a tree object. It contains 3 informations for each node in the tree : the file mode, with the same format as what stat() returns, except for some reason, permissions are 000 for directories and symbolic links ; the file name ; and the SHA-1 hash. These informations are written with the following format : file mode in octal ascii and no padding zero (”%o”) followed by a space character, then the filename followed by a NULL character, and the binary form of the SHA-1 hash. Nodes are sorted in a not-so-quite lexical order (take a look at base_name_compare in read-cache.c) and are not separated by any special character: the mode of a file just follows the SHA-1 hash of its predecessor. With all this new knowledge, you should be able to write some code that would return the SHA-1 from an arbitrary directory. Okay, since you must be at least as lazy as I am, you can take the script I wrote. Now, let’s take a look at a real life case : what commit is the latest nightly snapshot for the linux kernel from ? First, download the latest snapshot patch and its baseline, and extract the whole. Then, run my git-hash-tree.pl script with the directory containing the extracted kernel as an argument. It will return, after a while, the SHA-1 hash for the whole tree. During this long process, you also have plenty of time to git clone linus’s tree. Once you’re all done, you can search for the commit corresponding to the tree hash (let’s call it $hash) with the following command :


git-rev-list --all while read h; do git-cat-file commit $h grep -q "^tree $hash" && echo $h && break; done


If you just followed these steps, you should just have spent a great moment having no result at all. There are actually 2 things that prevent this method to properly work with the linux kernel nightlies :

• The snapshot patches contain a change to the top Makefile that doesn’t exist in the repository. You need to remove the -gitn from the EXTRAVERSION variable in the Makefile.
• git diff only includes diff headers for removal of empty files, so if you apply the snapshot patch with the patch utility (and you can’t apply it with git-apply since you don’t have a .git directory), empty files that were marked as deleted will still be on your tree. It happens with the current snapshot patch (2.6.22-rc7-git6): it doesn’t remove include/asm-blackfin/macros.h.

Note this is a naive method, because I haven’t dedicated much time going through git documentation and code to find better ways, if there are any. Also note it’s pretty much worthless to do this with the kernel nightly snapshots, since a file containing the SHA-1 hash of the corresponding commit can be found alongside the patch. I guess a similar method could be used with mercurial, though I could not find a documentation detailing what are the hashes calculated from (I’ve not searched a lot, I must say, but for git, it was just before my eyes).

(or: finally, a use for ActiveRecord#save!) I recently found Rescue To The Rescue!, a short article on structuring your Rails controllers differently. Today, I started using it in some code I was writing that I think really benefitted from the change. The code was a big case that did things like this:

 case some_variable
   when 'X'
     process-based-on-X
     if save
       post-processing
     else
       throw_an_error
     end
   when 'Y'
     process-based-on-Y
     if save
       some-other-post-processing
     else
       throw_an_error
     end
   when 'Z'
     process-based-on-Z
     if !save
       throw_an_error
     end
 end

In each case, the pre-save processing was unique, and -- more importantly -- the action to take after a successful save was different, so I couldn't just save-and-handle after the case statement. However, the throw_an_error code was the same in each case, so I just changed the code to do this:

 begin
   case some_variable
     when 'X'
       process-based-on-X
       save!
       post-processing
     when 'Y'
       process-based-on-Y
       save!
       some-other-post-processing
     when 'Z'
       process-based-on-Z
       save!
   end
 rescue ActiveRecord::InvalidRecord
   throw_an_error
 end

My mind was blown. Soooo much clearer now. I have an "everything's OK" code path, and a "bugger me, something broke" path, cleanly separated. But something didn't feel quite right... And now I know what it is. I'm bothered by the fact that I'm sailing very close to using exceptions as an ordinary, every day flow control mechanism. I think I've convinced myself that, in this case, it's OK, but I'm a little concerned that I might go power-hungry and start using it in places that it really isn't warranted. I guess I'll just have to keep an eye on myself. Just remember, everyone: save! isn't a dirty word!

For some time I have ignored my blog so I did not notice that when my co-admin upgraded our system from etch to lenny, it broke. When I found some time to look at it, it turned out to be because the blog was no longer running in production mode, so it pointed at the empty development database. This caused every URL to redirect to the login page, but since I didn’t have a valid user for that db, I couldn’t login. This, in turn, was due to the upgrade of mod_fcgid from the old 1.10 version in Etch to 2.1. Apparently, this used to work. Note the name of the module:

<IfModule mod_fastcgi.c>
   <Directory /var/www/vir/syn.theti.ca/typo>
    FastCgiIpcDir /tmp/fcgi_ipc/
    FastCgiServer /var/www/vir/syn.theti.ca/typo/public/dispatch.fcgi -initial-env RAILS_ENV=production -processes 15 -idle-timeo
ut 60
   </Directory>
</IfModule>

It’s strange that the section is called mod_fastcgi.c, not mod_fcgid.c. Either that was done to make it a “drop in” replacement for fastcgi, and later changed in the next release, or else this config was not actually used, but somehow my blog started in production mode before the upgrade anyway. In any case, the blog is back in production node with the new fcgid with the following module section (based on this one from the mod_fcgid home page).

<IfModule mod_fcgid.c>
   DefaultInitEnv RAILS_ENV production
   <Directory /var/www/vir/syn.theti.ca/typo>
     FCGIWrapper "/var/www/vir/syn.theti.ca/typo/public/dispatch.fcgi" .fcgi
   </Directory>
</IfModule>

Hope this is helpful to others making the upgrade.

The Age has published a remarkably stupid article about terrorism titled “It’s hard to prevent the hard to imagine” which contains some amusing statements such as “a plan to use liquid explosives hidden in soft-drink bottles and destroy commercial jets crossing the Atlantic. The scale of this plot, combined with the innovative bomb design, threatened to kill thousands of people and cause massive disruption to global commerce“. This however has been debunked by many chemists, here is one of the many expert analysis of the claims. Here’s another inane claim from The Age “So the perpetrators in the UK looked elsewhere and compiled a crude yet potentially deadly bomb from materials available in everyday life a mix of gas cylinders, petrol and nails. Finding a way to govern access to such otherwise mundane items will be expensive, and perhaps ultimately, impossible.“. It’s surprising that The Age editorial team were unable to find someone who knows the basics of chemistry or who has ever had a need to start a fire to review the article. Everyone should know that an oxidising agent is necessary for combustion - and a huge quantity of an oxidising agent is needed for an explosion. Petrol vapour won’t ignite if it’s too concentrated (it can displace enough oxygen to prevent combustion). When a gas cylinder is damaged and set alight you get a fireball surrounding it which may be several meters wide (the one occasion that I witnessed a damaged gas cylinder on fire the fire-ball was about 4 meters wide). But no explosion. To propell nails or other solid objects you need the combustion to be rapid and only on one side of the object. Combustion on all sides (IE a 4 meter wide fire-ball) will not propell nails. Here’s what a British Army bomb-disposal operator has to say about it in The Register. If you want to talk about dangerous items that are difficult to control how about cars? A terrorist who drove a 4WD along the footpath of Oxford St could potentially kill more people than the London bombings and would take almost no preparation. The article inevitably concludes with claims about the efforts that al Qaeda operatives are supposedly making to recruit people for terrorist missions. Naturally it ignores the best recruiting method of al Qaeda - the huge civilian death toll in Iraq since the US led invasion. The best available medical research (by researchers from Johns Hopkins University and published in The Lancet - the most reputable and prestigious medical journal) estimates that there were 655,000 “excess deaths” as a result of the invasion in the period up to July 2006. Over the last year the best available reports suggest that the violence in Iraq has got worse (among other things the number of US air-strikes is twice what it was last year). For more analysis of the fear-mongering being done by much of the media (including The Age) here’s another article from The Register. It’s interesting to read The Age’s article Truth first casualty of the internet?. Maybe they should publish an article titled Intelligence first casualty of print media?.

Share This

Always when I have to deal with RPM systems I feel so handicapped and it feels so wrong from multiple PoV.

This is a rant and a dear lazyweb (not that lazy, actually) type of post.

I would like to be able to have a few things solved when dealing with RPMs:

1. I would like a pbuilder like builder for specs
• I know about mock, but
• it accepts only src.rpm files (I know about rpmbuild -bs, but is broken, read further for the groovy details)
  
• I have some specs that have %include directives for the maintainer scripts and when rpmbuild -bs is ran, guess what, the extra files are left outside the src.rpm;oh yes, the directives are processed correctly for the binary rpm
• mock relies on bootstrapping a system using fedoralegacy.org, which was recently discontinued
• I have a redhat-9 chroot tarball which I would like to use and tell mock to use that without caring about upgrades (since my packages are not at all that dependent on the build system)
• mock fails to bootstrap even for fedora-devel-i386-core with the message "Cannot find a valid baseurl for repo: extras"
• the older versions fail to bootstrap because, apparently, it "Could not find useradd in chroot, maybe the install failed?"
• plague (how's that for irony?) relies on mock, which is broken
• mach seems to be deprecated
2. I would like a simple repo creation and maintainance tool, something in the line of mini-dinstall, which should have apt4rpm and yum support, but it seems that repos are still the "new shiny thing" which has no proper tools
3. I would like to be able to know from which version I am upgrading from the current package, not only that I am upgrading (in case you didn't knew, the only way to know that you are upgrading in a post/pre install/remove script, is the wonderful $1 parameter which indicates the number of packages installed after the script finishes or something along that line..
   

So my question is, is there any way to fix any of the issues above in a sane way, without me reimplementing apt-like functionality in rpm land?

Dekoor, a student choir which includes our friend Jense, from Utrecht participates in a television show Korenslag, a pop-idol like contest for choirs. As we've been devoted fans of Jense from the "Miracle Man" up to now, we attended the semi finals of this program. Being in the audience of this show that is broadcast live on national television is an experience in itself. An "applause master" instructed us when to be silent, when to applaud, and when to go absolutely crazy. To which we faithfully complied. During the show Dekoor did very well in our unbiased ears, although the others did okay too. Although we SMS'ed a fortune to vote for Dekoor, the choir "Re-Flax" got the popular vote. Quite to my suprise because they were the least original and dynamic of the four, in my opinion. Luckily the professional jury was again impressed and decided that Dekoor must go to the finals together with "Re-Flax". Unfortunately the finals are decided by popular vote (eur 0,60 per vote) which might work against them. So remember, next Thursday, 20.30 Nederland 1: the finals. Vote Dekoor! P.S. I made some pictures aswell.

Mark Greenaway writes about writes about robots being designed to remove weeds from farms. This seems like a bad idea given that we currently have an energy crisis due to CO2 emissions from power plants causing environment change (including reduced water supplies), and coal and nuclear power plants requiring water to produce electricity (pity about the reduced water supply). Introducing robots that require either electricity of petrol for operation and also maintenance (which probably requires plastic parts on occasion) seems to be more of a problem than a solution. Fortunately there is already work in progress on training sheep to eat weeds. Given that we already raise sheep for wool and food it makes more sense for them to eat unwanted vegetation (weeds in a vineyard is the example given) than to grow grass specially for them. Also Mark says that the recent increase in oil price is not good news. What would be ideal is a slow steady increase in petrol prices that allows individuals and companies to change to more efficient vehicles (smaller and slower vehicles, hybrid vehicles, and more use of public transport) and also allows governments to make changes (such as building new public transport infrastructure). Having the oil remain cheap until it starts to run out doesn’t do anyone any good (apart from the short-term interests of oil companies).

Share This

So with the GPLv3 and LGPLv3 finally out (hurray!) it’s time to actually pay attention to the practical consequences of an upgrade to the world’s premier copyleft licenses. One of the things I hadn’t noticed, though I should have, is that the LGPLv3 is not compatible with the GPLv2 – that is, GPLv2 programs that used LGPLv2 libraries cannot make use of new versions of those libraries released under the LGPLv3 without also updating their license to GPLv3 (or “GPLv2 or above”, or LGPLv2 or LGPLv3 or something similar). Fortunately one of the major LGPLv2 libraries out there – glibc – is reportedly going to remain available under the LGPL v2.1 but for other libraries, it could become a problem, and it’s something GPLv2 authors will need to pay attention to. Also fortunately the opposite problem – GPLv3 apps using LGPLv2 libraries isn’t a problem, because the LGPLv2 explicitly allows redistribution and modification under the terms of GPLv2 or later, so you can take an LGPLv2 library and incorporate it in a GPLv3 application and just release the whole thing under the GPLv3. Unfortunately, the LGPLv3 doesn’t have a similar clause, so when the GPLv4 comes out, if it ever does, we could easily find the GPLv4 apps can’t use LGPLv3 libraries, and GPLv3 apps can’t use LGPLv4 libraries. Really these are side-effects of a fundamental change: up until now, almost everything we’ve distributed – and certainly all the low level stuff: the kernel, the C library, the compiler, the shell and basic tools – have all been under a GPLv2 compatible license. Some of them have been more flexible than that, due to being under a BSD or MIT license, or being under the LGPL or having a “GPLv2 or later” clause, but we’ve been essentially able to ignore all that and just treat everything as GPLv2. There are a few exceptions that we have to deal with – the occassional app that’s not GPL compatible doesn’t cause much of a problem, because things that depend on it don’t usually create copyleft issues, which just leaves checking that it doesn’t use any GPL-only libs, such as gnutls or libreadline. Openssl is a nuisance, in that it’s a popular, GPL-incompatible library, but it’s also reasonably easy to deal with on a case by case basis. But with the GPLv3, that changes: as GPLv3 apps and LGPLv3 libraries get uploaded, the assumption that more or less everything is GPLv2 compatible goes out the window, and while some things are GPLv2 only (such as the kernel!) we also can’t assume everything is GPLv3 compatible. Fortunately, both the GPLv2 and GPLv3 have what is known as the “system library” exception, which allows you to not bother distributing the source to your C compiler or C library or kernel with your app, because everyone’s assumed to have one of those already. Unfortunately, the GPLv2 version of that exception includes the phrase “unless that component itself accompanies the executable”, so Debian has always cautiously assumed the exception doesn’t apply to us, because everything we distribute in main “accompanies” everything else in main – kernel, C library, C compiler, etc. Equally unfortunately, while the GPLv3 avoids that problem, but ends up with a clause that’s not terribly comprehensible, despite a couple of rewrites during the drafting process. What’s all that mean? The big thing is that it means Debian’s going to have to put a bit more attention into the system library exception, both for GPLv2 and GPLv3 apps, because our current policy of trying to avoid needing it is probably not going to be sustainable. I’m not sure how in depth we’ll need to go for that; we’ve already contacted the FSF licensing folks for an explanation of the GPLv2 clause in relation to the CDDL, but it’s possible the FSF might prefer to be ambiguous in responding so as to keep their options open for defending or enforcing the GPL later, in which case we might need to get legal advice of our own as to what we should do. The second thing is that in reviewing our understanding of the system library exception, we might be in a position to have a Debian GNU/OpenSolaris port, making use of the exception for the CDDL-licensed OpenSolaris kernel and libc, as Nexenta do. That seems to me to be almost certainly possible for GPLv3 (or “GPLv2 or above”) applications, and probably possible for GPLv2 apps as well. If so, that’ll let us expand Debian to supporting four kernels in the near future: Linux, Hurd, FreeBSD and OpenSolaris, which will be pretty fun, in my book. The downside, is that if we find an interpretation of the system library exception that lets us mix and match licenses comfortably, that same interpretation might be able to be used to let propietary software vendors mix and match GPLed software with non-free software, rather than just free software with different copyleft licenses – and thus defeat the entire point of copyleft. What’s worse, is that by following Debian’s lead, they’d have some moral authority in doing that, beyond just having a legal argument as to why it’s okay. And that’s the rub. (My thoughts on the license texts as posted to -legal are here, here, here and here.)

A comment on my previous post about my Linux Jobs Blog suggested that I shouldn’t fragment the information. However I believe that fragmenting the information is ideal due to the ability of RSS syndication to drive the cost of coalescing the information to almost zero! Currently there is a Linux job web site run by Linux Australia. It doesn’t have many adverts and isn’t even linked from the main Linux Australia web site. I believe that we can do better for the people who want Linux jobs and the people who have such jobs to advertise. If you have a central site the jobs have to be moderated (which takes work and delays listing), the larger the area that the site covers the greater the work is to do this. The solution is to have a distributed system with different people running listings for various regions and a syndication service to aggregate them. To start this I have created a blog which will have categories for the states and territories of Australia. Someone who is only interested in one region can visit the category for that region. Then recruiting agencies and companies which regularly hire Linux people can start their own RSS feeds to be syndicated in a planet instance for each state and territory. This gives a faster and more efficient response (adverts will appear quickly, can be changed or removed at any time, and less effort for moderation. I expect that recruiting agencies will occasionally post off-topic entries - but when their feed gets removed from the Planet installation they will probably make a commitment to do the right thing in future. Planet installations can syndicate other Planet installations, so we can easily have a Linux jobs Planet for Australia (possibly run by Linux Australia) that syndicates the feeds from each state and territory. In the long term I think that the best way of running this is to have Linux Australia run the central Planet instance and a LUG in each region run the local site. I started running it myself because I didn’t get a positive response when suggesting it to the relevant people in Linux Australia and LUV. That didn’t deter me, so I decided to set it up myself. If the idea takes off and if Linux Australia and the LUGs want to take it over I’ll be happy to use HTTP redirects to send the traffic to them - and help them with the sys-admin work if asked. Also there is nothing specific to Australia in this idea. I am only interested in Australia because if I was to attempt to do it in a larger area (such as the EU) then I could spend all my time on it without gaining a critical mass in any region. If you are interested in running this in your region then you need to set up a blog (for adverts that are sent to you via email) and a Planet installation that feeds from your blog and any other Linux job advert blogs in your region. If your country doesn’t already have a central Planet for jobs then creating a separate Planet installation for your country would be a good idea too. I will be happy to run a Planet installation for world-wide Linux job adverts (at least until I can convince an organization such as Linux International to run it) if this idea takes off in other countries. Some people have asked what the benefit of a Planet is over a mailing list. You have to subscribe to a mailing list while with a Planet you can immediately visit it if you suddenly decide to find a new job. Subscribing to mailing lists for jobs from all countries would never work, but visiting a Planet for Linux jobs world-wide and then visiting a sub-planet for regional jobs would be quick and easy.

Share This

Just a couple years ago, music and technology companies would advertise their DRM schemes. While these technologies only served to prevent users of computers and consumer electronics devices from doing things, the media and technologies companies tried to spin it positively. Think of all the wonderful media that the music, film, and publishing industries will be willing to distribute to you at the click of a button, they said. All they asked for in return is the keys to your computer and the legal right to attack and sue you if you try to take control. As everyone who purchased iTunes music and made the mistake of buying a non-Apple DAP incapable of reading Apple DRMed music knows, DRM is a bad deal for consumers. Users are always better off with an unencumbered media file. In all the excitement over major label content, some consumers didn't see this immediately. With time though, the inconvenience of a computer that does the Apple and the RIAA wants over what you want hit home. This, combined with activist projects like the FSF's Defective By Design, have turned the tide. The DRM label that used to be a badge of honor is now a stigma that smart companies are going out of their way to avoid. This past weekend, I saw this flier from Calabash Music in the crepe store across the street: The store served a general, non-technical audience. DRM-FREE, it turns out, is a good way to sell music. Not just to geeks but to any consumer who has been stymied unfairly by DRM or knows someone who has. That, it turns out, is a whole lot people. Consumers know what DRM is and they know don't like it. As consumers learn more about DRM, they want to avoid it. Seeing this, the companies that produce DRM are looking for ways to escape. The Apple/EMI deal seems to be an attempt to protect market share that the use of DRM is threatening. Others, like HBO's Bob Zitter, are disingenuously attempting to escape the stigma of DRM by simply rebranding the technology. Of course, DRM suffers from a much more fundemental problem than bad branding. The problem with DRM is that consumers don't like what it does and are only sometimes willing to suffer through it when not given the choice. Increasingly often, as with in the example of the flier I found, consumers have a choice. Things don't look good for DRM. For DRM opponents, the self-defeating nature of the technology is our greatest ally.

With the development of rebuildd and the fact that several people are interested in re-using my scripts, I feel the need to explain how this stuff works. Grid’5000 First, Grid’5000 is a research platform used to study computer grids. It’s not really a grid (it doesn’t use all the classic grid middleware such as Globus). Grid’5000 is composed of 9 sites, each hosting from 1 to 3 clusters. Inside clusters, nodes are usually connected using gigabit ethernet, and sometimes another high speed network (Myrinet, Infiniband, etc). Clusters are connected using a dedicated 1OG ethernet network. Grid’5000 is in a big private network (you access it through special gateways), and one can access each node from any other node directly (no need for complex tunnelling). Using Grid’5000 nodes When you want to use some nodes on Grid’5000, you have to use a resource manager to say “I’d like to use 50 nodes for 10 hours”. Then your job starts. At that point, you can use a tool called KaDeploy to install your own system on all the nodes (think of it as “FAI for large clusters”). When KaDeploy finishes, the nodes are rebooted in your environment, and you can connect as root. At that point, you can basically break the nodes the way you want, since they will be restored at the end of your job. Running Debian QA tasks on Grid’5000 None of that was Debian-specific. I will now try to explain how QA tasks are run on Grid’5000. The scripts mentioned below are in the debcluster directory of the collab-qa SVN repository. When the nodes are ready, the first node is chosen to play a special role (it’s called the master node from now on). A script is run on the master node to prepare it. This consists in mounting a shared NFS directory, and run another script located on this shared NFS directory to install a few packages, configure some stuff, and start a script (masternode.rb) that will schedule the tasks on all the other nodes. masternode.rb is also responsible for preparing the other nodes. Which consists in mounting the same shared NFS directory, and executing a script (preparenode.rb) that installs a few packages and configures some stuff. After the nodes have been prepared, they are ready to execute tasks. To execute a task, masternode.rb connects to the node using ssh and executes a script in the shared directory. Those scripts are basically wrappers around lower-level tools. Examples are buildpackage.rb, and piuparts.rb. Now, some specific details:

• masternode.rb schedules tasks, not builds. Tasks are commands. So it is possible, in a single Grid’5000 job, to mix a piuparts test on Ubuntu and an archive rebuild on Debian. When another QA task is created, I just have to write another wrapper.
• Tasks are scheduled using “longest job first”. This doesn’t matter with piuparts tests (which are usually quite short) but is important for archive rebuilds: some packages take a very long time to build. If I want to rebuild all packages in about 10 hours, openoffice.org has to be the first build to start, since building openoffice.org takes about 10 hours itself… So one node will only build openoffice.org, and the other nodes will build the other packages.
• I use sbuild to build packages, not pbuilder. pbuilder’s algorithm to resolve build-dependencies is a bit broken (#141888, #215065). sbuild’s is broken as well (#395271, #422879, #272955, #403246), but at least it’s broken in the same way as the buildds’, so something that doesn’t build on sbuild won’t build on the buildds, and you can file bugs.
• I use schroot with “file” chroots. The tarballs are stored on the NFS directory. Which looks ineffective, but actually works very well and is very flexible. A tarball of a build environment is not that big, and this allows for a lot of flexibility and garantees that my build environment is always clean. If I want to build with a different dpkg-dev, I just have to:
  • cp sid32.tgz sid32-new.tgz
  • add the chroot to schroot.conf
  • tell buildpackage.rb to use sid32-new instead of sid32
• Logs and (if needed) resulting packages are written to the NFS directory.

Comments and questions welcomed :)

My wife was very excited to hear that debian will be celebrating the 10th anniversary of the Social Contract with a virtual pancake party. She could care less about the social contract, she's just happy to have an excuse to have me make her pancakes again. In any case, I thought I'd share my favorite recipe for pancake batter:

Oatmeal Pancakes	
2 eggs
4 T. veg. oil
1 3/4 cup buttermilk
1 cup all-purpose flour
1 cup rolled oats
3 Tbl. brown sugar
1/2  tsp. cinnamon
2 tsp. baking powder
1/2 tsp. baking soda
1/2 tsp. salt
Beat eggs in medium-sized bowl.  Beat in vegetable oil until well mixed, then stir in buttermilk.  Add remaining ingredients and stir to blend. 

It seems to me that we need to have syndicated feeds for Linux job adverts. To start this I have created a new blog for Linux job adverts, it will have categories for the states and territories of Australia (with a feed for each category) and I will also create Planet installations that take feeds from the blog as well as any other Linux jobs RSS feeds. I will create the Planet installations when I have feeds to add. If your company regularly advertises Linux jobs in Australia then please create an RSS feed of the adverts and I will syndicate it. If your company occasionally advertises Linux jobs then you can email them to me. As of this moment there are no positions advertised, but I hope that to change soon.

Share This

The Gnash GC (garbage collector) is enabled by default in CVS HEAD since yesterday’s night, and its performance nearly reaches the RC (reference count) speed, which is really good news I’ve made newer experimental packages for Debian (source here), in case someone is interested. As always, feedback is really welcome I hope the current versions of Gnash get out of the NEW queue soon.

Via the Planet Debian aggregator and his blog, Sven followed up on my post regarding Lucas' plot of the package age distribution. As some of my points didn't seem to make it across, I will reiterate them more plainly:

• GNUplot, while easy to use, creates charts that aren't terribly pretty;
• Lucas' original chart had, to paraphrase an expression by Tufte, a poor 'ink to paper ratio': the data is too concentrated in the last quartile;
• for that very reason, taking logs is a good thing here

Sven also addresses the fact that what we really want is to see the quantiles of the data set. Quite right, and taking logs makes that easier. Consider the two charts below which plot the 'package age in days' as an empirical cumulative distribution function using built-in R functions ecdf and plot.stepfun (rather than redoing it ad-hoc as I had done), and also add explicitly quantiles. The two charts use the exact same instructions; however the second chart transforms the x-axis to a logarithmic scale. While it is close to impossible to find the 25 or 50 percentile on the first chart, it becomes a lot easier on the second chart because the x-axis is 'stretched' using the log transform. About one quarters of the distribution appears to be rebuild within 1.5 months old, and about half is younger than four months (as a quick call to summary(pkgAge) confirms). Reading these proprtions off the original chart, or the non-log chart, is much more difficult.