Eddy, I think you’re getting too wrapped up in implementation details here. What all these revision control systems are doing is providing branches which are by some reasonable metric cheap to create and cheap to work with - the way they do it differs but to the user visible result is the similar if unremarkable. The main reason people bother mentioning this is that CVS has a problem with this since it stores data in RCS files but they aren’t really set up to handle branches well - they store everything as delta trees originating at the head of the current trunk. This means that operations on a branch require that CVS walk a very long delta chain to construct the current branch head which gets very expensive if there are a lot of changes between the two revisions, for example on a long lived maintenance branches.

Mid august, “responsability” will get a new meaning: my wife is into the 15th week of our third pregnancy.
We actually already should've become parents last november, but almost halfway through the first pregnancy the child had stopped growing and was diagnosed with trisomy 13, because of a hereditary Robertsonian translocation that Veerle turned out to be having. Short of waiting for the pregnancy to spontaneously abort, the only way to deliver the child was induced labour. We already knew from the karyogram that had uncovered the trisomy syndrome that it was a boy, though it was impossible to see that. The little fellow was about two thirds the length he should've been, and weighed about one third the normal weight.

We visited a fertility clinic a few weeks afterwards, but were told that the actual chance of us getting a healthy baby was a lot higher when we would naturally conceive than through in vitro fertilization. The only advantage of the latter method would be near elimination of the 50% chance to a failed pregnancy of natural conception. (For clarification: due to the translocation, there's a two-in-four chance that a conceived child carries an incorrect number of chromosomes; such embryos have a near-zero chance of making it through the pregnancy)

By the end of summer, Veerle was pregnant again. This time however it lasted for only seven weeks, after which the foetus died. Examination of a tissue sample again uncovered trisomy 13 syndrome. And it was again a boy.

Now it's the third pregnancy, and almost three weeks ago a chorionic villus sampling was performed to determine if this time the child would be healthy. Although a FISH (“fluorescent in situ hybridization”) test performed in the first two days showed no sign of problems, it was only after the liberating phonecall that came in yesterday that we could take a deep breath and smile. The baby has inherited the Robertsonian translocation though, but that doesn't lessen our joy.

As a side effect of the karyogram, we also know the gender weeks before ultrasound might reveal it. However, I haven't secured the necessary permit to disclose this information. I'll tell everyone in half a year from now...

Bus line #23 here starts in Mutxamel, goes through San Juan, reaches the Hospital, and then goes all the way to Alicante. Many months ago already, a new bus line was created: 34L. This line shares parts of its way with #23, namely from Mutxamel to the Hospital. After that, they diverge. After all this time, still today (and every other day too) many people mistakenly take 34L thinking it is #23. And it’s not as if the buses don’t carry a big yellow-on-black sign with their number...

I should've known that posting something about ID cards would spark a number of reactions. In fact, I did—which is one of the reasons why I posted that; nothing like a bit of discussion every once in a while. Brian M. Carlson writes:

The difference between the US and Belgium, for example, is that Belgium has comprehensive privacy legislation.

Well, yes. Obviously, ID cards require some privacy legislation in order to prevent abuse of said cards. In Belgium, only the police is allowed to require me to show them my ID card; everyone else can ask, but I can tell them to go screw themselves. Of course they can then also refuse to provide me with a given service, but I may be able to sufficiently prove my ID with other means. The point is, whether or not a designated ID card exists does not impact the need for privacy legislation; if you want to protect your privacy, you need such legislation anyway, ID or no ID. The fact that you don't have an ID card does not mean in any way that people can't ask you to prove your ID; and the existence of an ID card does not mean that all of a sudden all other ways for me to prove my ID are null and void. "Anonymous" writes:

You seem to take as a given that we need an official government-issued ID card, and from there you draw the fairly reasonable conclusion that if we need one it ought to exist separately from a driver's license. I agree that your assumption leads to your conclusion, but I disagree with your assumption.

Well, no, I do not assume that we need an official government-issued ID card. I know we do. That's not to say we need government-issued ID in all cases where we need to provide proof of ID; but when we do, there should be a way that does not overload some document into something which it was never intended to be. "bignose" writes:

You miss the main complaint of a national ID card system: that it would be *mandatory* for a person who wishes to operate in the country. That's the important part that is not true for a driver's license: not having a driver's license does not make the person an unperson.

It is not necessary for me to have an ID card to "operate" in Belgium. There are a few things where the ID card is indeed required; e.g., the last time I personally had to use my ID card was during the last elections. If I would have had an encounter with the police in that time, they could have asked for it, too; but that didn't happen. Also, when doing my tax declaration online, I used the electronic keys on the ID card, so that I could only update my tax declaration—and not, say, my neighbour's. But that's only doable when you have an electronic ID card system, which is not necessarily a good idea (in contrast to plain old paper ID cards, I do have privacy concerns with electronic versions—details on request) Of course, the fact that I can "operate" in Belgium without the use of an ID card does not mean it is legal for me to walk around without one; but that is not entirely the same thing. There were some more comments to my previous blog post, but I don't disagree with those.

Matt Bottrell writes about the Oracle Linux offerings presented at LCA 2008 [1] The one thing that Oracle does which I really object to is the “unbreakable” part of their advertising. They have pictures of penguins in armour and the only reasonable assumption is that their system is more secure in some way. As far as I am aware they offer no security features other than those which are available in Red Hat Enterprise Linux, CentOS, and Fedora. The unbreakable claims were also made before Oracle even had their own Linux distribution, which gave them even less reason for the claims. If someone is going to be given credit for making Linux unbreakable then the contributors list for the SE Linux project [2] is one possible starting point. Another possibility is that credit could be given to Red Hat for introducing so many security features to the mainstream Linux users before any other distribution. In terms of improving the security of databases it’s probably best to give credit to Kaigai Kohei and the PostgreSQL team for Security Enhanced PostgreSQL [3]. I believe that NEC also deserves some credit for sponsoring Kaigai’s work, I am not sure whether NEC directly sponsored his recent work on SE-PostgreSQL but they certainly sponsored his past work (and are credited on the NSA web site for this). Oracle’s Linux distribution is based on CentOS and/or Red Hat Enterprise Linux (RHEL). The situation with RHEL is that the source is freely available to everyone but binaries are only available to people who pay for support. CentOS is a free recompile of RHEL and a good choice of a distribution if you want a server with long-term support and don’t want to pay Red Hat (I run many servers on CentOS). While Matt gets most things right in his post there is one statement that I believe to be wrong, he writes “One of the craziest statements I heard during the talk was that Oracle will only support their products running under a VM if it’s within Oracle VM“. My knowledge of Xen causes me to have great concerns about reliability. My conversations with MySQL people about how intensive database servers are and how they can reveal bugs in the OS and hardware are backed up by my own experience in benchmarking systems. Therefore I think it’s quite reasonable to decline to support software running under someone else’s Xen build in the same way as you might refuse to support software running under a different kernel version (for exactly the same reasons). Matt however goes on to make some very reasonable requests of Oracle. The demand for native packages of Oracle is significant, I can’t imagine official Debian package support appearing in the near future, but RPM support for RHEL etc would make things easier for everyone (including Oracle). A better installation process for Oracle would also be a good thing. My observation is that most Oracle installations are not used for intensive work and use database features that are a sub-set of what MySQL offers. I’ve seen a few Oracle installations which have no more than three tables! The installation and management of Oracle is a significant cost factor. For example I used to work for a company that employed a full-time Oracle DBA for a database with only a few tables and very small amounts of access (he spent most of his time watching videos of fights and car crashes that he downloaded from the net). Adding one extra salary for a database is a significant expense (although the huge Oracle license fees may make it seem insignificant).

• [1] http://matt.bottrell.com.au/item/163/


• [2] http://www.nsa.gov/selinux/info/contrib.cfm


• [3] http://www.kaigai.gr.jp/index.php?sepgsql

Share This

ARPA just turned 50 and Ronda Hauben, long time author on the politics and culture of the net, has written a nice reflective piece on the significance of the organization of ARPA. Challenging the idea that military roots had dug into the organizational logic of the ARPA institution and especially the ARPAnet, she instead provides the following narrative:

The organizational structure of ARPA made possible the creation of the
computer science research office within ARPA begun by Licklider. That
office has demonstrated the importance of the support for basic
research in the field of computer science. The IPTO supported a
general area of research, one with a far reaching impact. The
achievements of this research office were not specific defense related
applications, nor were the goals narrowly aimed at defense specific
applications. If this reality is not recognized, however, it is
possible to mistakenly attribute significant computer science
achievements to defense specific objectives. A common and widespread myth exists that the Internet has grown out of
a defense specific objective, i.e. from the goal to create a computer
network that could survive a nuclear war. This is a striking example
of how a false narrative can spread and gain public credence. This false narrative finds its roots in the failure to understand that
ARPA was not an agency created for defense specific applications, but
to support the basic research which would lead to new concepts and
ideas.

With /usr/bin/screen -dm you can start a screen non-interactively, eg. from cron, passing the command to run and its arguments as normal arguments:

  @reboot    screen -dm rtorrent -p 12345-12346

However, I wanted to start from cron a screen that would run various commands, each in its own window. AFAIK you can’t do this with a simple invocation, but you can prepare a custom screenrc file that defines which commands to run, with the screen command. For example:

  % cat ~/.screenrc_foo
  escape "^Oo"
  startup_message off
  hardstatus alwayslastline "%w"
  screen 0 command0 -x -y
  screen 1 command1 -z -w
  screen -t windowname 2 command2
  screen -t windowname 3 sh -c 'cd ~/quux && ./wrapper.sh'

Then you start it from cron like this:

  @reboot    screen -dm -c ~/.screenrc_foo

Thanks to Luca Capello and Romain Francoise in #debian-devel for the tip. Update: Laurence J. Lane points on IRC the -X option to screen, so you could do without the screenrc file and do it like this:

  % screen -dmS NAME command0 -x -y
  % screen -S NAME -X screen command1 -z -w
  % screen -S NAME -X screen -t windowname command2
  ...

XQuery 1.0 and XSLT 2.0: news and how to use them in Debian (long post about the second generation of the XSL family and how to use the related languages in Debian)

• XPath 2.0, XQuery, et al.
• the above stuff .. in Debian
• links

---

The 2nd generation of the XSL family At the beginning of 2007, W3C released a family of interrelated specification: XPath 2.0, XSLT 2.0, XQuery (1.0). The 3 specifications are based on the very same underlying data model which (finally!) supports typed values and exploits them to some extent, for example permitting some form of static type checking in XPath and XQuery. Types usually come from XML Schema, with both built-in and user-defined types supported. A brief overview of the 3 specifications and what they change in the state of the art follows. XPath 2.0 XPath 2.0 basically consists in XPath 1.0 + 3 new macro features. The first one is a revamp of those language features which are dealing with types, in order to better integrate them with the new typed data model. XPath now has operators to check whether a value belongs to a given type (so yes, type information are kept at run time), and to cast them from one type to another. The basic new type constructor is now the sequence, which replaces the old node-set solving many annoying issues, such as the impossibility of having a node ordering other than document order. The second feature is the improvement of standard library of functions which was ridiculously small in XPath 1.0; it is much better now. Additionally, 2nd generation languages (XSLT 2.0 and XQuery) now support the ability to define functions which will then be visible to inner (XPath) expressions, pushing yet forward the possibilities of plain old XPath. Functions can now also declare types in their signatures (for their arguments and return value) and untyped arguments will be automatically casted to them upon invocation. Hence, even if you are not using an implementation which assigns types to your XML trees, once you "enter" the typed world calling a typed function (almost all standard library functions are decently typed) you will be able to stay there avoiding annoying casts everywhere. Finally XPath 2.0 has turned into a powerful purely functional language and is now powered by constructs like conditionals, for-each loops, existential/universal quantifiers, and existentially-quantified comparison operators for sequences. Here is a complex expression to hwet your appetite (or scare you away ...), comments come as (: smiley faces :)

for $book in /bookshelf//books
return
  if ((every $author in $book/authors/author
       satisfies $author/nativeLang eq "it_IT")
      and $book/lang eq "it_IT")
  then $book
  else ()
(: think about the trouble of writing this in XSLT/XPath 1.0 ... :)

XSLT 2.0 XSLT 2.0 is what I would call a "bug fix release" of XSLT 1.0 + the routinary reworking of the language to deal with typed values, which is not sensibly different than what has been done for XPath 2.0. The fixed "bugs" are several, starting from the annoying issue of result tree fragments. They are basically tree snippets that in XSLT 1.0 you were able to generate for future use. Unfortunately they were not thaaat reusable, given that you were not even able to navigate them with XPath operators! Now the specification is much more clear and distinguishes final result trees from ordinary variables, which can now contain sequences of (navigable) tree nodes. Another important "bug" fixed is the new ability to output multiple documents with a single XSLT stylesheet: it marks the end of stupid extra post processing to be added in pipeline to a XSLT processor. Other minor "bugs" fixed are a limited amount of backtracking capabilities among imported templates, regular expression support directly in the language, and powerful grouping constructs on the lines of SQL's GROUP BY (but much more powerful). Here is a template snippet exploiting the latter feature:

<xsl:for-each-group select="*" group-starting-with="h1">
  <div>
    <xsl:apply-templates select="current-group()" />
  </div>
<xsl:for-each-group>

XQuery XQuery is the end of the chains imposed by XML-based syntaxes. Why the heck one has to use an XML syntax (as in the above snippet) only because she is manipulating XML tress is one of the mysteries of XML technologies which have always been floating around in my head. XQuery is the (supposedly) SQL equivalent for databases of XML documents, but is actually much more than that. I depict it in my head as the XML manipulation language with a syntax I can finally stand. Technically it is XPath 2.0 (say 80% of the whole language) + some extra ingredients (say 20%); so remember that every XPath 2.0 expression is also a XQuery expression. The main extra ingredient is the so called FLWOR expression (to be read: "flower expression", which in addition to the "smiley faces" used for comments gives a "back to 1968"-flavour to the language .... erm FLWOR/flower/flavour, no pun intended). A FLWOR expression is very similar to SQL's SELECT-FROM-WHERE: it lets you generate a tuple stream by iterating on sequences (F: for clauses), binding expression values to names (L: let clauses), filter out tuples which do not satisfy a required condition (W: where clause), order the survived tuples (O: order by clause), and finally return a sequence built using the residual tuple stream (R: return clause). The other interesting extra ingredient is the ability to build the XML snippets you want to manipulate. Within XQuery you do that using plain XML syntax (the only place where a sane-minded programmer actually wants to see it!) which also supports a classical interpolation mechanism to embed expressions which will be evaluated inside XML snippets, and also the other way around. A canonical XQuery example is:

for $t in doc("books.xml")//title,
    $e in doc("reviews.xml")//entry
where $t = $e/title
return <review>  $t, $e/remarks  </review>
(: braces denote the escaping context where XQuery expressions will
   be evaluated inside snippets; plain XML syntax is used for the
   other way around :)

But remember: XQuery for XML is much more than SQL for RDBMS, thanks to the implicit templating mechanism implemented by interpolation, and thanks to several language features fostering modularity (user-defined functions, library modules, XPath 2.0 standard library, ...) you can basically do with it any kind of XML manipulation you can imagine. I don't think I will ever write myself any other single line of XML output in DOM or XSLT ...

---

Cool, how can I use it in Debian? ... if only this stuff were decently supported in the open source world. Last time I checked, the author of most parts of the GNOME toolchain for dealing with XML (libxml2, libxslt, ...) was not intentioned to implement XSLT 2.0, not even mentioning XQuery. This comes as no surprise, the whole GNOME XML toolkit is written in C, and XSLT 2.0 / XQuery have reached a level of complexity and formal specification which usually entails a higher level approach. So on the GNOME side we are stuck. The other open source implementations of XSLT 2.0 / XQuery I'm aware of are Saxon and Galax. Saxon is an XSLT 2.0 and XQuery implementation written in Java, which is unfortunate per se. Additionally, it is also unfortunate that it is only partially open source. Indeed, Saxon is split into SaxonB (for "basic") which is open source under the Mozilla Public License and SaxonSA which is commercial. While SaxonSA is a fully conformant, XML Schema-aware processor, with support for static typing, SaxonB is a basic-conformant processor with no type-aware features and actually much less optimized than SaxonSA. This is annoying. (Much more annoying is the fact that SaxonSA's author is the only editor of the XSLT 2.0 specification and that instead of his mail address the specification includes an URL pointing to the website selling SaxonSA ...) Galax is an open source (IBM CPL / Lucent license) OCaml implementation of XQuery which is not fully conformant to the specification (though it gets quite close) which is type-aware and implements static typing. The only XQuery / XSLT 2.0 implementation available in Debian at the time of writing is SaxonB. The binary package is libsaxonb-java, kudos to Michael Koch and the Debian Java Maintainers for having packaged it (and to have stood some annoying pings of mine bug #408842). To execute XQuery code you just have to aptitude install libsaxonb-java, prepare a query.xq file containing your query, and then execute something like:

CLASSPATH=/usr/share/java/saxonb.jar \
java net.sf.saxon.Query query.xq

note that the information in README.Debian are still referring to old Saxon versions, see bug #465894 which proposes a more up to date README.Debian. Similarly, to perform a XSLT 2.0 transformation you have to do something like:

CLASSPATH=/usr/share/java/saxonb.jar \
java net.sf.saxon.Transform -ext:off -s:input.xml -xsl:style.xsl -o:output.xml

Do not remove the -ext:off flag when processing untrusted stylesheets!, see bug #465885 for the reason. I've written some handier (1-liner) shell script helpers which remove the need of invoking java manually. They are attached to bug #465894 and I've proposed their addition to the saxonb package. Using them the above invocations become:

saxonb-xquery query.xq
saxonb-xslt -ext:off -s:input.xml -xsl:style.xsl -o:output.xml

Galax in Debian ... well, since long time I've been planning to package it (the ITP has been filed some months ago: bug #447984) and the authors sent me a newer version than what is available online to gather feedback before the long overdue final release. Unfortunately I've been lagging behind in finishing the packaging (which is tricky due to the need of binding libraries to several different languages: OCaml is native, but there is also Java for example). Hopefully this post will give me some renewed motivation for finishing the work ...

---

References

• Debian package w3-recs, ships the whole list of W3C Recommendations for offline consultation; it includes all the specifications we have discussed in this post
• Saxon: home page of Saxon, a XSLT 2.0 / XQuery processor written in Java
• Galax: home page of Galax, a XQuery processor written in OCaml
• Debian package libsaxonb-java, ships SaxonB as a Debian package

---

Acknowledgements Thanks to godog for his helpful comments.

---

Update: the helpers I've proposed have been accepted into the official package, I've also made available their manpages. Also the pending changes to README.Debian has been accepted. Kudos again to Michael Koch for his quick feedback on my patches!

So today I'm a little bit lazy and I've got the day off work. As my previous plan suggested I wanted to spend at least some of the day tackling semi-random bugs. Earlier I picked a victim: less. less rocks, and I use it daily. I even wrote an introduction to less once upon a time. So lets take a look at two bugs from the long-neglected pile. These two issues are basically the same:

• gzip: zless seems spew raw gzip to output for empty files
• zless thinks gzipped empty file is binary

They seem like simple ones to fix, with the same root cause. Here's an example if you want to play along at home:

 cp /dev/null testing
 gzip testing
 zless testing.gz

What do you see? I see this:

"testing.gz" may be a binary file.  See it anyway? 

When I select "y" I see the raw binary of the compressed file. So, we can reproduce it. Now to see why it happens. /bin/zless comes from the gzip package and is a simple shell script:

#!/bin/sh
# snipped a lot of text
LESSOPEN=" gzip -cdfq -- %s"; export LESSOPEN
exec less "$@"

So what happens if we run that?

$ LESSOPEN=" gzip -cdfq -- ~/testing.gz" /usr/bin/less ~/testing.gz
"/home/skx/testing.gz" may be a binary file.  See it anyway? 

i.e. it fails in the same way. Interestingly this works just fine:

gzip -cdfq -- ~/testing.gz   less

So we've learnt something interesting and useful. We've learnt that when LESSOPEN is involved we get the bug. Which suggests we should "apt-get source less" and then "rgrep LESSOPEN ~/less-*/". Doing so reveals the following function in filename.c:

	public char *
open_altfile(filename, pf, pfd)
	char *filename;
	int *pf;
	void **pfd;
 
/* code to test whether $LESSOPEN is set, and attempt to run the 
   command if it is */
		/*
		 * Read one char to see if the pipe will produce any data.
		 * If it does, push the char back on the pipe.
		 */
		f = fileno(fd);
		SET_BINARY(f);
		if (read(f, &c, 1) != 1)
		 
			/*
			 * Pipe is empty.  This means there is no alt file.
			 */
			pclose(fd);
			return (NULL);
		 
		ch_ungetchar(c);
		*pfd = (void *) fd;
		*pf = f;
		return (save("-"));

That might not be digestible, but basically less runs the command specified in $LESSOPEN. If it may read a single character of output from that command it replaces the file it was going to read with the output of the command instead! (i.e. Here less failed to read a single character, because our gzipped file was zero-bytes long! So instead it reverted to showing the binary gzipped file.) So we have a solution: If we want this to work we merely remove the "read a single character test". I can't think of circumstance in which that would do the wrong thing, so I've submitted a patch to do that. Bug(s) fixed. Incidentally if you like these kind of "debuggin by example" posts, or hate them, do let me know. So I'll know whether to take notes next time or not..

I enjoyed reading the ArchitectNotes on the Varnish wiki:

Take Squid for instance, a 1975 program if I ever saw one: You tell it how much RAM it can use and how much disk it can use. It will then spend inordinate amounts of time keeping track of what HTTP objects are in RAM and which are on disk and it will move them forth and back depending on traffic patterns.

Well, today computers really only have one kind of storage, and it is usually some sort of disk, the operating system and the virtual memory management hardware has converted the RAM to a cache for the disk storage.

So what happens with squid's elaborate memory management is that it gets into fights with the kernel's elaborate memory management, and like any civil war, that never gets anything done.

Varnish was designed by FreeBSD developer Poul-Henning Kamp. There is something to be said about user-space apps designed by kernel developers: they're always super fast because kernel developers know how to make the most of the system, having spent hours optimizing the kernel itself for some usage patterns.

This is why git (designed by Linus Torvalds, for those of you who've lived under a rock for the past three years) is so much faster than other comparable systems, for example. It uses tricks such as giving the O_NOATIME flag to open() to avoid updating access times on files where it knows that they don't matter, and it's a "huge time-saver". (Of course, you can also just mount your partitions with noatime and enjoy a similar performance boost in all applications!)

I spent a while yesterday thinking over the software projects that I'm currently interested in. It is a reasonably short list. At the time I just looked over the packages that I've got installed and the number of bugs. I'm a little disappointed to see that the bugfixes that I applied to GNU screen have been mostly ignored. Still I have the day off work on Thursday and Friday this week and would probbly spend it releasing the pending advisories I've got in my queue, and then fixing N bugs in a single package. The alternative is to build a quick GPG-based mailing list manager. I'd like a simple system which allowed users to subscribe, and only accepted GPG-signed mails. The subscriber could choose to receive their messages either signed (as-is) by the submitter or encrypted to them. So to join you'd do something like this:

subscribe foo@example.org [encrypted]
--BEGIN PUBLIC KEY --
...
--ND PUBLIC KEY--

There is the risk, with a large enough number of users, that a list could DOS the host if it had to encrypt each message to each subscribers. But if the submissions were validated as being signed by a user with a known key it should be minimal, unless there is a lot of traffic. The cases are simple:

• foo-subscribe => Add the user to the list, assuming valid key data found
• foo-unsubscribe => Do the reverse.
• foo:
  • If the message is signed accept and either mail to each recipient, or encrypt on a per-recipient basis.
  • If the message is not signed, or signed by a non-subscriber drop it.

There are some random hacks out there for this, including a mailman patch (did I mention how much I detest mailman yet today?) but nothing recent.

The problem with the Internet is that it never goes to sleep. And if you study the Internet as I do, that means spending a lot of time starting at the screen tagging, copying, pasting, downloading, watching, and assessing. I have taken a step or two back away as I have moved from life as a postdoc to working full time but ever since Anonymous declared war on Scientology last month, I have been spending more time online again (more than I want to at least), tracking and following some of their developments, including spending time on some of their IRC channels, downloading lots and lots of videos as well as all sorts of great flyers. Well today there was an IRL action loosely coordinated by Anonymous to protest the CoS and I have to say, I am simultaneously surprised and unsurprised at what I think was a pretty vigorous response and turnout. I was not too surprised as the attack against and criticism of Scientology is one of the older wars spurred on in part by the net and net ethics and it always seems to crop of from time to time. But I think the depth of smack talking I witnessed on IRC, (often clearly youthful in nature as this example reveals—>, [mizzanon] I will have my full license in approx. 3-4 weeks, as soon as I do, i will be driving around my entire region with tons of
anti-propaganda) led me to believe that they could not pull of this global day of action. But they did and perhaps with the exception of New Zealand (where there are probably better things to do anyway), they seemed to garner a fair bit of participants. It will be interesting to see how long they will continue to generate this type of support. The next day of action will be around the middle of March.

Today I've been mostly unwell. Although I have managed to write some minor new code, and watch a little bit of Doctor Who on DVD. Recently several people have been ranting about Ruby on Rails. I like it, but I wouldn't use it for personal development in a hurry. Deployment is fiddly, and upgrades are annoying. But one thing that I utterly condemn Rails for is helping to spread bad paging throughout the online world. So, what is "bad paging" and why is it important? Well cool URLs don't change, right? "Bad paging" is any user-interface which presents you with a limited view upon a changing list of items which is non-bookmarkable. Consider the following "list". Assume it represents your view of a collection of items numbering 100+. You may only view ten items at a time; clicking "next", or "previous", to navigate your viewport:

1.  first item
2.  second item
..
10. tenth item
[see next: /start/1] [see prev]

Whats wrong with this picture? It is subtle, but this list is broken. The issue is that when the list grows new items are prepended to the front, yet the navigation is linked to the starting page number. If that description wasn't clear consider what happens if you want to bookmark the page containing item 11. How can you? Right now it is at /start/1. If a ten new items are appended to the head of the list then it will instead become /start/2 - as items that are currently numbered 1-10 will be shifted forward to become items 11-20, and and they will be on page /start/1 instead. The solution is simple enough once you consider what you want to happen:

• Either append items to the end of the list.
  • Such that /start/1 always gives the items 11-20.
• Number the links in the reverse order.

So why does nobody do that? (As a counter example look at my website: Rather than the 'Show previous' items linking you to the changing link /start/1, it instead links you to /start/569 (for example).

Earlier today I watched the presentation Jonathan Oxer's gave during LCA about Package caching solutions. Although it was certainly an interesting presentation and although I very much agree that my current local mirror is wasting a lot of diskspace and bandwidth, I'm still not going to switch from debmirror to any of the available caching solutions, because (unless I'm really missing something) none of them scratches my itch. My local mirror currently consists of five architectures (i386, amd64, hppa, sparc and s390) and only has unstable and testing. I use it for:

1. (fast and convenient) updating of my systems
2. doing Debian Installer builds and installation tests
3. test builds of installation CDs (using debian-cd)

Now, the last one is somewhat hard (debian-cd uses hardlinks to packages on a local mirror instead of retrieving packages), so let's concentrate on the first two. Caching is great if you have a large number of machines – of the same architecture and that are all likely to need roughly the same packages – sitting behind the proxy: the first one triggers the download of the package and the rest gets it almost instantaneously. It is a lot less great if you have only one, maybe two systems per architecture: most of the time you'll still end up going down that (relatively) slow ADSL connection. An important reason why I have my mirror is so that when I do my daily updates for sid or run an installation test, the packages are already available locally. I really don't want to double or even treble the time needed for installation tests just because some required packages aren't yet available locally and need to be downloaded over that slow line. So, I have my partial mirror. Somewhat tuned (I exclude some ridiculously large debug packages for example, saving about 10GB), but still with a lot of junk^Wpackages on it I'll never ever use, especially for hppa, sparc and s390 as those systems only have fairly basic installations. Getting rid of that would significantly reduce my daily sync and allow me, for example, to also have a mirror of stable and oldstable, keep old versions of packages and probably still save a lot of diskspace. Wishlist What we should have is a hybrid solution: a program that will present itself as a proxy to clients, but is smart enough to pre-fetch new packages that are likely to be needed in a sync run, based on usage date from the the proxy and configuration settings. Some ideas for features/configuration options it could have:

• should support both source and binaries (debs and udebs)
• transparently retrieve packages that are requested but not available locally (just like a regular proxy)
• options to always sync (per architecture):
  • all required and standard packages
  • packages listed in a certain config file
  • packages in a certain section (e.g. udebs in the debian-installer section)
• for packages that have been used (requested by clients) within the last X days: pre-fetch any new version (per architecture)
• options to include security/volatile updates in that
• be smart about ABI changes: when a package name changes because the ABI version changes, also pre-fetch the new package; an alternative solution for that could be:
• for recently used packages: ensure that their dependencies are also pre-fetched; this would also provide support when packages are renamed (dependencies in the transition packages would ensure the new packages are pre-fetched)
• expire packages from the mirror that have not been used for X days
• allow faster expiration of certain types of packages (-dev or -dbg for example)
• options to keep X previous versions that are no longer referenced in the Packages files for Y days

Possibly such an implementation could even be used on some of the lower tier Debian mirrors. Unfortunately, unlike some of my esteemed colleagues, I'm not able to just whip up something like this, so I'm condemned to wait and see if there's someone else who'd like to pick up this idea. I am of course more than willing to help develop this idea further and to test it. Now, if I've totally failed in my research and something like this already exists, a pointer in the right direction would be much appreciated.

How to work with python-m2crypto This is a little howto on how to understand the m2crypto Python module, beyond the examples provided:

1. Build the documentation as explained here
2. Look at the documentation, which does not contain a single comment
3. Click the links that show the source (for example, of M2Crypto.SMIME.SMIME.verify)
4. Look what openssl function gets called by the function you are interested in (for example, pkcs7_verifysomething))
5. Google the name of that openssl function, trying a few variations, until you find the openssl manpage that tells you what it does, in detail (for example, pcks_verify)
6. Understand what it does, and backport the understanding to the thin convenience layer that m2crypto adds on top of it
7. Realise that now you also understand the corresponding openssl smime -verify commandline invocation, and that next time instead of reading the openssl manpage you should look at the openssl api docs instead.

So, I'm back from Holidays, to find we now have a new Government in Ireland. (An unexpected choice of government, of which more later.) We were camping in Fouesnant, in Brittany; Family & Friends might like to see the holiday snaps here). Camping in France went well. With three children aged 7, 4 and 2, there is only so much you can expect from a family holiday. Like it or not, you have to mind the toddler fairly consistently, and the older kids would like your full attention too. This is a holiday ''for'' the kids; plan also a break ''from'' the kids later (a long-weekend, with kids at grandparents, for example). This was our first time camping abroad, and its well recommended. While we went touring with a camper van with the two older children two years ago, this is different in that we were at the campsite for almost two weeks: the kids got to meet and play with others in a way they couldn't before. Also, there were good facilities at the site in Fouesnant, and the swimming pools in particular were appreciated (Caveat emptor though: there were no lifeguards or supervision. We went off-peak, and there were apparently two accidents in the time we were there: a broken kneecap and chin when people came down the slides too soon after others and collided). It was the first time the children (a) had pocket money to spend, and (b) were able to walk to a (campsite) shop to spend it. Lessons were learnt in the finiteness of pocket money, especially by Caoimhe . The weather was fairly good: two wet days (though not overly so, compared to what we're used to in Galway); enough to give the 'authentic camping experience'. And for the children especially, just warm enough. Long drives South were not an option with young kids, and we wanted to see Carnac and Brittany in general. We went touring to two places: Mont St. Michel, on the way down, Carnac, and locally around Quimper and Concarneau. At seven, Cormac is beginning to be old enough to enjoy this, but Sadhbh at 2 is too young, and easily aggravated. Caoimhe can appreciate some of it, such as the well recommended Breton museum in Quimper, seeing the local costumes and clothing is just her style at the moment. We're looking forward to going back when the kids are older, or we can enjoy touring on our own. The camp site is dominated by English / Dutch / Irish. It is still term-time in Britain and Ireland, which meant the children on the site was full of 2-4 year-olds, with a shortage of playmates for Cormac. Next time, synchronise with English school times, etc. We went during Cormacs term-time because it was much cheaper (1000 Euro) than high-season. If Cormac was having difficulty in school things would have been different, as it is, we had no qualms in doing so. Things will probably be different next year with Caoimhe also in school at the same time, and we would go later. Suffice it to say I think synchonisation of school timetables is a bad idea, as it has led to these huge price differentials and parents taking their kids from school. As a friend with experience of the French school system points out, they've spent the last 40 years desynchronising their timetables ... So would we go again, or buy our own tents, etc? possibly. The brochure for next year arrived already, with large discounts for booking early ... Tags holiday, family, france, brittany, cormac, coimhe, sadhbh

Jaldhar Vyas points to the rogues gallery of oil producing states and argues that Neoconservatism is good, well at least it gets rid of the dictators such as Saddam Hussein while it grabs the oil. As he ightly points out, we are researching new alternative energy sources, and just as importantly (or even more importantly) energy conservation. But we have been doing this for decades. Sooner or later we have to realize that the "Oil Crisis" isn't due to a shortage of energy sources: its due to deliberate politics to make us dependent on oil. Peak Oil is "only" a crisis for us consumers, dependent on oil to drive our cars, heat our houses, and supply our food. For oil companies, it means (1) they are shipping oil at a greater rate than ever before, (2) at higher demand than ever before, and (3) in circumstances where pesky politics like OPEC can't interfere and try and moderate the price down (I'm not arguing OPEC or cartels are good; its just that its long-term 'maximize the benefit for the oil producing nations' strategy interferes with oil companies shorter-term profit maximization). For the Oil Companies, its a good thing. In this context we have to see our current dependence on oil as the result of deliberate policy by the oil companies. The small filip that USanian governments have done towards alternative energy, such as funding Hydrogen fuel-cell research, nuclear power, etc. are so long-term, and so small in scale, that they have to be seen as deliberately not solving the problem. Thinking that research into alternative sources is good is gpoing to save us (disclaimer: as a scientist I think research is a good thing :-)) is missing the real problem. Its politics: the oil companies have simply been better at politics than the strugging alternatives; we need to recognize this, and fight for alternatives. When oil prices reach $100/barrel, and we struggle to pay rising food and energy bills, its because other have deliberately caused the crisis. Oil has often been called a Curse because of the effects 'Oil Wealth' have caused so many countries. They are not co-incidentally dictatorships; Saddam Hussein remained in power in Iraq for so long because it benefitted the West (see the weapons supplies to Iraq in the so-called "Iran-Contra" affair, for example). Ditto Saudi Arabia, remaining 'friends' of the USA despite its ties to 911. Overthrowing Saddam Hussein was about consolidating even more power in the Gulf than America had previously, not democracy. What was planned for Iraq was democracy in name only: all real power was already neutralised, with decisions to US benefit cemented in place before the parliament met: non-negotiable contracts already signed giving US companies control. This is straight imperialism, and cannot be considered democracy or a good thing. Tags environment, oil, politics

This week I have been primarily looking over ticketing systems. So far I've looked at the following:

roundup

Initially this was the package that I liked best. It is small, self-contained, and easy both to install and manage. The significant downside was that it expected all users to create an account first, prior to submitting a ticket. If they didn't and they mailed submit@example.org they'd receive a reply back which read "Unknown user: sender@example.net". Shame because it was reasonably featureful.

otrs2

This was a little fiddly to get going with, but actually suprisingly good. Had I not experimented with roundup I'd have said it were the best I looked at.

request-tracker

Big. Heavy. Intense. It looks ugly, but provides a bamillion (metric) configuration options and can be used to do almost anything you'd like. If it didn't such up so many resources I'd have had no qualms about using it. Even the mail-gateway was nice and simple to get working.

I'm going to take a look around some more and see if there are any non-packaged systems to look at. Pointers welcome. All I want is for incoming emails to appear in some sort of queue where they may be assigned to an owner. If the original submitter can view the ticket online great, but no big deal if not. I'd expect that replies sent to the ticket would be routed back to the submitter, and that replies to those would similarly update the ticket. I guess thats standard though?

No Debian content here. Just like last year I decided to watch one of the largest sport events in the USA - the final game of the National Football League Super Bowl XLII (roman numerals are different for the sake of it ). I came to like the Japanese take on American football, so I watched the Superbowl last year and was appalled at the lack of any entertainment value in the game. I was told that it was one the worst Super bowls ever, so I gave it another try. Just like last year the American football looks to be very lawyer-friendly: bundles of complex rules and exceptions. The game could have been much simpler. Like for example, a lot of rues come from the fact that the game timer almost never stops. It is good for TV companies as they can plan their broadcast times better, but horrible for the stop-go game like the American football is. Just stopping the timer on every point the ball is stopped like they do in final seconds would cut out a lot of rules. Playclock and all. Rules on use of timeouts and stuff. Of course the game time would need to be adjusted down a bit. Now you have 4×15 minutes of ‘game’ where at most 20 minutes of that is in action. That game time is a lie - most of the time people are just standing around. Terminology irks me quite a lot. Why is it a “1st down” when there have been two “1st down”s just a few minutes ago? Something along the lines of “down counter reset” would be much clearer. And what is cryptic “1st & 10″ doing in the middle of the screen? It would have been much clearer if that info would be split: “Downs: 1/4″ and a measuring arrow between the ball and the yellow line with “10yd” written on it. And then there is an obscured “two in two” or “two for two” or “two in O” phrases which apparently means the same as “two 1st downs out of two tries” in the context. Can’t the commentators be a bit more elaborate and actually say the units of measurement? “Giants strength first tonight”? How about “in the lead”, English anyone? The camera placing editing is just horrible - the camera direction is switching all the time and that is confusing as hell. In normal football when professional camera teams are working, you always get all the shots from one side and even one general angle. If for clarity reasons a camera shot from other side is needed (usually in a replay) they always say that the image is from the reverse side in large letters on the screen. Here a lot of close-up shots were all over the place and even rotated around inside one shot! That might look fancy, but is just crappy camera work and editing. Then commentators start talking walls of numbers and walls of names. Unless you know by hearth who plays at what position, there is no way to understand what those people are saying. Than is just sloppy commentary. Is FOX incompetent in all possible ways? Then again the funky rules came into play just at the end of the first quarter - I have no idea why the ball was moved to the end line. Apparently the defender did or didn’t something. Did not see anything worth penalizing the team like that at all. They dropped the term ‘pass interference’ later on. Isn’t interfering with the pass the whole point of the defense play in this game??? Lawyer game, I’m telling you. And later on, pulling on guys arm while he is catching and being tackled apparently is just fine. And holding is illegal??? On the big penalty on the Giants the judge could have just said that the ball is out of bounds. Giving a 10yd penalty for such a minor thing is just ridiculous. The halftime show was kind of meh. Staged to extreme, but no soul nor sense. And what were all those people doing on the field? Isn’t that bad for the grass? And the commercials .. Average at best. Horrible at worst. For example a government commercial saying parents to protect their prescription medicine to .. help out your neighborhood drug dealer??? Oh and does anyone still doubt that media is controlled by republicans after that Coke commercial where the democrat is portrayed as bald and ugly and the republican is portrayed as the nice guy going for the compromise. And then they both Segway into the sunset? Brainwashing and prejudice embedded in a Coke commercial !?! WTF is going on in USA !? The 12-man-on-the-field penalty in the 3rd quarter is another legalese bullshit in the game. It is clear that the extra player had no effect on the game so it is just ridiculous for the judge to penalize the team for that. A couple small penalties were given for Patriots as well, judge bias was not as prevalent as last year. The camerawork stayed the same through the game - confusing, flashy and sloppy. Other cameras are in the shot line all the time. The game however did compensate for that all. The 4th quarter was very nice and exiting. I had no idea of the teams before the game, but I did like the Giants more - they just seemed to have more will and soul in their game. So I rooted for them since the 2nd quarter and it was very satisfying to see them win the game in the final seconds. Summary: good game, great final quarter, horrible coverage and crappy sideshows (adverts and halftime show).