In this post I will say some things about localization that I've never felt it was politic to say before. (It still isn't, but what the hey.) But I first want to mention that one of the reasons Christian Perrier is awesome is because he manages to steer projects in ways that avoid (most of) the problems below, and because he has managed to scale rather impressively far. But not far enough that these things haven't gotten under my skin. So, I hate localization. Hate, hate, hate. I wish it would die. I wish everyone spoke the same language. I hate that we have to be politically correct about it, to the point that even if everyone does speak English (hello, Denmark?), we still have to worry about localization into their native language. I especially hate that, after we waste time and resources doing that, everyone uses the software in English anyway (hello, most of Europe). I hate that the typical best effort localization of software is typically so incomplete that it's not actually usable by people who really do need the translation. Either because "best effort" == 80% translated, which is sometimes worse than 0%, or because the documentation doesn't get translated too. And that even with what we consider a complete translation, there are still all sorts of English bits that nobody would ever consider translating. Such as the names of commands, of command-line switches, of markup keywords and configuration files. I hate that translating software and documentation always involves bolting an enormous, nasty build system onto its side, and that the bulk of translations always greatly outweighs the size of the code. I hate that these systems constantly produce deltas that I have to avoid cluttering my commits with. I hate the expectation that every programmer somehow become an expert in internationalization, rather than letting them devote their time to something they're good at. But none of that is what really bothers me about localization. What really, really bothers me is that I often see the need to localize code inhibiting that code becoming better. Just two examples:

• Oh no, we worry, if we fix this a message, we will have to do 100 times more work to update the translations. Surely, then, anyone who wants to fix a bad message is an anglocentric who should be ignored.
• Rather than allow the computer to dynamically generate correct and expressive English on the fly, to actually talk to us, we have to reduce everything to gettextable static strings. The poster child for this disease is that it becomes easier to tack on "(s)" or reword entirely, than to deal with the cumulative complexity of plural forms in every language.

(PS, I also hate that we have to keep "l10n" and "i18n" straight, and that the words are so unwieldly in this f5g modern English that we represent them based on their length.)

This is a rant. A rant which goes to the grub maintainers, and one that could go nearly identically to many people in the KDE environment or many other open source projects. I really like grub. I really like grub 0.97 despite that it s been unmaintained for years and not booting on two of my important machines. I should like grub 2 because its configuration looks more straightforward and for its better features - direct booting of .iso images, from LVM and RAID. But actually, I have learned to hate grub 2 since it is not finished and badly documented, and that its existence is already being used as an excuse for grub 0 s development having stopped years ago (and it being renamed to grub-legacy to clearly show that it s the unloved child) - and things looks like this is not going to change any time soon. grub 0.97 is the de facto standard in booting Linux, its most prevalent advantages probably being its capability of reading filesystems and its command shell, which allows booting kernels and other software that one wasn t planning to boot when the system was shut down. There is a small fraction of people who still use lilo despite its disadvantages, for example on achitectures that are not supported by grub, if one wants to have the boot FS on LVM, if one just prefers lilo for non-technical reasons or on systems that grub doesn t work on. The last three systems, by the way, that I still run lilo on are Pyramid pizzaboxes with supermicro-made mainboards from 2002, which just refuse to boot if grub is used and the vendor just says that grub 0.97 is still beta and that they do not support grub on their systems: Use Lilo. Grub s last upstream release, 0.97, dates back to 2005, and upstream has stopped developing this grub release years ago. grub 2, the new way to go, had its first release in 2003, and the last stable release that people are supposed to use dates back to februar 2008. grub 2 is clearly a step in the right direction, but there are still bugs, and the most prevalent show stopper against grub 2 is the almost complete absence of any documentation. I have, for example, not yet found good documentation about the new config file format, which is a completely different deal from what we are used to with grub 0. The commands manual pages do little more than list the (obvious!) command line options without giving much more information about what the actually do, let alone usage examples. The documentation (which used to be really excellent for grub 0), has been moved to a wiki, which has some useful content but cannot keep up with the excellent documentation that grub 0 used to have. Additionally, grub2 does not seem to have a grub shell which can be called from within the host system, which makes installing grub 2 in exotic situations much harder than it used to be with grub 0. I am also quite disappointed that despite the grub upstream web pages claiming that grub 2 is under active development, the last release is more than fifteen months in the past while there are still so many shortcomings. The Debian maintainers of grub 2 are regularly packaging grub 2 subversion snapshots, which is a good idea in face of upstream s release policy which is quite different from what s being told in the wiki. I am, however, reluctant to rely on a development snapshot of a software to boot productive systems. Guys, you cannot declare a widely used, well-documented software legacy when the replacement is still in a nearly unuseable shape and the docs are hidden in the helpful files named .c and .h. This is simply unacceptable. It s a drama that the docs are still in such a pitiful shape fifteen months after your last release. But, grub is not the only free software project that so blatantly scores developer fun over the needs of the end user. I know that it is a boring task to fix bugs and to work around design shortcomings in the existing version of software, and that it is much more fun to hack away on new designs and new code, but dropping old versions before the new version is up to par with the old one is annoying the users and driving people away to other software packages. This applies to grub, a quite low-level program, but also to larger and more complex projects like KDE who has recently pulled the same stunt by dropping development and support for KDE 3.5 while KDE 4 is still missing many many features and KDE 3.5 s stablity, thus driving people (back) to GNOME or alternative desktop environments. This we give a damn about people who actually use our software, our compilers run without users as well attitude is a show of disrespect for the user base and makes me very sad as this does indeed show that free software is in no way superior to its commercial counterparts. Sadly, I do not have a solution for this issue since I know that free software developers are usually volunteers who do not have to report to anybody, and that they do not need to care for their user base. But I hope that people begin to reconsider their attitude and think about what kind of horrible damage is being done to the user base and to free software s reputation.

In Myanmar, Aung San Suu Kyi and 2 000 monks and activists are being detained, as the miltary junta in the country presses new, ridiculous charges against Suu Kyi a few days before the end of her house arrest. Suu Kyi, recipient of the Nobel Peace Prize and proponent of democracy, is leader of the opposition and the greatest threat to the junta s grasp on power. Given the inhumane conditions imposed upon the Burmese, and the ruthless slaughter of those who do not play by the military s rules, it is not hard to imagine that she would win any fair election, as soon as she would get a chance to run. However, if the junta are successful in pushing charges Suu Kyi is accused of breaching the house arrest after an American man sneaked into her house uninvitedly she could be locked up until after the elections in 2010, effectively disarming the threat she poses to the military regime. She is threatened to be kept in the notorious Insein Prison without medical care. As she is suffering from serious illness, this puts her life at risk. I do not usually campaign for political topics, but I ve been to Myanmar and this issue touches me deeply. The people in Myanmar need our support. Even though the 2007 Burmese anti-government protests seriously weakened the military and brought hope to the lives of millions that have been oppressed for years, the events quickly vanished from the media and allowed the junta to regain control. We must not let that happen again. It is of utmost importance to show our support and not look away. One way of doing so is by increasing visibility. If you care, please write to your newspapers, publish on your blog, and do everything else you can to raise awareness. Another way to help the Burmese people is through signing the avaaz.org petition to UN Secretary General Ban Ki-moon. Best to do all of the above! Unfortunately, avaaz.org force-subscribes everyone who signs a petition to their newsletter, which may well not be what you want. However, please do not let it stop you from signing the petition. You can later unsubscribe from the (infrequent) newsletter. I have informed the staff members of this concern. If you would like to voice your opinion too, write to info t avaaz dot org and encourage them to make the newsletter subscription optional. But make sure to also sign the petition, please.

(The much-delayed part 2 in an ongoing series: shiny things in telepathy-glib) Over a year ago I explained my first major round of work on telepathy-glib, with which it could be used to implement D-Bus services. I started to write this post shortly after, but then got distracted by a bee or something; recent discussion about having a D-Bus binding in GLib has finally prompted me to finish it. The second major round of work on telepathy-glib dealt with implementing client code, and went sufficiently well that telepathy-qt4 (currently under development) uses essentially the same model. Again, while telepathy-glib is intended for Telepathy implementors, I think TpProxy's ideas are generically useful, particularly for "large" D-Bus APIs.

Some background: dbus-glib and libtelepathy

The life-cycle of a DBusGProxy (PD clipart credit: karderio and Andy)

The API provided by dbus-glib for client code is, er, rather less than ideal. The basic object is DBusGProxy, which has a few problems. There's one per interface, rather than one per object; this leads to a maze of twisty GObjects, all suspiciously similar. Our old client library, libtelepathy, got round this by subclassing DBusGProxy to make helper objects called TpChan, TpConn etc., which were simultaneously the proxy for the "main" interface (Channel, Connection etc.), and a factory for proxies for the other interfaces (like Channel.Type.Text and Channel.Interface.Password). This still wasn't ideal, but it was a start. If you use a DBusGProxy in any way after it has emitted the destroy signal, this is considered to be invalid use of the API, and dbus-glib asserts all over the place. We'd rather it didn't do that - remote objects becoming invalid is a fact of life, and if you don't immediately discard all your references to those objects, remote operations on them should just give you a runtime error. You have to be prepared to handle such runtime errors already, because any IPC call can fail at any time. The main API for calling methods uses varargs and is rather subtle. Notably, the calling convention for variant parameters (of D-Bus signature 'v') is not consistent with the calling convention for any other container. For any other container, for instance a GPtrArray *, you pass in a GPtrArray ** pointing to a GPtrArray * variable containing NULL; on success, a pointer to a new GPtrArray is written into that variable. For variant parameters, though, you must pass in a GValue * pointing to a zero-filled GValue, and the result will be written into that GValue. dbus-binding-tool generates thin wrappers around this method-calling API which provide type-safety, although they can't directly specify a non-default timeout. As for signals, before binding to a signal, you have to tell dbus-glib about its signature. This tells dbus-glib how it should push the signal arguments onto the C stack (in GObject terminology: how it should marshal them), to match the signature of the signal-handler function. The documentation claims that this isn't necessary for objects supporting the Introspect method, but that's untrue - on the other hand, if the documented behaviour actually happened, I would consider this to be a critical bug in dbus-glib, since it would enable remote objects to crash dbus-glib clients (by causing signal arguments to be pushed onto the stack in a way that does not match the signature of the signal handler). Of course, the signal-adding function takes varargs, and has to be called exactly once per signal per object. If you don't, dbus-glib asserts when you connect to the signal; if you call it twice, dbus-glib asserts anyway.

TpProxy TpProxy is a GObject subclass vaguely inspired by Python's dbus.proxy.ProxyObject. It isn't a subclass of DBusGProxy, which lets us encapsulate dbus-glib almost completely - in fact, all the references to DBusGProxy are in a separate header, proxy-subclass.h, only used by subclasses of TpProxy (as opposed to normal library API users). TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. As explained below, I believe that this is the most natural object mapping for D-Bus interfaces; in practice it tends to lead to clearer code for us, since one remote object often maps nicely to one UI element (of course, if your objects only have one interface, this approach is no better or worse than dbus-glib's). TpProxy provides support for the following common D-Bus things:

• Calling methods
• Connecting to signals
• Objects becoming invalid
• Objects with optional interfaces
• Extensibility

It doesn't currently provide any particular help with Properties - the Properties interface is just another interface as far as we're concerned. We find that, in practice, that's not a problem (even though we use D-Bus properties extensively). In telepathy-qt4 we basically reinvented TpProxy in C++, as Tp::DBusProxy. I've put in some comments below illustrating how the general ideas we used in TpProxy translate into a different object mapping.

Calling methods Method calls on TpProxy are done through auto-generated wrapper functions, much like DBusGProxy. These look something like this:

typedef void (*tp_cli_channel_callback_for_close) (TpChannel *proxy,
    /* any 'out' arguments would go here - Close() doesn't have any */
    const GError *error, gpointer user_data,
    GObject *weak_object);
TpProxyPendingCall *tp_cli_channel_call_close (TpChannel *proxy,
    gint timeout_ms,
    /* any 'in' arguments would go here - Close() doesn't have any */
    tp_cli_channel_callback_for_close callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object);

Methods for some interfaces, like DBus.Properties, are available on TpProxy itself; others, like Telepathy's Channel.anything, are only available on a particular subclass, like TpChannel. Things to note here:

• It's an asynchronous call with a callback. General design principle: "this is IPC, get over it". We don't try to hide the fact that IPC is taking place by doing pseudo-blocking calls.
• The timeout is explicit (although you can always pass -1 to let libdbus pick a "reasonable" default for you). This is because the remote service might not respond, and API users should be at least vaguely aware of this: "this is IPC, get over it".
• The callback isn't a GCallback; it has an explicit signature, for some semblance of type-safety (it's not much, but it's better than nothing).
• The callback takes a GError, because any IPC call can fail for any reason.
• A TpProxyPendingCall * is returned. This is a pointer to an object that only exists until just after the callback is called (so you can ignore it if you don't want it), with a method you can call to cancel the method call (with hindsight this is not ideal, and if I was writing this API now, I'd use gio's GCancellable). "Cancel" is perhaps slightly misleading: the method call still takes place (it was already in libdbus's outgoing queue) but the result gets ignored and the callback isn't called. This is mostly so you an object can forget all the calls it was busy making at the time that the results become irrelevant (e.g. the object is destroyed).
• There is an extra weak_object argument, which is weakly referenced and is passed to the callback: in practice, most users of telepathy-glib use this as well as or instead of user_data. If the weak_object runs out of references, the callback is automatically cancelled, which means that in practice, explicit cancellation is almost never needed.
• The generated code has as its first argument a suitable subclass of TpProxy, in this case TpChannel (we tell the code generator about subclasses).
• You can't see it in the API, but the TpProxy gains an extra ref while a method call is in progress, so you never have to worry about what happens if the proxy gets unreffed during a method call.
• There's an explicit destructor for the user_data, which is called even if you "cancel" the method call.

In telepathy-qt4, these methods are on small generated helper classes similar to dbus-python's dbus.Interface, one per interface (we provide accessors for them on the non-generated DBusProxy subclass). Instead of using callbacks, we return a temporary QObject per call, which emits a Qt signal on success or failure (that's how all async calls in QtDBus work).

Connecting to signals Similarly, signals have some generated functions:

typedef void (*tp_cli_channel_signal_callback_closed) (TpChannel *proxy,
    /* any arguments would go here - Closed doesn't have any */
    gpointer user_data, GObject *weak_object);
TpProxySignalConnection *tp_cli_channel_connect_to_closed (TpChannel *proxy,
    tp_cli_channel_signal_callback_closed callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object,
    GError **error);

API notes for these:

• The callback doesn't take a GError any more, because signals can't fail; the weak_object and the user_data are the same as for method calls
• This isn't actually a GObject signal at all: those aren't particularly typesafe, and lack namespacing. This might not be very binding-friendly, we haven't tried doing that yet... it might be worth introducing a signal with details, like in dbus-glib, for which these functions are "C bindings".
• The returned TpProxySignalConnection lets you disconnect from the signal, just like the TpProxyPendingCall above; it can safely be ignored
• Similarly, if the weak_object dies, then the signal automatically disconnects
• It is possible for connecting to a signal to fail: this happens if the proxy doesn't actually have the requested interface. If this happens, it's graceful, not a crash. In practice, the flow of code is almost always such that error can safely be NULL because the interfaces are already known.

In telepathy-qt4 the per-interface generated helper classes emit Qt signals. In many cases, we end up proxying these signals by responding to them in the hand-written Tp::DBusProxy subclass by emitting a different signal, in order to give them a nicer representation.

Becoming invalid This is IPC, and any error can happen to you at any time. We found that it was useful to have a general concept of "this object is no longer useful", so we introduced the concept of an object becoming invalidated. The invalidation reason is stored on the object as a GError. If this GError is NULL (as it is initially), then the object is still expected to be valid; if it's non-NULL, the remote object has vanished and the local proxy no longer works. There are several ways a TpProxy can become invalid:

• its bus name falls off the bus (the process exited or crashed)
• application-specific reasons (usually triggered by a D-Bus signal in a subclass, like the Telepathy Channel's Closed signal)
• the GObject gets disposed

Method calls on an invalidated object always fail, with the invalidation reason as the error. This means that calling a method on a TpProxy should never crash your process. The method's callback has to be prepared to handle an error in any case, because this is Sparta^WIPC, so there's no loss in giving it another error. Connecting to signals on an invalidated object always fails, and any existing signal connections are disconnected when it becomes invalidated. This means that if a new remote object appears, and it happens to have the same object path, bus name etc. as the old one, you won't get its signals. A TpProxy can either be bound to a unique bus name or a well-known bus name. In the equivalent code in telepathy-qt4, we call this "stateful" or "stateless" - these names don't capture the intention perfectly, but they're the best we could come up with. A proxy for a stateful API like Telepathy's Channel should always bind to a unique name - when the unique name falls off the bus, the TpProxy becomes invalidated automatically, representing the fact that the Channel you had no longer exists, and nothing can be a perfect replacement for it (the VoIP call has already been terminated, you've already left the chatroom, or whatever - if you make another Channel, that's e.g. a distinct VoIP call). A proxy for a stateless API like Telepathy's ConnectionManager, where an exiting process can disappear, be service-activated again, and still have exactly the same API, should bind to the well-known name. Proxies for well-known names aren't invalidated when the process exits. Qt doesn't have a natural equivalent for GError, so the invalidation reasons in telepathy-qt4 are just a pair of strings, the namespaced name and the message - basically a libdbus DBusError. The principle is the same, though.

Optional interfaces As mentioned above, TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. I believe that this is the most natural object mapping for the D-Bus object model, because the D-Bus interfaces on a remote object can behave like any of these:

• classes and subclasses (Telepathy.Channel.Type.Text is a Telepathy.Channel)
• shared interfaces (some Telepathy.Channel objects, of many Types, implement Telepathy.Channel.Interface.Group; any D-Bus object can implement DBus.Properties)
• optional features (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.ChatStates, some do not)
• discoverable extensions (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.Messages, which is more or less Text 2.0)

The way in which interfaces are discovered is also variable. The "classic" D-Bus way to discover interfaces would be to call Introspect. Telepathy services still support that style of introspection, but we don't use it for anything beyond d-feet:

• Introspect returns a blob of XML, which you have to parse into something useful; if it was in a D-Bus data structure, you'd already have some sensible data structure.
• Finding out more information about the supported interfaces than their names isn't very useful: in a static language like C you already need to know the methods, signals, properties and their signatures in order to write and compile your code, and in a dynamic language like Python consumers of Introspect end up being remotely crashable by services with unexpected method signatures (this is one of dbus-python's big mistakes in my opinion).
• If a method that you want to call turns out to be missing, the worst that can happen (in a competently written service) is that it fails with a D-Bus error - and you have to handle D-Bus errors anyway, because this is IPC and anything could fail.
• Static bindings like dbus-glib have poor support for removing interfaces for which C code exists at runtime, whereas Telepathy needs features like an IM connection that might or might not support avatars (we don't get to find out until we've successfully connected to the server).

Instead, our older interfaces have a method on the "base class" (Channel, Connection etc.) called GetInterfaces, which just returns an array of strings. Newer interfaces (like Account), and older interfaces that have been ported (like Channel), have a property called Interfaces which is, again, an array of strings - this lets us combine the download of the interfaces list with downloading other basic information in a GetAll call. To cater for all these ways to use interfaces, the TpProxy base class has very basic support for interface discovery - you can ask it whether it supports an interface, and you can tell it that it does, in fact, support an interface. Asking which interfaces are supported is directly useful for library users; it's also used as a check by the generated method-call and signal-connection stubs, which return a canned error (Telepathy.Error.NotImplemented in Telepathy's case) if the object isn't known to have the interface. Telling TpProxy that it does support an interface is intended for use by subclasses, and might have been protected if we were writing C++: what happens in practice is that a concrete subclass like TpChannel knows how to discover the fully supported interfaces for this particular object, does so, and calls methods on the TpProxy to tell it which interfaces can work.

Extensibility The API stubs used by TpProxy to call methods and bind to signals are generated from XML documents containing an augmented form of D-Bus introspection data. These XML documents contain various Telepathy-specific extensions, but none of the extensions are language-specific - telepathy-glib, telepathy-python and telepathy-qt4 all operate from exactly the same XML specification. telepathy-qt4's different object-mapping did require us to tighten up some rules that were previously only conventions, but even so, the format is the same. I think this is vitally important - it just doesn't scale to have one set of language-specific annotations in a D-Bus API for each language that will have bindings. One thing that we make heavy use of is what I call "Ugly_Case", which is camel-case with underscores at word boundaries. This gives us a simple, unambiguous and foolproof rule to use in code generators whenever we generate any of the popular conventions for identifiers:

• CamelCase: delete underscores
• javaCamelCase: force everything before the first underscore to lower case, then delete underscores
• lower_case: force everything to lower case, retain underscores
• UPPER_CASE: force everything to upper case, retain underscores

dbus-glib, by contrast, applies a complex and subtly buggy algorithm to the CamelCase D-Bus names, which results in our SendDTMF method being mapped into GLib as send_dt_mf, and requires that telepathy-glib's Python reimplementation of dbus-binding-tool uses a bug-for-bug compatible implementation (we ended up reimplementing dbus-binding-tool in order to generate the TpProxy-based bindings). Our code generation tools are still rather ad-hoc, because so is our spec format, but for draft interfaces it's possible to copy them into individual projects and use them to generate additional methods for TpProxy or any subclass. This is how we implement unfinished APIs like geolocation in Empathy, for instance. The Interfaces or GetInterfaces hook described above is entirely usable for these extension interfaces, and in fact that's how they're set up. In telepathy-qt4, the generated method stubs are genuine C++ methods, so we can't just append methods to an existing class: this is why we have one helper class per interface, so individual clients can generate a helper class for their particular version of an unfinished API, and instantiate an object of this class attached to a particular Tp::DBusProxy. When an interface becomes stable, it can be added to the set of interfaces for which code is generated in telepathy-glib or telepathy-qt4, at which point any client or service that was already using the final draft of that interface can easily be ported to the library version using sed (the API remains the same).

Becoming ready The concept of being "ready" does not directly exist in TpProxy, but is implemented in TpChannel, TpConnection and TpConnectionManager. The idea is that a freshly created proxy object isn't really fully usable yet - you have to connect to basic signals and recover the initial state of the remote object, as well as checking which extension interfaces are supported. In many well-designed D-Bus APIs you can do this initial setup with a few signal connections and a DBus.Properties.GetAll call. After doing that initial setup (which has to be done asynchronously, because it's IPC), the TpProxy subclass has a local cache of the remote object's state (not necessarily in the same format as the representation on the bus), which can be accessed synchronously at any time, and will be updated in response to change-notification signals. TpChannel and subclasses like it have explicit support for checking for readiness, and having a callback called when the object is ready or invalidated (whichever happens first). The "ready" property also supports the GObject notify signal. telepathy-qt4 took this concept and ran with it - there is library support for objects that can become ready, including a mixin. If I had as much time to work on telepathy-glib as I wish I had, this would be one of the first telepathy-qt4 features to be added to telepathy-glib.

Features becoming ready Going beyond that, in an extensible framework like Telepathy it's probable that not every client understands (or wants to understand) every feature of every object, so it's highly inefficient for every proxy object to subscribe to all the change notification signals and cache all the remote state on the off-chance that the library user wants them. In the API for TpContact (which is not actually a TpProxy for various reasons) we introduced the concept of optional features; telepathy-qt4 extended this idea throughout the library. The idea is that each library user knows which of the optional features are "interesting" to it, and makes a single asynchronous call (which expands into a series of D-Bus calls inside the library) to download the state for core functionality plus all of the selected features, and subscribe to change notification for all of those. This is another telepathy-qt4 thing that telepathy-glib still needs to catch up with; until someone works out how to make the clone() syscall apply to programmers, I fear all libraries are doomed to lag behind how their designers want them to look :-) (license: CC-BY-3.0 / LGPL-2.1+)

---

Most people (at least those who hear me shouting on irc) probably know that I'm not a fan of cdbs. While working on geoclue, which builds a lot of binary packages I realized, that cdbs wastes an insane amount of time by calling dh_* for every single package. One of the reasons geoclue uses cdbs is that the maintainer likes the short debian/rules file. We'll, compare yourself: cdbs:

#!/usr/bin/make -f
# -*- makefile -*-
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
include /usr/share/cdbs/1/rules/patchsys-quilt.mk
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/autotools.mk
include /usr/share/cdbs/1/rules/utils.mk
common-binary-predeb-arch:: list-missing
clean::
    rm -f providers/gypsy/Makefile
    rm -rf debian/tmp

Build time: dpkg-buildpackage -B 132.20s user 41.52s system 90% cpu 3:11.13 total dh(1):

#!/usr/bin/make -f
# -*- makefile -*-
include /usr/share/quilt/quilt.make
build: $(QUILT_STAMPFN) build-stamp
build-stamp:
    dh build
    touch $@
%:
    dh $@
clean: clean-patched unpatch
clean-patched:
    dh clean
    rm -f providers/gypsy/Makefile
    rm -f build-stamp
.PHONY: patch unpatch clean clean-patched build 

Build time: dpkg-buildpackage -B 85.63s user 32.91s system 91% cpu 2:08.91 total So cdbs wasted about one minute of time using a 2.4 GHz Intel Core2Duo. I'm wondering how much more time it takes on slow architectures like mips(el) and arm(el). One more argument against the unmaintaible mess of makefiles called cdbs.

Going for a run--or a race--almost always meant grabbing the GPS and often also meant setting a target pace and distance. For the last four and a half years, this was measured by a Garmin Forerunner 201. That's the large rectangular model in the original "brick" form factor. I've come to love the device. Training is great because the distance log, as well as the pacing, keeps you honest. Racing is probably better because it helps a lot with the pacing (but then I also had PRs in race where I had forgotten the GPS at home). But there is a downside. This model sometimes takes forever to find tracking, and has interference / weak signal problems when it is cloudy or moist. But worst of all, it just bonks out in the 'urban jungle' downtown where it drops signals too easily. And now after all these years the display had some minor damage, the strap didn't really hold anymore, and it was generally time for something new. On Tuesday it even lost a quarter mile when we were going for a short but really fast three miler. But then a few days prior I had followed fellow Debian marathoner Christian and used my birthday at the end of this month (as well as the upcoming Boston Marathon) as an excuse for conspicuous comsumption. After some price comparison, I ordered a factory reconditioned Garmin Forerunner 405 from this web discounter at a nice rebate to the regular price. It arrived this afternoon, seemingly shining new and I have been fiddling with it for the last little while. This device features wireless data transfer to a usbstick. This meant booting the laptop in windoze for the first time in years to load the 'client software' after which data transfer proceeded. The Garmin Connect site has very slick presentation and aggregation of the data. The trouble is of course how to get the data there when running Linux... Christian had mentioned the garmin-forerunner-tools package. Unfortunately, this seems to really be written for the Forerunner 305 models as it doesn't see the device at all. Some more googling lead to this page and the gant tarball. All still fairly raw, but with some prodding in the settings of the 405 ('pairing' set to 'on', 'force send' set to 'yes'; which may have to be reset each time ?) I got my two xml files off the gps watch. Yay. We'll see what mode I will settle one. With the 201 and its ancient serial port, I basically just dropped the run and training histories which their fairly limited data collections. Last but not least, fellow Oak Park runner Peter Sagal had a humorous Runner's World column on the whole GPS geekyness. If he'd only known how to pair it with programming geekyness...

It s now been almost a month since I got my Kindle 2 and wrote about my first impressions of it. In that amount of time, I figure I have read, on the Kindle, the equivalent of 1500 to 2000 printed pages. I ll start with my personal reactions to the device, and then move on into some less subjective observations about it. What I ve Used It For To give you a quick idea of what I m reading:

• David Copperfield (downloaded at feedbooks.com)
• Part of Houston Smith s Why Religion Matters, purchased at the Amazon Kindle Store (currently reading it)
• Cory Doctorow s Eastern Standard Tribe (from feedbooks)
• Most of Simon Travaglia s funny BOFH series through 2000 (built myself)
• Various other bits and pieces of things here and there.

The Gut Reaction I want everything to be available on the Kindle, and I want to read everything on it. Whether that s just some excitement at a new device (still possible after a month, I guess), or something more lasting, I m still a little unclear about but I suspect it s something more lasting. I have read far, far more since I have had the Kindle than I had for quite some time prior, excepting a certain Hitchhiker s Guide to the Galaxy marathon one Christmas vacation. This leads me to think that the Kindle s main competitor isn t the Sony PRS-505, but rather the XBox, Wii, Netflix, and Hulu. Not that I watched that much TV to start with, but I haven t even been keeping up with the Daily Show since I got the Kindle. Not only that, but it prompted me to go to the local used bookstore (all proceeds go to charity) and buy a paper copy of David Copperfield for $10, so Terah could read it with me while I reread it on the Kindle. So it seems that Amazon s stats which show that people tend to read more once they get the Kindle, and don t cut down on paper book buying are about right in my case. Plus there s the fact that I ve loaded up the Kindle with over a hundred free books of all sorts of descriptions, some from Amazon s own free book catalog (extensive in itself), and others from Feedbooks, Gutenberg, or other places. That s taken up some time itself, and I now have achieved what I tend to achieve with all devices like this (iPod, MythTV, etc): more material than I can ever possibly get through. Why I Like It So, what exactly IS it about this thing that I like? Part of it has to be the size of the device. Granted, I wish the screen were larger, but even with the leather case, the device is a pretty nice size. I find that I keep two or three books that I m reading at the top of my list, and it s great to be able to carry around a single device and be able to select from a book as the mood strikes. Part of it is the built-in dictionary and search tools, which I ve already discussed. In the end, Amazon s execution of the concept is just done really, really well. It doesn t require some sort of proprietary Windows app like Sony s reader does. It doesn t even require a PC at all. It acts as reliable as a book, and that s something. The highlight/clippings feature is also pretty nice. You can highlight a section of text, which then causes it to be rendered with an underline on-screen just like you can underline something in a book. Better yet, when you re reading the book, you can call up a list of all such passages (as well as your bookmarks, etc), see them with an excerpt of text, and jump directly to them. No more need for millions of tiny pieces of paper marking a page that contains a marking. Even better than that, the Kindle creates a .txt (yes, plain text) file in the documents/ directory for you. Each time you bookmark or highlight a passage, it adds a little summary of the passage to that file, as well as information about the location of it. For highlights, the summary is the content of the highlight itself. So if you re going to go blog about the book later, there are a bunch of ready-made quotes you can cut-and-paste later. You can read that file both on the Kindle and on a PC. Sweet. So, I guess I would say that the Kindle makes it easier to read complex texts, and fun to read lighter ones. Availability of Content There are several ways you can get content onto the Kindle. The first, of course, is Amazon s own Kindle Store. Quite the clever bit of engineering there. You hit buy , and generally by the time you can walk over to the Kindle and pick it up, the book s on it, thanks to the integrated cell-network wireless modem. Subscriptions to newspapers and magazines work similarly each issue is automatically delivered in the middle of the night. However, it seems that the magazines such as New Yorker and Atlantic aren t exactly complete replicas of the print editions for some reason. The pricing of Kindle content is generally cheaper than paperback editions of books, but not by much. I think it should be more heavily discounted than it is, and part of that may be due to the large commission Amazon takes on things. Ironically, Amazon also has a tremendous array of free public domain and copyrighted books, and an equally staggering number of books available for under $1. For the classics, and other things out of copyright, my favorite place to turn is Feedbooks. They ve got ebooks available for download for just about any format, including Kindle, and generally do a good job of providing a real linked Table of Contents and the like. Kindle can read DRM-free content in .txt or .mobi format. I ve also built my own ebooks for the Kindle using Mobiperl. It takes an ePub source directory and makes a MobiPocket file out of it, which can be installed on the Kindle directly. For things I find on Project Gutenberg, GutenMark is a nice tool to create a good-looking HTML representation. I also use quote-fixer to help with my other pet peeve: straight quotes and lack of em-dashes on a device that can, and should, display both. Screen I m still of two minds about the screen. On the one hand, it doesn t give me eyestrain like reading on a PC, laptop, or cellphone screen does. Also, it looks remarkably book-like. A well-prepared Kindle etext feels so professional on the screen, so real, so lovingly-prepared, that it s fun to read. The Feedbooks David Copperfield is an excellent example of this, as is Houston Smith s Why Religion Matters. There are several things I don t like about the screen. One is that, while reading a book, it leaves a margin around the edge of the text. I can t figure out why it doesn t use all the available screen real-estate, especially since its built-in web browser does. Another is that, while not terribly reflective, it is still somewhat reflective, and glare from the sun or a light at a bad angle can make the screen unreadable unless you turn it a bit to a different angle. But my biggest gripe about the screen is that it s not as high-contrast as I d like. Still, I very much enjoy the Kindle, so I guess it s not as huge a problem as I think it could be sometimes. Pet Peeve I get really annoyed when I download content for the Kindle and discover that it doesn t have a linked Table of Contents (meaning I can click on a chapter and go directly to it). I m even more annoyed when it uses straight typewriter quotes instead of the angled typesetter s quotes or smart quotes . The Kindle is capable of beautifully rendering them, as well as things such as em dashes, and when I see what is obviously some cheap attempt at a conversion of some ancient ASCII document, rather than a proper attempt to make a good-looking book, I get annoyed. And so it was that I requested, and got, a $0.80 refund for my purchase of Sherlock Holmes: The Complete Collection and instead paid $3.60 for The Complete Sherlock Holmes. Yes, I could have spent a few hours converting all these from Gutenberg, but $3.60 is so cheap that it wasn t worth it. Customer Service Amazingly, Amazon s customer service for the Kindle is not their usual Asian form letter service. It s Americans, who seem to read, understand, and care about problems. I was pleasantly surprised. Too bad the rest of Amazon doesn t learn a thing or two from it. Conclusion I think the Kindle is worth it. It will pay for itself exceptionally quickly for people like me that don t live anywhere near a large library, and enjoy reading out-of-copyright material. Even if it doesn t pay for itself that quickly, the convenience of being able to carry several actively reading books with you on a single small device is pretty nice. No more pile of books in my suitcase for travel.

Luke Peterson from UC Davis came to visit me in Reno and we spent the last weekend hacking on the Python Dynamics package that uses SymPy to calculate equations of motion for basically any rigid body system.

On Friday we did some preliminary work, mostly on the paper, Luke showed me his rolling torus demo that he did with the proprietary autolev package. We set ourselves a goal to get this implemented in SymPy by the time Luke leaves and then we went to the Atlantis casino together with my boss Pavel and other guys from the Desert Research Institute and I had my favourite meal here, a big burger, fries and a beer.

On Saturday we started to code and had couple lines of the autolev torus script working. Then we went on the bike ride from Reno to California. I took some pictures with Luke's iphone:


Those mountains are in California and we went roughly to the snow line level and back:

This is Nevada side:


That was fun. Then we worked hard and by the evening we had a dot product and a cross product working, so we went to an Irish pub to have couple beers and I had my burger as usual.

On Sunday we spent the whole day and evening coding and we got the equations of motion working. On Monday we worked very hard again:



and fixed some remaining nasty bugs. I taught Luke to use git, so our code is at: http://github.com/hazelnusse/pydy, for the time being we call it pydy and after we polish everything, we'll probably put it into sympy/physics/pydy.py. If you run rollingtorus.py, you get this plot of the trajectory of the torus in a plane:

It's basically if you throw a coin on the table, e.g. this model takes into account moments of inertia, yaw (heading), lean, spin and the x-y motion in the plane. Depending on the initial conditions, you can get many different trajectories, e.g for example:

or:


This is very exciting, as the code is very short, and most of the things that Luke needs are needed for all the other applications of sympy, e.g. a good printing of equations and vectors (both in the terminal and in latex), C code generation, fast handling of expressions, nice ipython terminal for experimentation, plotting, etc.

Together with the atomic physics package that we started to develop with Brian sympy will soon be able to cover some basic areas of physics. Other areas are general relativity (there is some preliminary code in examples/advanced/relativity.py) and quantum field theory and Feynman diagrams - for that we need someone enthusiastic that needs this for his/her research --- if you are interested, drop me an email, you can come to Reno (or work remotely) and we can get it done.

My vision is that sympy should be able to handle all areas of physics, e.g. it needs good assumptions (if you want to help out, please help us test Fabian's patches here), then faster core, we have a pretty good optional Cython core here, so we'll be merging it after the new assumptions are in place. Then sympy should have basic modules for most areas in physics so that one can get started really quickly. From our experience so far in sympy/physics, those modules will not be big, as most of the functionality is not module specific.

all programs that interact with e-mail are broken in one way or another. Please be careful. - Lars Wirzenius

I seem to have a cunning knack of finding problems with configuring server software, particularly involving e-mail, where a) I can t find answers in Google, b) most people I go and ask for help say they d usually ask me such things, and c) if I go onto IRC or mailing lists I end up helping other people and not getting any help with my problem. It s quite likely this is just because I m something of a perfectionist, so the ridiculous crappy hacks people come up with and seem content to entrust their mail to are unacceptable to me for one reason or another. Anyway, in my ongoing quest for the perfect mail system, I ve painted myself into a corner again. (I m currently running with postfix, postgrey, clamav-milter, dspam, dovecot using LDA, managesieve and the cmusieve & antispam plugins. If I can get the current incarnation working, I ve had enough requests to write up a full HOWTO, and seen enough around with pretty questionable content, that I ll probably do it before too long.) I ve got postfix s local transport configured to hand mail to dspam over LMTP, using mailbox_transport = lmtp:unix:/dspam/dspam.sock. dspam is configured to listen here, add X-DSPAM-Result and signature headers, and then deliver the mail with dovecot s deliver LDA (which I ve set to 4750 root:dspam). From dspam.conf: ServerParameters "--deliver=innocent,spam -d %u"
ServerDomainSocketPath "/var/spool/postfix/dspam/dspam.sock"
...
Preference "spamAction=tag"
Preference "signatureLocation=headers"
...
TrustedDeliveryAgent "/usr/lib/dovecot/deliver" My dovecot configuration is pretty standard, using PAM for both passdb and userdb, and provides the auth-master socket that deliver needs. The problem I have is that postfix s local transport is qualifying the local username with the FQDN of the machine before delivering it to dspam with LMTP (the local mail transfer protocol), even for locally-originated mail which was only addressed with a bare username! dspam doesn t mangle it or care if the user is local or not, and then cheerfully invokes deliver -d robot101@omega.example.co.uk, which returns EX_NOUSER (addressee unknown) because my username is just robot101. From mail robot101: Mar 6 02:25:40 omega postfix/pickup[13607]: DAA4942F41F: uid=1000 from=<robot101>
Mar 6 02:25:40 omega postfix/cleanup[13637]: DAA4942F41F: message-id=<20090306022540.DAA4942F41F@omega.example.co.uk>
Mar 6 02:25:40 omega postfix/qmgr[12552]: DAA4942F41F: from=<robot101@omega.example.co.uk>, size=339, nrcpt=1 (queue active)
Mar 6 02:25:40 omega dovecot: auth(default): passwd(robot101@omega.example.co.uk): unknown user
Mar 6 02:25:40 omega dspam[13527]: Delivery agent returned exit code 67: /usr/lib/dovecot/deliver -d robot101@omega.example.co.uk
Mar 6 02:25:40 omega postfix/lmtp[13640]: DAA4942F41F: to=<robot101@omega.example.co.uk>, orig_to=<robot101>, relay=omega.example.co.uk[/dspam/dspam.sock], delay=0.08, delays=0.05/0.01/0.01/0.03, dsn=4.3.0, status=deferred (host omega.example.co.uk[/dspam/dspam.sock] said: 421 4.3.0 <robot101@omega.example.co.uk> Delivered (in reply to end of DATA command)) So, no e-mail for me. Dearest lazyweb, which of the three components is behaving wrongly, and how can I fix it? (And no, I m not just going to switch to GMail. I store my data on hard drives, which are sometimes in servers, not in the cloud . Until about a month ago, most people I knew spoke about clouds which were made of particles of water in the sky, rather than as a data storage media. What if it rains? ;) Update: The problem is fixed! Even though arguably the problem is dspam s for not knowing which users are local or not, it s fixable in dovecot 1.1.x using the auth_username_format = %n option. Thanks so much to Angel Marin for helping me out. Update 2: There s also a patch for dspam floating around which adds a StripRcptDomain option, which makes the LMTP server truncate the e-mail address at the @, so essentially assumes everyone to be a local user. The problem with both of these fixes is that they re both blunt instruments which will break virtual users on the same host. I think the real fix would be something more like a LocalDomains option in dspam, to choose which domains should be considered local and truncated from the e-mail addresses for delivery purposes.

New day, new luck... and next try... ;-)

Today I received two emails with even newer BIOS updates for my problem. The second, latest version 1.80p did actually fix the problem of not booting with 4 GB of RAM. I haven't tried the first, previous version 1.80k yet, so I can't tell if that worked as well.

Anyway, despite the fact that the machine now recognizes 4 GB dual-ranked memory, Linux uses 3.1 GB of it - at least when not using Xen... But first without Xen, standard Debian 2.6.26-1-686 kernel:

[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 2.6.26-1-686 (Debian 2.6.26-13) (waldi@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-24)) #1 SMP Sat Jan 10 18:29:31 UTC 2009
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[ 0.000000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
[ 0.000000] BIOS-e820: 0000000000100000 - 00000000bff30000 (usable)
[ 0.000000] BIOS-e820: 00000000bff30000 - 00000000bff40000 (ACPI data)
[ 0.000000] BIOS-e820: 00000000bff40000 - 00000000bfff0000 (ACPI NVS)
[ 0.000000] BIOS-e820: 00000000bfff0000 - 00000000c0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
[ 0.000000] BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
[ 0.000000] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 2048MB of RAM.
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] WARNING: at arch/x86/kernel/cpu/mtrr/main.c:706 mtrr_trim_uncached_memory+0x123/0x183()
[ 0.000000] Modules linked in:
[ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.26-1-686 #1
[ 0.000000] [] warn_on_slowpath+0x40/0x66
[ 0.000000] [] _spin_lock_irqsave+0x16/0x2f
[ 0.000000] [] _spin_unlock_irqrestore+0xd/0x10
[ 0.000000] [] release_console_sem+0x173/0x18c
[ 0.000000] [] vprintk+0x2d2/0x2de
[ 0.000000] [] generic_get_mtrr+0x2e/0x11e
[ 0.000000] [] printk+0x14/0x18
[ 0.000000] [] mtrr_trim_uncached_memory+0x123/0x183
[ 0.000000] [] mtrr_bp_init+0x20c/0x214
[ 0.000000] [] setup_arch+0x254/0x6bb
[ 0.000000] [] printk+0x14/0x18
[ 0.000000] [] start_kernel+0x62/0x2d7
[ 0.000000] =======================
[ 0.000000] ---[ end trace 4eaa2a86a8e2da22 ]---
[ 0.000000] update e820 for mtrr
[ 0.000000] modified physical RAM map:
[ 0.000000] modified: 0000000000000000 - 000000000009fc00 (usable)
[ 0.000000] modified: 000000000009fc00 - 00000000000a0000 (reserved)
[ 0.000000] modified: 00000000000e4000 - 0000000000100000 (reserved)
[ 0.000000] modified: 0000000000100000 - 00000000bff30000 (usable)
[ 0.000000] modified: 00000000bff30000 - 00000000bff40000 (ACPI data)
[ 0.000000] modified: 00000000bff40000 - 00000000bfff0000 (ACPI NVS)
[ 0.000000] modified: 00000000bfff0000 - 00000000c0000000 (reserved)
[ 0.000000] modified: 00000000fffc0000 - 0000000140000000 (reserved)
[ 0.000000] 2175MB HIGHMEM available.
[ 0.000000] 896MB LOWMEM available.
[ 0.000000] Entering add_active_range(0, 0, 786224) 0 entries of 256 used

Now, when I try to boot Xen hypervisor & kernel, Linux recognizes 4 GB of memory:

[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 2.6.26-1-xen-686 (Debian 2.6.26-13) (waldi@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-24)) #1 SMP Sat Jan 10 22:52:47 UTC 2009
[ 0.000000] Reserving virtual address space above 0xf5800000
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000f5730000 (usable)
[ 0.000000] 3199MB HIGHMEM available.
[ 0.000000] 728MB LOWMEM available.
[ 0.000000] Entering add_active_range(0, 0, 1005360) 0 entries of 256 used
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0 -> 4096
[ 0.000000] Normal 4096 -> 186368
[ 0.000000] HighMem 186368 -> 1005360
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[1] active PFN ranges
[ 0.000000] 0: 0 -> 1005360
[ 0.000000] On node 0 totalpages: 1005360
[ 0.000000] DMA zone: 32 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 4064 pages, LIFO batch:0
[ 0.000000] Normal zone: 1424 pages used for memmap
[ 0.000000] Normal zone: 180848 pages, LIFO batch:31
[ 0.000000] HighMem zone: 6399 pages used for memmap
[ 0.000000] HighMem zone: 812593 pages, LIFO batch:31
[ 0.000000] Movable zone: 0 pages used for memmap
[ 0.000000] DMI 2.3 present.
[ 0.000000] ACPI: RSDP 000F9BC0, 0014 (r0 ACPIAM)
[ 0.000000] ACPI: RSDT BFF30000, 0030 (r1 A M I OEMRSDT 6000816 MSFT 97)
[ 0.000000] ACPI: FACP BFF30200, 0081 (r2 A M I OEMFACP 6000816 MSFT 97)
[ 0.000000] ACPI: DSDT BFF30360, 3417 (r1 P4V88 P4V88001 1 INTL 2002026)
[ 0.000000] ACPI: FACS BFF40000, 0040
[ 0.000000] ACPI: APIC BFF30300, 0052 (r1 A M I OEMAPIC 6000816 MSFT 97)
[ 0.000000] ACPI: OEMB BFF40040, 003F (r1 A M I OEMBIOS 6000816 MSFT 97)
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
[ 0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[ 0.000000] IOAPIC[0]: apic_id 2, version 3, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: IRQ0 used by override.
[ 0.000000] ACPI: IRQ2 used by override.
[ 0.000000] ACPI: IRQ9 used by override.
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] Allocating PCI resources starting at c4000000 (gap: c0000000:3ffc0000)
[ 0.000000] PERCPU: Allocating 28552 bytes of per cpu data
[ 0.000000] NR_CPUS: 32, nr_cpu_ids: 2
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 997505
[ 0.000000] Kernel command line: root=/dev/md2 console=tty0
[ 0.000000] Enabling fast FPU save and restore... done.
[ 0.000000] Enabling unmasked SIMD FPU exception support... done.
[ 0.000000] Initializing CPU#0
[ 0.000000] PID hash table entries: 4096 (order: 12, 16384 bytes)
[ 0.000000] Xen reported: 2792.268 MHz processor.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] console [tty0] enabled
[ 0.004000] console [hvc-1] enabled
[ 0.004000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.004000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.004000] Software IO TLB enabled:
[ 0.004000] Aperture: 64 megabytes
[ 0.004000] Kernel range: c3bce000 - c7bce000
[ 0.004000] Address size: 27 bits
[ 0.004000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.004000] Memory: 3888208k/4021440k available (1843k kernel code, 124136k reserved, 742k data, 196k init, 3275968k highmem)

Sadly, the machine gets so slow that it needs >20 mins (1100 secs) to boot. Most likely something is broken with PAE on this machine so it get's awfully slow.
I'm curious what the Asrock support will say about this!? ;)

Side note: I'm really satisfied with the good response from Asrock Support. Asrock might not be the company with the best mainboards available, but its support seems very good to me: very responsive and care taking!

Here s for the second installment of my review of this past year s Summer of Code at Debian. See the previous part here: Debian Summer of Code 08 : Where are they now (part 1/3). I apologize for being so late at getting this second part out but I have been very busy. Still, I ll get the last part out before FOSDEM. Those of you who ever had to write a Java compiler (ok, Java subset, but the OOP part was here ) in brainfucking Ada will understand what I went through working on two of my most loathed languages. Debian NAS, improve support of Debian on NAS devices Presentation There is a large range of inexpensive Network storage devices available on the market. For some of them, such as Linksys NSLU-2 and Thecus N2100, we have added support, but there is many many more devices we could support. For this summer we look forward at supporting multiple Marvell Orion based devices (as outlined in Martin Michlmayr s talk Running Debian on Inexpensive Network Storage Devices), such as Revogear Kuro Box Pro, Buffalo Linkstation, QNAP TS-109+, If you don t have old computers lying around to turn into NAS servers, you need to sleep at night without the soothing sound of computer fans or if you actually pay your own electricity bill, you might want to have a look at standalone NAS devices. They re cheap and can be made vastly more capable by slapping a Debian on it. If you ever heard of DD-WRT, you know the spirit. The project was mentored by Riku Voipio, with help from Martin Michlmayr. The project proposal (sorry, Google cache) was introduced by Martin, who did a presentation about it the previous year at FOSDEM. Student Per Andersson was a 24 year old student working towards a MSc in computer science at Chalmers University of Technology in Sweden. He had been looking for ways to join Debian but with school still being priority one, he didn t find time to dive in. Result This project was successful. The Kurobox Pro is now supported and several useful tools were packaged to make life easier with these NAS devices. Martin Michlmayer is still working on Debian NAS related stuff. Per was happy to be invited to the Emdebian work session in Extremadura and has been active within debian, maintaining the packages he created during the Summer of Code. Cran2deb, generate Debian packages from R packages Presentation GNU R has become the preeminent platform for computing with data . The CRAN archives contain over 1300 source packages of very high-quality, and BioConductor has again almost as many focuses on bioinformatics. We want more of these in Debian, and going beyond the 50+ packages we currently have suggests more scripting and automation. R is a pretty big among statisticians and all of them they wasted no time writing their own package to work on particular research subject. It s a lot like Perl with CPAN or LaTeX with CTAN. It s always a pain to discovery that a particular R package is not wihtin Debian and having to resort to unmanaged installation of said packages. The project was mentored by Dirk Eddelbuetel. The project proposal (which is nowhere to be found but seemed to be good) was introduced by Dirk, along with another proposal he did for R. Student Charles Blundell is a research student at.. hum.. didn t do my homework about that. Anyway, you can find him around R related projects. Result This project was successful. Cran2deb is happilly turning more than 1400 of the ~1500 CRAN R packages, all with correct dependencies. The work has since been moved to R-Forge. It s working, we re almost there. We just need it to be polished and we ll get a whole bunch of new packages into Debian. Charles pinged me about the status of Cran2Deb after the previous post. He admits that he hasn t done much about cran2deb recently because of his new position as a research student but hopes to commit again to it soon. I do encourage him to get these R packages into Debian. I had to manually install some packages myself when I had to use R for school because they weren t into Debian and it s not pretty. Mergemaster, interactively merge changes in configuration files Presentation FreeBSD has a shellscript called mergemaster which is used to interactively merge changes in configuration files, based on 3-way diffs. Debian s approach to configuration file differences is much more primitive: either keep the original file, or blow it away (including all local changes) and use the Debian-provided file. It would be nice to get a system such as mergemaster into Debian. Important is to remember that Debian contains two often-used configuration file management systems: ucf, and conffiles; porting mergemaster in such a way that it will be used in both cases would be great. The handling of configuration files during upgrades has always been a little.. brutal, with the user being asked at gunpoint to make a good decision, lest the upgrade won t continue or configuration files get borked (ever tried automerging nagios configuration files?). Having a less stressful upgrade experience is a good thing since the point of Debian is to make package management a stressless thing. The project was mentored by, hum, Manoj Srivastava. I have no idea who came up at first with the proposal. UPDATE: Wouter Verhelst mailed to say that he made the original proposal. Student Max Wiehle was a physics student at the University of Heidelberg. He did a Summer of Code stint (Archive.org copy..) as a student for Beagle Project in 2006 which, I suppose, was successful. He s been active in the past with Gnome and desktop related projects. Result This project was somewhat successful. He posted an update one month into the program with repositories with code to test. Last commit to the mergecf branch of project was September 19th but it was never merged in. According to Steve McIntyre, it s dead, Jim. I couldn t find any further public involvement of Max within Debian. PAS NSS Debian Installer, improve support of PAM and NSS at install-time Presentation It would be very important for the Debian allowing the user to configure additional PAM and NSS modules (eg. LDAP, NIS) during the installation process inside the Debian Installer. To do this, we have to provide tools and helpers to modify /etc/nsswitch.conf and /etc/pam.d/common-*, as well as changing the maintainer scripts for the packages libpam-* and libnss-* to apply the required changes at install time using debconf and these helpers. To be honest, I will probably never use this. I don t do that many coordinated installs in the same place to warrant doing funny authentication with PAM and NSS, and if I did, I would probably use a more elaborate tool to personalize the install, like FAI. On the other hand, I can see the appeal of being done with authentication mechanisms before the first boot. The project was mentored by Fabio Tranchitella. The proposal came from the student. Student Juan Luis Belmonte was a computer science student. He worked in a couple of companies in the area of Sarragossa. He is now founding debug_mode=ON. Result This was quite a disappointement after seemingly good work. Although Juan was satisfied with the project, the PAM package maintainer (Steve Vorlon Langasek) was not. He was never asked about this project (but didn t intervene timely either when the accepted projects were announced though). In his words, it was the wrong solution to the problem . You can find his lenghty rationale on the wontfix bug report that resulted from the project. It really was a problem of communication with the Debian developpers since Juan could certainly have done the right work if pointed to it. Juan didn t ask thoroughly for existing work and Steve didn t publicize his (enough). That s all for now. The information is quite fragmented I admit. Most of it was pulled from Google, mailing lists, commit logs, blogs, whatever. If some projects are lacking in information here, it s because I couldn t find it readily (which is an issue in itself!). In my next post, I ll try to give a student point of view of the Summer of Code in general, and more specifically, at Debian. It will be post 2.5/3 since it s getting a little longer than I planned. Release early, release often, as the say.
If you re a student or a mentor mentioned above, feel free to fill any of the blanks in my report. It s much appreciated. You re not a student or mentor mentioned above and have an opinion on how to improve the next Debian Summer of Code ? Feel free to comment.

And once again I was bitten by problems on my Debian laptop mounting directories from the file server via NFS. After a Debian dist-upgrade I couldn t login to KDE any more. Shell login worked somehow but I quickly found out I could neither read nor write any files there. Apparently an ls -al showed me the right permissions (not just unmappable numeric UIDs or something of that kind) and an id -a confirmed that my LDAP PAM configuration still worked. But reading or writing any file just lead to Permission denied . I m not sure if it was an update in the nfs-common package in Sid. But I had the same problem before and it took me hours until I finally figured out that Debian s NFS seems to use non-standard defaults. Namely the sec parameter when mounting the NFS share. According to the Solaris documentation the default is sec=sys which means that NFS uses the locally acquired UIDs and GIDs. Like /etc/passwd, NIS or LDAP/PAM. But on Debian it seems to default to sec=krb5 or something. As I have close to no idea how to set up Kerberos and don t want to (and talking to other people hardly anyone has used Kerberos either) I figured that it s not really a sane default. I didn t even ask for NFSv4 - just NFSv3. Perhaps I undeliberately set some /etc/default/nfs-common configuration setting wrong or whatever. It was just strange. So I set sec=sys in the options of the NFS share of my /etc/fstab and the problem was fixed. Actually I wonder what network file systems other people use in a Debian environment. NFS somehow feels antiquated to me anyway.

I'm going to re-do my blog sometime soon.The host my current blog is running on is going to retire "anytime soonish" (I happen to be responsible for that, so I will know when). So I will need to move my blog someplace else (I want to merge it into my regular web page, too). Right now, it's based on an ancient version of Pyblosxom with a couple of custom hacks, and is pretty much unmaintainable.So I'm also looking for an appropriate blog software, but I'm not yet sure about my requirements, so I don't have a clear software favorite yet. I don't need a browser based editor, and I'm not interested in interactive features such as comments or trackbacks.Mnemosyne is one of my favourites, because it's a rather low-level, bare-bones approach. Its templating is XML-based (KID) and it's Python.Ikiwiki actually calls itself a "wiki compiler", but I know that some people use it for their blogs. It also does valid XHTML, although its templating isn't designed for XML safety.Features that I would like to have:

• XHTML output, ensuring valid XML, and appropriate templating
• File-based data storage, no MySQL
• Python preferred, but pretty much 'Anything but PHP'
• Static rendering / caching
• Tag based organization
• Search engine optimization (static pages, sensible URIs)
• Atom and RSS 2.0 output, ensuring valid XML

I guess I'll just end up writing a blog software for myself. The current text files are an odd mixture of HTML and plain text anyway, so I'll need to write an importer for the old data myself anyway (plus then I can also do some URI mapping to redirect old links). My current web pages are generated via an XSLT transformation, so I maybe just re-use that for templating.The thing which probably gives the largest issues is pagination of list views. (Especially when you want the pagination to be stable even when new entries are added while a user is browsing the pages, which is nice to have because of search engines and links). I don't think a fully static rendering will work here.

I'm going to re-do my blog sometime soon.The host my current blog is running on is going to retire "anytime soonish" (I happen to be responsible for that, so I will know when). So I will need to move my blog someplace else (I want to merge it into my regular web page, too). Right now, it's based on an ancient version of Pyblosxom with a couple of custom hacks, and is pretty much unmaintainable.So I'm also looking for an appropriate blog software, but I'm not yet sure about my requirements, so I don't have a clear software favorite yet. I don't need a browser based editor, and I'm not interested in interactive features such as comments or trackbacks.Mnemosyne is one of my favourites, because it's a rather low-level, bare-bones approach. Its templating is XML-based (KID) and it's Python.Ikiwiki actually calls itself a "wiki compiler", but I know that some people use it for their blogs. It also does valid XHTML, although its templating isn't designed for XML safety.Features that I would like to have:

• XHTML output, ensuring valid XML, and appropriate templating
• File-based data storage, no MySQL
• Python preferred, but pretty much 'Anything but PHP'
• Static rendering / caching
• Tag based organization
• Search engine optimization (static pages, sensible URIs)
• Atom and RSS 2.0 output, ensuring valid XML

I guess I'll just end up writing a blog software for myself. The current text files are an odd mixture of HTML and plain text anyway, so I'll need to write an importer for the old data myself anyway (plus then I can also do some URI mapping to redirect old links). My current web pages are generated via an XSLT transformation, so I maybe just re-use that for templating.The thing which probably gives the largest issues is pagination of list views. (Especially when you want the pagination to be stable even when new entries are added while a user is browsing the pages, which is nice to have because of search engines and links). I don't think a fully static rendering will work here.[Update: Sean pointed out that by using title-only in archive pages I can avoid having to paginate. Thanks! I also received TONS of software recommendations. Ranging from PHP over bash (URGH, I want valid XHTML) to mod_lisp based (no special Apache modules, please. Should run everywhere.)]

To all of my readers: I wish you a happy new year 2009!
Being late with new years greetings, because of family reasons like visiting, eating and such, I hope these images will compensate you:



More images can be found in my Gallery.


Auch meinen deutschsprachigen Lesern w nsche ich alles Gute und ein sch nes neues Jahr 2009!



Das gestrige Ereignes "Leuchtturm in Flammen" in Warnem nde war wieder sehr gut besucht. Nicht nur die Promenade war mehr als voll, sondern auch der Strand. Wir standen in H he des Kurhause am Strand, was vielleicht 100-200m zu weit weg f r mein Nikkor 18-105mm 1:3.5-5.6G ED Objektiv war, aber es gab ja trotzdem ganz gute Bilder.

Nach Beendigung der Veranstaltung str mten die Menschenmassen jedenfalls wieder zu ihren Autos, was in einem sch nem Verkehrschaos endete. Auf der Parkstra e ging es sicherlich fast eine Stunde nur in beiden Seiten mit Stop and Go voran. Auch die Nutzer von ffentlichen Verkehrsmitteln hatten wenig Freude daran, da der Bus ja genauso im Stau stand wie all die anderen Autos.

A few hours and this year will be over. Instead of making a summary of 2008, I feel the need to list a few of my goals for 2009:

• Finally complete the building of our house;
• Buy a piano and start again playing it, I really miss it;
• Buy a carom billiard, I really miss it;
  
• Run the Turin Marathon 2009 in spring (my first marathon ever);
  
• Run the Budapest Marathon 2009 in autumn (plus: in less than 3.5 hours);
• Launch ZeNEM.hu and OXA (two business projects of mine);
• Further develop our database marketing company in Hungary, and make it grow;
  
• Start practicing a martial art for self-defense;
• Start the training for my PPL (plus: complete it in 2009);
  
• Travel across Spain (and attend to the DebConf in Extremadura);
• Teach Greta and Sophia to swim;
• Deploy SATISFLY to the launch customer;
• Buy a new car for my wife;
  

This list is not exhaustive, but writing down these items helps me. After a so-so 2008, I'm looking forward for a new exciting year: I feel very motivated and I'm sure that 2009 will be positive and rich of satisfactions.

Happy New Year to everybody, may the Force be with you.

Happy solstice! High point of the season for me, and today was a two party day. My sister has reached the point where a good b-day present for her is merging the blog style comments branch into ikiwiki and installing linux on yet another laptop. So we had a little installation party with cake and Carcassonne. Except for wireless it was a success. Since I outsourced the other present to the UK, perhaps someone can sort out Christmas by telling me how to get Intel 5100 wireless working in Debian? ;-) At the solstice party tonight, all three of the recent inhabitants of the bottom of Wortroot washed up on the porch, each of us looking a bit scraggly (especially the current resident). Jeremy told this story:

His friend decided to drive his SUV down the driveway to the farm, without consulting with him first. (This driveway was blasted out of the hillside by amateurs, using dynamite, in an earlier era of life at the farm. It has been basically unmaintained since, progressively more eroded and rutted, and over the last 20 years only the Yellow Truck (with no brakes) has had free reign over it.) He got down the hill ok, and he passed the small field, and then he got to the narrow place. (Where the creek is eroding the driveway away faster than we can cut into the hill to widen it.) And Jeremey told him to stop, but he kept on going.. sliding right off the driveway, and rolling the SUV into the creek. (The creek here is, now that I think of it, exactly as wide as your average SUV is tall, and in a gully exactly as deep as your average SUV is wide. It must have fit perfectly.) They tried winching it out. That didn't work. They didn't even think to ask neighbor Jimmie to once again haul a vehicle up the driveway. His tractor couldn't have gotten this one out. They had to rent a "small backhoe on treads, like a tank" to do the job.

Ok, maybe you have to have lived there to appreciate this. :-)