Search Results: "tach"

9 May 2009

David Paleino: OpenPGP key transition announcement

Because of the recently announced attack against the SHA-1 digest algorithm, I finally decided to move away from my old 1024-bit DSA OpenPGP key, landing to a shiny new 4096-bit RSA one. The old key will continue to be valid for some time, but I prefer all future correspondence to come to the new one. I would also like this new key to be re-integrated into the web of trust. I'm attaching a file with
the complete text of the transition, clearsigned by both keys to certify the transition. You can find the transition statement also on http://www.hanskalabs.net/key-transition_20090509.txt . You can check the validity of the file with something like:
$ gpg --decrypt key-transition_20090509.txt   gpg --decrypt
If you need to transition your key too, and you're one of my signees, then let's please coordinate, so that we have the new keys cross-signed, without spurious signatures fromgoing-to-be-revoked keys Smile Thank you for your help.
AttachmentSize
key-transition_20090509.txt4.03 KB

7 May 2009

Biella Coleman: Go Elsevier

It looks like publishing fake journals was becoming closer to habit for Elsevir rather than a one-time exceptional mistake.
Scientific publishing giant Elsevier put out a total of six publications between 2000 and 2005 that were sponsored by unnamed pharmaceutical companies and looked like peer reviewed medical journals, but did not disclose sponsorship, the company has admitted. Elsevier is conducting an internal review of its publishing practices after allegations came to light that the company produced a pharmaceutical company-funded publication in the early 2000s without disclosing that the journal was corporate sponsored.
I really hope people use this an an opportunity to detach fully from them.

Biella Coleman: Go Elsevier

It looks like publishing fake journals was becoming closer to habit for Elsevir rather than a one-time exceptional mistake.
Scientific publishing giant Elsevier put out a total of six publications between 2000 and 2005 that were sponsored by unnamed pharmaceutical companies and looked like peer reviewed medical journals, but did not disclose sponsorship, the company has admitted. Elsevier is conducting an internal review of its publishing practices after allegations came to light that the company produced a pharmaceutical company-funded publication in the early 2000s without disclosing that the journal was corporate sponsored.
I really hope people use this an an opportunity to detach fully from them.

1 May 2009

Obey Arthur Liu: Z rich, Switzerland for the summer (housing? DebConf?)

Z rich Google Earth I got a summer internship at Google in Z rich, Switzerland, so I ll be moving there this summer. Any DD working there ? I already went to Z rich once last year and it was quite a cool place. I briefly met Cate and I know that there are other DDs there (hi Madduck!). I look forward to meeting more of you. Now there are two issues I d like help with, dear Z richer Lazyweb, housing and flight to DebConf. Can I haz apartment Looking for an apartment in Z rich seems very very very hard.. So far I only found a few apartment/hotel geared towards rich expats staying for few months (the kind like Citadines). It s ungodly expensive (like 2200+ CHF a month for one room), but sure available and well placed.. Here s what I need:

I don t have much more criterias, as long as I can find something. We re about a few dozens Google interns all (desperately) looking for accomodation in Z rich. We re all good neighbours, don t party all night and all have very good references. Please contact me if you \<know someone who\>* know something, I d be very grateful. Can I haz DebConf I m going to DebConf this year in Spain, along with all the Google Summer of Code students who can make it. I m going to reserve my flight in the next few days. I think I ll only go to DebConf proper, arriving on the 23rd and leaving on the 31st of July. I m not decided yet on the last point. So, any of you coming to DebConf from Z rich ?

Simon McVittie: Implementing D-Bus clients with telepathy-glib (and telepathy-qt4)

(The much-delayed part 2 in an ongoing series: shiny things in telepathy-glib) Over a year ago I explained my first major round of work on telepathy-glib, with which it could be used to implement D-Bus services. I started to write this post shortly after, but then got distracted by a bee or something; recent discussion about having a D-Bus binding in GLib has finally prompted me to finish it. The second major round of work on telepathy-glib dealt with implementing client code, and went sufficiently well that telepathy-qt4 (currently under development) uses essentially the same model. Again, while telepathy-glib is intended for Telepathy implementors, I think TpProxy's ideas are generically useful, particularly for "large" D-Bus APIs.

Some background: dbus-glib and libtelepathy
GObject-CRITICAL: Assertion failed: proxy->convenient
The life-cycle of a DBusGProxy (PD clipart credit: karderio and Andy)
The API provided by dbus-glib for client code is, er, rather less than ideal. The basic object is DBusGProxy, which has a few problems. There's one per interface, rather than one per object; this leads to a maze of twisty GObjects, all suspiciously similar. Our old client library, libtelepathy, got round this by subclassing DBusGProxy to make helper objects called TpChan, TpConn etc., which were simultaneously the proxy for the "main" interface (Channel, Connection etc.), and a factory for proxies for the other interfaces (like Channel.Type.Text and Channel.Interface.Password). This still wasn't ideal, but it was a start. If you use a DBusGProxy in any way after it has emitted the destroy signal, this is considered to be invalid use of the API, and dbus-glib asserts all over the place. We'd rather it didn't do that - remote objects becoming invalid is a fact of life, and if you don't immediately discard all your references to those objects, remote operations on them should just give you a runtime error. You have to be prepared to handle such runtime errors already, because any IPC call can fail at any time. The main API for calling methods uses varargs and is rather subtle. Notably, the calling convention for variant parameters (of D-Bus signature 'v') is not consistent with the calling convention for any other container. For any other container, for instance a GPtrArray *, you pass in a GPtrArray ** pointing to a GPtrArray * variable containing NULL; on success, a pointer to a new GPtrArray is written into that variable. For variant parameters, though, you must pass in a GValue * pointing to a zero-filled GValue, and the result will be written into that GValue. dbus-binding-tool generates thin wrappers around this method-calling API which provide type-safety, although they can't directly specify a non-default timeout. As for signals, before binding to a signal, you have to tell dbus-glib about its signature. This tells dbus-glib how it should push the signal arguments onto the C stack (in GObject terminology: how it should marshal them), to match the signature of the signal-handler function. The documentation claims that this isn't necessary for objects supporting the Introspect method, but that's untrue - on the other hand, if the documented behaviour actually happened, I would consider this to be a critical bug in dbus-glib, since it would enable remote objects to crash dbus-glib clients (by causing signal arguments to be pushed onto the stack in a way that does not match the signature of the signal handler). Of course, the signal-adding function takes varargs, and has to be called exactly once per signal per object. If you don't, dbus-glib asserts when you connect to the signal; if you call it twice, dbus-glib asserts anyway.

TpProxy TpProxy is a GObject subclass vaguely inspired by Python's dbus.proxy.ProxyObject. It isn't a subclass of DBusGProxy, which lets us encapsulate dbus-glib almost completely - in fact, all the references to DBusGProxy are in a separate header, proxy-subclass.h, only used by subclasses of TpProxy (as opposed to normal library API users). TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. As explained below, I believe that this is the most natural object mapping for D-Bus interfaces; in practice it tends to lead to clearer code for us, since one remote object often maps nicely to one UI element (of course, if your objects only have one interface, this approach is no better or worse than dbus-glib's). TpProxy provides support for the following common D-Bus things:
  • Calling methods
  • Connecting to signals
  • Objects becoming invalid
  • Objects with optional interfaces
  • Extensibility
It doesn't currently provide any particular help with Properties - the Properties interface is just another interface as far as we're concerned. We find that, in practice, that's not a problem (even though we use D-Bus properties extensively). In telepathy-qt4 we basically reinvented TpProxy in C++, as Tp::DBusProxy. I've put in some comments below illustrating how the general ideas we used in TpProxy translate into a different object mapping.

Calling methods Method calls on TpProxy are done through auto-generated wrapper functions, much like DBusGProxy. These look something like this:
typedef void (*tp_cli_channel_callback_for_close) (TpChannel *proxy,
    /* any 'out' arguments would go here - Close() doesn't have any */
    const GError *error, gpointer user_data,
    GObject *weak_object);
TpProxyPendingCall *tp_cli_channel_call_close (TpChannel *proxy,
    gint timeout_ms,
    /* any 'in' arguments would go here - Close() doesn't have any */
    tp_cli_channel_callback_for_close callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object);
Methods for some interfaces, like DBus.Properties, are available on TpProxy itself; others, like Telepathy's Channel.anything, are only available on a particular subclass, like TpChannel. Things to note here:
  • It's an asynchronous call with a callback. General design principle: "this is IPC, get over it". We don't try to hide the fact that IPC is taking place by doing pseudo-blocking calls.
  • The timeout is explicit (although you can always pass -1 to let libdbus pick a "reasonable" default for you). This is because the remote service might not respond, and API users should be at least vaguely aware of this: "this is IPC, get over it".
  • The callback isn't a GCallback; it has an explicit signature, for some semblance of type-safety (it's not much, but it's better than nothing).
  • The callback takes a GError, because any IPC call can fail for any reason.
  • A TpProxyPendingCall * is returned. This is a pointer to an object that only exists until just after the callback is called (so you can ignore it if you don't want it), with a method you can call to cancel the method call (with hindsight this is not ideal, and if I was writing this API now, I'd use gio's GCancellable). "Cancel" is perhaps slightly misleading: the method call still takes place (it was already in libdbus's outgoing queue) but the result gets ignored and the callback isn't called. This is mostly so you an object can forget all the calls it was busy making at the time that the results become irrelevant (e.g. the object is destroyed).
  • There is an extra weak_object argument, which is weakly referenced and is passed to the callback: in practice, most users of telepathy-glib use this as well as or instead of user_data. If the weak_object runs out of references, the callback is automatically cancelled, which means that in practice, explicit cancellation is almost never needed.
  • The generated code has as its first argument a suitable subclass of TpProxy, in this case TpChannel (we tell the code generator about subclasses).
  • You can't see it in the API, but the TpProxy gains an extra ref while a method call is in progress, so you never have to worry about what happens if the proxy gets unreffed during a method call.
  • There's an explicit destructor for the user_data, which is called even if you "cancel" the method call.
In telepathy-qt4, these methods are on small generated helper classes similar to dbus-python's dbus.Interface, one per interface (we provide accessors for them on the non-generated DBusProxy subclass). Instead of using callbacks, we return a temporary QObject per call, which emits a Qt signal on success or failure (that's how all async calls in QtDBus work).

Connecting to signals Similarly, signals have some generated functions:
typedef void (*tp_cli_channel_signal_callback_closed) (TpChannel *proxy,
    /* any arguments would go here - Closed doesn't have any */
    gpointer user_data, GObject *weak_object);
TpProxySignalConnection *tp_cli_channel_connect_to_closed (TpChannel *proxy,
    tp_cli_channel_signal_callback_closed callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object,
    GError **error);
API notes for these:
  • The callback doesn't take a GError any more, because signals can't fail; the weak_object and the user_data are the same as for method calls
  • This isn't actually a GObject signal at all: those aren't particularly typesafe, and lack namespacing. This might not be very binding-friendly, we haven't tried doing that yet... it might be worth introducing a signal with details, like in dbus-glib, for which these functions are "C bindings".
  • The returned TpProxySignalConnection lets you disconnect from the signal, just like the TpProxyPendingCall above; it can safely be ignored
  • Similarly, if the weak_object dies, then the signal automatically disconnects
  • It is possible for connecting to a signal to fail: this happens if the proxy doesn't actually have the requested interface. If this happens, it's graceful, not a crash. In practice, the flow of code is almost always such that error can safely be NULL because the interfaces are already known.
In telepathy-qt4 the per-interface generated helper classes emit Qt signals. In many cases, we end up proxying these signals by responding to them in the hand-written Tp::DBusProxy subclass by emitting a different signal, in order to give them a nicer representation.

Becoming invalid This is IPC, and any error can happen to you at any time. We found that it was useful to have a general concept of "this object is no longer useful", so we introduced the concept of an object becoming invalidated. The invalidation reason is stored on the object as a GError. If this GError is NULL (as it is initially), then the object is still expected to be valid; if it's non-NULL, the remote object has vanished and the local proxy no longer works. There are several ways a TpProxy can become invalid:
  • its bus name falls off the bus (the process exited or crashed)
  • application-specific reasons (usually triggered by a D-Bus signal in a subclass, like the Telepathy Channel's Closed signal)
  • the GObject gets disposed
Method calls on an invalidated object always fail, with the invalidation reason as the error. This means that calling a method on a TpProxy should never crash your process. The method's callback has to be prepared to handle an error in any case, because this is Sparta^WIPC, so there's no loss in giving it another error. Connecting to signals on an invalidated object always fails, and any existing signal connections are disconnected when it becomes invalidated. This means that if a new remote object appears, and it happens to have the same object path, bus name etc. as the old one, you won't get its signals. A TpProxy can either be bound to a unique bus name or a well-known bus name. In the equivalent code in telepathy-qt4, we call this "stateful" or "stateless" - these names don't capture the intention perfectly, but they're the best we could come up with. A proxy for a stateful API like Telepathy's Channel should always bind to a unique name - when the unique name falls off the bus, the TpProxy becomes invalidated automatically, representing the fact that the Channel you had no longer exists, and nothing can be a perfect replacement for it (the VoIP call has already been terminated, you've already left the chatroom, or whatever - if you make another Channel, that's e.g. a distinct VoIP call). A proxy for a stateless API like Telepathy's ConnectionManager, where an exiting process can disappear, be service-activated again, and still have exactly the same API, should bind to the well-known name. Proxies for well-known names aren't invalidated when the process exits. Qt doesn't have a natural equivalent for GError, so the invalidation reasons in telepathy-qt4 are just a pair of strings, the namespaced name and the message - basically a libdbus DBusError. The principle is the same, though.

Optional interfaces As mentioned above, TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. I believe that this is the most natural object mapping for the D-Bus object model, because the D-Bus interfaces on a remote object can behave like any of these:
  • classes and subclasses (Telepathy.Channel.Type.Text is a Telepathy.Channel)
  • shared interfaces (some Telepathy.Channel objects, of many Types, implement Telepathy.Channel.Interface.Group; any D-Bus object can implement DBus.Properties)
  • optional features (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.ChatStates, some do not)
  • discoverable extensions (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.Messages, which is more or less Text 2.0)
The way in which interfaces are discovered is also variable. The "classic" D-Bus way to discover interfaces would be to call Introspect. Telepathy services still support that style of introspection, but we don't use it for anything beyond d-feet:
  • Introspect returns a blob of XML, which you have to parse into something useful; if it was in a D-Bus data structure, you'd already have some sensible data structure.
  • Finding out more information about the supported interfaces than their names isn't very useful: in a static language like C you already need to know the methods, signals, properties and their signatures in order to write and compile your code, and in a dynamic language like Python consumers of Introspect end up being remotely crashable by services with unexpected method signatures (this is one of dbus-python's big mistakes in my opinion).
  • If a method that you want to call turns out to be missing, the worst that can happen (in a competently written service) is that it fails with a D-Bus error - and you have to handle D-Bus errors anyway, because this is IPC and anything could fail.
  • Static bindings like dbus-glib have poor support for removing interfaces for which C code exists at runtime, whereas Telepathy needs features like an IM connection that might or might not support avatars (we don't get to find out until we've successfully connected to the server).
Instead, our older interfaces have a method on the "base class" (Channel, Connection etc.) called GetInterfaces, which just returns an array of strings. Newer interfaces (like Account), and older interfaces that have been ported (like Channel), have a property called Interfaces which is, again, an array of strings - this lets us combine the download of the interfaces list with downloading other basic information in a GetAll call. To cater for all these ways to use interfaces, the TpProxy base class has very basic support for interface discovery - you can ask it whether it supports an interface, and you can tell it that it does, in fact, support an interface. Asking which interfaces are supported is directly useful for library users; it's also used as a check by the generated method-call and signal-connection stubs, which return a canned error (Telepathy.Error.NotImplemented in Telepathy's case) if the object isn't known to have the interface. Telling TpProxy that it does support an interface is intended for use by subclasses, and might have been protected if we were writing C++: what happens in practice is that a concrete subclass like TpChannel knows how to discover the fully supported interfaces for this particular object, does so, and calls methods on the TpProxy to tell it which interfaces can work.

Extensibility The API stubs used by TpProxy to call methods and bind to signals are generated from XML documents containing an augmented form of D-Bus introspection data. These XML documents contain various Telepathy-specific extensions, but none of the extensions are language-specific - telepathy-glib, telepathy-python and telepathy-qt4 all operate from exactly the same XML specification. telepathy-qt4's different object-mapping did require us to tighten up some rules that were previously only conventions, but even so, the format is the same. I think this is vitally important - it just doesn't scale to have one set of language-specific annotations in a D-Bus API for each language that will have bindings. One thing that we make heavy use of is what I call "Ugly_Case", which is camel-case with underscores at word boundaries. This gives us a simple, unambiguous and foolproof rule to use in code generators whenever we generate any of the popular conventions for identifiers:
  • CamelCase: delete underscores
  • javaCamelCase: force everything before the first underscore to lower case, then delete underscores
  • lower_case: force everything to lower case, retain underscores
  • UPPER_CASE: force everything to upper case, retain underscores
dbus-glib, by contrast, applies a complex and subtly buggy algorithm to the CamelCase D-Bus names, which results in our SendDTMF method being mapped into GLib as send_dt_mf, and requires that telepathy-glib's Python reimplementation of dbus-binding-tool uses a bug-for-bug compatible implementation (we ended up reimplementing dbus-binding-tool in order to generate the TpProxy-based bindings). Our code generation tools are still rather ad-hoc, because so is our spec format, but for draft interfaces it's possible to copy them into individual projects and use them to generate additional methods for TpProxy or any subclass. This is how we implement unfinished APIs like geolocation in Empathy, for instance. The Interfaces or GetInterfaces hook described above is entirely usable for these extension interfaces, and in fact that's how they're set up. In telepathy-qt4, the generated method stubs are genuine C++ methods, so we can't just append methods to an existing class: this is why we have one helper class per interface, so individual clients can generate a helper class for their particular version of an unfinished API, and instantiate an object of this class attached to a particular Tp::DBusProxy. When an interface becomes stable, it can be added to the set of interfaces for which code is generated in telepathy-glib or telepathy-qt4, at which point any client or service that was already using the final draft of that interface can easily be ported to the library version using sed (the API remains the same).

Becoming ready The concept of being "ready" does not directly exist in TpProxy, but is implemented in TpChannel, TpConnection and TpConnectionManager. The idea is that a freshly created proxy object isn't really fully usable yet - you have to connect to basic signals and recover the initial state of the remote object, as well as checking which extension interfaces are supported. In many well-designed D-Bus APIs you can do this initial setup with a few signal connections and a DBus.Properties.GetAll call. After doing that initial setup (which has to be done asynchronously, because it's IPC), the TpProxy subclass has a local cache of the remote object's state (not necessarily in the same format as the representation on the bus), which can be accessed synchronously at any time, and will be updated in response to change-notification signals. TpChannel and subclasses like it have explicit support for checking for readiness, and having a callback called when the object is ready or invalidated (whichever happens first). The "ready" property also supports the GObject notify signal. telepathy-qt4 took this concept and ran with it - there is library support for objects that can become ready, including a mixin. If I had as much time to work on telepathy-glib as I wish I had, this would be one of the first telepathy-qt4 features to be added to telepathy-glib.

Features becoming ready Going beyond that, in an extensible framework like Telepathy it's probable that not every client understands (or wants to understand) every feature of every object, so it's highly inefficient for every proxy object to subscribe to all the change notification signals and cache all the remote state on the off-chance that the library user wants them. In the API for TpContact (which is not actually a TpProxy for various reasons) we introduced the concept of optional features; telepathy-qt4 extended this idea throughout the library. The idea is that each library user knows which of the optional features are "interesting" to it, and makes a single asynchronous call (which expands into a series of D-Bus calls inside the library) to download the state for core functionality plus all of the selected features, and subscribe to change notification for all of those. This is another telepathy-qt4 thing that telepathy-glib still needs to catch up with; until someone works out how to make the clone() syscall apply to programmers, I fear all libraries are doomed to lag behind how their designers want them to look :-) (license: CC-BY-3.0 / LGPL-2.1+)

28 April 2009

Florian Ragwitz: Implementing Typed Lexical Variables

For quite some time perl provided a form of my declarations that includes a type name, like this:
my Str $x = 'foo';
However, that didn't do anything useful, until Vincent Pit came along and wrote the excellent Lexical::Types module, which allows you to extend the semantics of typed lexicals and actually make them do something useful. For that, it simply invokes a callback for every my declaration with a type in the scopes it is loaded. Within that callback you get the variable that is being declared as well as the name of the type used in the declaration. We also have Moose type constraints and the great MooseX::Types module, that allows us to define our own type libraries and import the type constraints into other modules. Let's glue those modules together. Consider this code:
use MooseX::Types::Moose qw/Int/;
use Lexical::Types;
my Int $x = 42;
The first problem is that the perl compiler expects a package with the name of the type used in my to exist. If there's no such package compilation will fail. Creating top-level namespaces for all the types we want to use would obviously suck. Luckily the compiler will also try to look for a function with the name of the type in the current scope. If that exists and is inlineable, it will call that function and use the return value as a package name. In the above code snippet an Int function already exists. We imported that from MooseX::Types::Moose. Unfortunately it isn't inlineable. Even if it were, compilation would still fail, because it would return a Moose::Meta::TypeConstraint instead of a valid package name. To fix that, let's rewrite the code to this:
use MooseX::Types::Moose qw/Int/;
use MooseX::Lexical::Types qw/Int/;
my Int $x = 42;
Let's also write a MooseX::Lexical::Types module that replaces existing imported type exports with something that can be inlined and returns an existing package name based on the type constraint's name.
package MooseX::Lexical::Types;
use Class::MOP;
use MooseX::Types::Util qw/has_available_type_export/;
use namespace::autoclean;
sub import  
    my ($class, @args) = @_;
    my $caller = caller();
    my $meta = Class::MOP::class_of($caller)   Class::MOP::Class->initialize($caller);
    for my $type_name (@args)  
        # get the type constraint by introspecting the caller
        my $type_constraint = has_available_type_export($caller, $type_name);
        my $package = 'MooseX::Lexical::Types::TYPE::' . $type_constraint->name;
        Class::MOP::Class->create($package);
        $meta->add_package_symbol('&'.$type_name => sub ()   $package  );
     
    Lexical::Types->import; # enable Lexical::Types for the caller
 
1;
With that the example code now compiles. Unfortunately it breaks every other usecase of MooseX::Types. The export will still need to return a Moose::Meta::TypeConstraint at run time so this will continue to work:
has some_attribute => (is => 'ro', isa => Int);
So instead of returning a plain package name from our exported function we will return an object that delegates all method calls to the actual type constraint, but evaluates to our special package name when used as a string:
my $decorator = MooseX::Lexical::Types::TypeDecorator->new($type_constraint);
$meta->add_package_symbol('&'.$type_name => sub ()   $decorator  );
and:
package MooseX::Lexical::Types::TypeDecorator;
use Moose;
use namespace::autoclean;
# MooseX::Types happens to already have a class that doesn't do much
# more than delegating to a real type constraint!
extends 'MooseX::Types::TypeDecorator';
use overload '""' => sub  
    'MooseX::Lexical::Types::TYPE::' . $_[0]->__type_constraint->name
 ;
1;
Now we're able to use Int as usual and have Lexical::Types invoke its callback on MooseX::Lexical::Types::TYPE::Int. Within that callback we will need the real type constraint again, but as it is invoked as a class method with no good way to pass in additional arguments, we will need to store the type constraint somewhere. I choose to simply add a method to the type class we create when constructing our export. After that, all we need is to implement our Lexical::Types callback. We will put that in a class all our type classes will inherit from:
Class::MOP::Class->create(
    $package => (
        superclasses => ['MooseX::Lexical::Types::TypedScalar'],
        methods      =>  
            get_type_constraint => sub   $type_constraint  ,
         ,
    ),
);
The Lexical::Types callback will now need to tie things together by modifying the declared variable so it will automatically validate values against the type constraint when being assigned to. There are several ways of doing this. Using tie on the declared variable would probable be the easiest thing to do. However, I decided to use Variable::Magic (also written by Vincent Pit - did I mention he's awesome?), because it's mostly invisible at the perl level and also performs rather well (not that it'd matter, given that validation itself is relatively slow):
package MooseX::Lexical::Types::TypedScalar;
use Carp qw/confess/;
use Variable::Magic qw/wizard cast/;
use namespace::autoclean;
my $wiz = wizard
    # store the type constraint in the data attached to the magic
    data => sub   $_[1]->get_type_constraint  ,
    # when assigning to the variable, fail if we can't validate the
    # new value ($_[0]) against the type constraint ($_[1])
    set  => sub  
        if (defined (my $msg = $_[1]->validate($  $_[0]  )))  
            confess $msg;
         
        ();
     ;
sub TYPEDSCALAR  
    # cast $wiz on the variable in $_[1]. pass the type package name
    # in $_[0] to the wizard's data construction callback.
    cast $_[1], $wiz, $_[0];
    ();
 
1;
With this, our example code now works. If someone wants to assign, say, 'foo' to the variable declared as my Int $x our magic callback will be invoked, try to validate the value against the type constraint and fail loudly. WIN! The code for all this is available github and should also be on CPAN shortly. You might notice warnings about mismatching prototypes. Those are caused by Class::MOP and fixed in the git version of it, so they'll go away with the next release. There's still a couple of caveats, but please see the documentation for that.

25 April 2009

Ingo Juergensmann: Cleaning and modding of an A3000

Finally I was able to clean up one of my m68k buildds: Spice. Actually I regularly clean up all of my machines so they continue to work properly. Usually I do this maintenance work every year around christmas, but this time was different. Anyway. I took some pictures:



In fact I assumed more dust inside after 512 days of uptime, but I was suprised that the dust was moderate. So the cleaning was easy and fast. This is the result:



Next and final step was to enable the Laptop mode. Sadly there was no /proc/sys/vm/laptop_mode, but the final solution wasn't that difficult either to have a portable Amiga 3000:

Amiga 3000 in Laptop mode


Well, the reason for that Laptop mode is quite simply: although the A3000 is quite silent with its Papst fan, its SCSI/SCA disks aren't. To safe some noise I plan to move the A3000/autobuilder to the guestroom, but when there's some visitors, I want to be able to relocate the machine without interrupting the build. So attaching a UPS and a wireless bridge is the way to go for this purpose.

Russell Coker: Creating a Double-Ended Bun


The people who made the above magazine advert gave it two top-halves to the burger bun. But I think that there is actually a demand for such buns, and that it is possible to make them! Traditional buns have a flat bottom where they rest on a baking tray. One solution to this problem would be to bake in outer space, another possible solution would be to develop a rapid baking process that allows baking in a free-fall aeroplane, but both of these would be unreasonably expensive. It seems that it would be viable to bake double-ended buns by having a rapidly rising column of hot air to suspend the bun. The terminal velocity of a bun would probably not be that high (maybe 60Km/h) and it should be quite easy to have a pipe full of hot air that bakes the buns. As slight variations in the density and shape of the bun would affect the air-flow it would be necessary to closely monitor the process and adjust the air speed to keep the bun afloat. Manufacturing cheap ovens that use LASERs to monitor the position of the bun should not be difficult. This might blow the sesame seeds off the bun, but this problem may also be solvable through careful design of the bun shape to make it less aerodynamic and by strongly attaching the seeds. I m not sure how you would do this.

24 April 2009

Stefano Zacchiroli: howto: uploading to people.d.o using dput

dput + mini-dinstall to ease people.d.o uploads As DDs we often want to offer unofficial packages, but still preserving a bit of "authoritativeness". A widespread solution is to upload somewhere under http://people.debian.org/~LOGIN as an APT repository. I've been using that solution too since ... ever, but I've never been happy about how to advertise the packages, probably just because I've never liked dpkg-scan packages,sources and hackish scripts built on top of them. My goal, as simple as it can be, is to work as I do to prepare "real" packages to be uploaded to the Debian archive, and just upload to people.d.o as the final step. Thanks to DSA which satisfied a request of mine by installing a while ago mini-dinstall on ravel, today I've finally set up a work-flow that enabled me to reach my goal. This post is a quick howto to implement that setup. If you have suggestions about where (wiki.d.o? devref?) to paste these info so that other DDs can more easily find them in the future, please let me know, via comments or mailing me. Step 1: configure mini-dinstall on people.debian.org On people.debian.org, create a ~/.mini-dinstall.conf file containing something like the following:
    [DEFAULT]
    mail_to = zack
    incoming_permissions = 0750
    architectures = all, i386, amd64
    archive_style = flat
    dynamic_reindex = 1
    archivedir = /home/zack/public_html/debian/
    generate_release = 1
    release_origin = Zack
    release_label = Zack
    release_description = Unofficial Debian packages maintained by Stefano Zacchiroli
    [zack-unstable]
    release_suite = zack-unstable

Highlights of that conf: The resulting repository will be accessible using the following /etc/apt/sources.list lines:
    deb http://people.debian.org/~zack/debian zack-unstable/
    deb-src http://people.debian.org/~zack/debian zack-unstable/

Step 2: configure dput The default incoming dir of mini-dinstall, relative to the configured archivedir, is mini-dinstall/incoming. The following entry for your ~/.dput.cf (on the machine you upload from) will therefore setup your dput for uploads:
    [people.debian.org]
    fqdn = people.debian.org
    method = scp
    login = *
    incoming = /home/zack/public_html/debian/mini-dinstall/incoming
    post_upload_command = ssh people.debian.org mini-dinstall -b

The only interesting detail here is the post upload command, which will run a "pulse" of mini-dinstall on people.d.o just after the upload, to process the uploaded file. Alternatively, you can leave mini-dinstall running on people.d.o, but I consider it unnecessary (because usually you are the only one able to upload to your home dir), unelegant, and complicated (because you then need to verify mini-dinstall is running). Step 3: enjoy (i.e., upload) Now you can prepare your packages as you always do (which means using cowbuilder!) and, when you are done with your foo_1.0-1_amd64.changes, upload it doing:
    $ dput people.debian.org foo_1.0-1_amd64.changes

Signing the Release file The above setup does not sign Release files, which means your APT users will receive complaints from the secure APT machinery. To fix that you must sign the Release file with some key and distribute the key to your users to be processed by apt-key add. You have a couple of ways to sign the release file automatically:
  1. (/me doesn't like this) use a GPG key created for the occasion, store that key (including the private part) on people.d.o, and use the mini-dinstall contrib script sign-release.sh by adding something like the following to your mini-dinstall.conf:
      release_signscript = ~/bin/sign-release.sh
    
    
  2. (/me likes this) use some script to sign the Release file remotely and send the resulting deatched signature back to people.d.o just after the mini-dinstall pulse; that way you can sign with your own private key, which will be reasonably felt as more trustworthy, being part of the Debian keyring. To achieve that, the most elegant way would be to fix debsign wishlist #465240 (volunteers? :-) ). As I'm lazy, I'm using my own sign-remote script, which is a bit more generic. Remote signing should be attached to dput upload hook as well. All in all, here is my actual ~/.dput.cf stanza:
      [people.debian.org]
      fqdn = people.debian.org
      method = scp
      login = *
      incoming = /home/zack/public_html/debian/mini-dinstall/incoming
      post_upload_command = ssh people.debian.org "mini-dinstall --batch" && sign-remote people.debian.org:~/public_html/debian/zack-unstable/Release
    
    
Enjoy! Update: do both mini-dinstall run and signing in post_upload_command, following gregoa's suggestion via comment

23 April 2009

Matthew Palmer: Insane/Brilliant Idea of the Day

I've been talking serial consoles with a couple of the other guys at work: how nice they are to have for machines in the datacenter, how annoying it is that vPro serial-over-LAN doesn't seem to be robust (yet?), and how serial access concentrators are lung-and-kidney expensive (especially when you've got 50-some racks to outfit). This discussion, combined with my ongoing embedded-hardware-fascination lust for a SheevaPlug appears to have spurred my brain into coming up with a Brilliant Idea: tie a SheevaPlug to a pile of USB to serial adapters and use that as your per-rack serial concentrator. Imagine: faffenheimer, a dedicated server you manage for a customer, and located in rack 27 of your DC, has just crashed, and you'd like to know WTF has happened rather than just blindly reboot, but you're in the office 15 minutes away from the DC floor, and the customer's going to want that machine back up and running pretty quickly.
workstation:~/porn$ ssh rack27.serial
rack27:~$ sconsole faffenheimer
[screen session attached]
[minicom running, shows the horror of a kernel crash dump]
[oh look at that, kernel bug]
^A ESC
[pgup pgup]
[enter]
[pgdn pgdn]
[enter]
^A >/tmp/faffenheimer-crash-dump
^A d
rack27:~$ exit
workstation:~/porn$ scp rack27.serial:/tmp/faffenheimer-crash-dump ~
workstation:~/porn$ powercycle rack27 faffenheimer
Shiny! We got a crash dump in a minute or so (rather than having to take phonecam photos of KVM screens in the DC), never had to leave our comfy seat, and the machine's on it's way back up. We're now free to pursue diagnostic activities on that crash dump at our leisure. 10 minutes later, the downtime for faffenheimer that was automatically set when we ran powercycle runs out and Nagios sends us threatening messages. Hmm, something's gone wrong here. Back into the console...
workstation:~/porn$ ssh rack27.serial
rack27:~$ sconsole faffenheimer
[screen session attached]
[Boot is hung waiting for root password after initrd has bombed]
[Type root password]
[Oh look, the root MD appears to have come asunder]
[clickety-click... fixee fixee]
[reboot]
The more I think about this, the more I reckon I'm onto a bona fide winner. The sheevaplug is a powerful ARM-based system with USB/ethernet/SD ports that is packaged literally in it's own power supply wall wart -- it's a plastic box with power plug prongs poking out the side. That's all there is to it. The USB to serial adapter things are likely to be a bit more of a pain, but I've played with enough of them by now to not be too scared. So, you plug the Sheeva into a power socket, plug an Ethernet cable and USB hub into the Sheeva, configure things a bit so that the system knows which serial adapter maps to which machine, and you're away. Oh, and the best bit: the Sheeva apparently draws as little as 2W when idle. A whole datacentre's worth of serial goodness for about a server's worth of power. The cost per rack should be somewhere below AU$250, especially in bulk. Let's see if I can convince work to spring for a Sheeva, a USB hub, and a half dozen or so USB to serial adapters to test this whole thing out. Given that the whole thing looks like it'd cost less than AU$250 (plus my R&D time), I can't imagine it'll be too hard a sell to at least give it a go. Watch this space...

22 April 2009

Gunnar Wolf: The streets and the numbers of Cuernavaca

I usually use this blog to post about stuff I have written or that is somehow related to my work / professional life. This time, however, I'll just use it to share with you a short text my father published in the column he writes in the La Uni n de Morelos newspaper, Academia de Ciencias de Morelos: La Ciencia, desde Morelos para el mundo. My father has lived for over 20 years in Cuernavaca, la Ciudad de la Eterna Primavera (the city of the eternal Springtime), ~80Km south of Mexico City. Cuernavaca was for long years known mostly as a weekend city for Mexico City's middle-upper class, but it has grown way beyond that. According to Wikipedia, in Cuernavaca's metropolitan area there are about 700,000 inhabitants. And the city is blatantly built with no planning, no urbanistic analysis of any kind. I have been familiar for many years with the program to put some rational order renumber Cuernavaca's streets. Roberto Tamariz, the sociologist who built my father's house (yes, in a stroke of genius he acted as an architect for a piece of land he had bought - the results are quite decent, given his real occupation, but his lack of architectural background tends to literally come up from the floor every now and then), was involved in a municipal project to renumber Cuernavaca's streets, probably some 15 years ago. Anyway, Roberto's work was never really finished due to the severe lackings of our political culture. In this (one-and-a-third pages long, quite easy to read) text I am attaching to this post, my father writes about Cuernavaca's strange street naming system, the mathematical solutions (and political intrincacies) about renumbering a two dimensional space (and, of course, he could not help but wander into the third- and fourth-dimentional spaces, although very briefly). I like how this guy writes, all in all. Enjoy!
AttachmentSize
Calles y numeros de Cuernavaca.pdf231.99 KB

Matthew Palmer: Really, Really Distributed Revision Control

While I'm not a fan of cut-n-paste coding, on the odd occasion it's handy to grab a snippet of code out of someone else's blog and plop it in. Jeff Atwood of Coding Horror has a solution to some of the downsides of ripping a bit of code from somewhere on the Internet: publishers of code snippets generate a new GUID, tag each code snippet with a GUID, and you paste that comment in with the rest of the code when you take it. Then, if you (or someone else) needs to look at the context of the chunk of code, find it's author, look for improvements or commentary, or see who else has used that snippet in their own projects, you can search for that GUID and you should come up with only that GUID as a result.
What I propose is this:
// codesnippet:1c125546-b87c-49ff-8130-a24a3deda659
- (void)fadeOutWindow:(NSWindow*)window 
        // code
 
Attach a one line comment convention with a new GUID to any code snippet you publish on the web. This ties the snippet of code to its author and any subsequent clones. A trivial search for the code snippet GUID would identify every other copy of the snippet on the web:
http://www.google.com/search?q=1c125546-b87c-49ff-8130-a24a3deda659
How very, very cunning. He also proposes that if you modify a snippet and republish it, you should keep the old GUID and also add one of your own, so you can track the origin and help other people find your alternate version. Attaching unique identifiers to chunks of code and sending them all around the Internet. Does this sound like distributed revision control to anyone else? A lot easier to master than Git's UI, too...

Russell Coker: Vibration and Strange SATA Performance

Almost two years ago I blogged about a strange performance problem with SATA disks [1]. The problem was that certain regions of a disk gave poor linear read performance on some machines, but performed well on machines which appeared to be identical. I discovered what the problem was shortly after that but was prevented from disclosing the solution due to an SGI NDA. The fact that SGI now no longer exists as a separate company decreases my obligations under the NDA. The fact that the sysadmins of the University of Toronto published all the most important data entirely removes my obligations in this regard [2]. In their Wiki they write after SGI installed rubber grommits around the 5 or 6 tiny fans in the xe210 nodes, the read and write plots now look like and then some graphs showing good disk performance appear. The problem was that a certain brand and model of disk was particularly sensitive to vibrations. When that model of disk was installed in some machines then the vibrations would interfere with disk reads. It seems that there was some sort of harmonic frequency between the vibration of the disk and that of the cooling fans which explains why some sections of the disk were read slowly and some gave normal performance (my previous post has the graphs which show a pattern). Some other servers of the same make and model didn t have that problem, so it seemed that some slight manufacturing differences in the machines determined whether the vibration would affect the disk performance. One thing that I ve been meaning to do is to test the performance of disks while being vibrated. I was thinking of getting a large bass speaker, a powerful amplifier, and using the sound hardware in a PC to produce a range of frequencies. Then having the hard disk securely attached to a piece of plywood which would be in front of the speaker. But as I haven t had time to do this over the last couple of years it seems unlikely that I will do it any time soon. Hopefully this blog post will inspire someone to do such tests. One thing to note if you want to do this is that it s quite likely to damage the speaker, powerful bass sounds that are sustained can melt parts of the coil in a speaker. So buy a speaker second-hand. If someone in my region (Melbourne) wants to try this then I can donate some old IDE disks. I can offer advice on how to run the tests for anyone who is interested. Also it s worth considering that systems which make less noise might deliver better performance.

21 April 2009

Ingo Juergensmann: Finally finished!

This morning the build of the Parma Polyhedra Library (ppl) finished on my m68k buildd:

buildd@spice:~$ head logs/ppl_0.10-4_20090228-2019
Automatic build of ppl_0.10-4 on spice by sbuild/m68k 98
Build started at 20090228-2019
....
Finished at 20090421-0636
Build needed 1231:16:23, 1342752k disk space


Yes, the build needed 51 days 10:16:23 on that 68040/40, mostly because the build needed some amount of memory that exceeded the available 64 MB. Nevertheless I'm not the only one that wasn't very happy about the long build times of ppl as bug #517659 shows. It kept me away from shutting down the buildd for a long time to do the yearly clean up of the inner parts (removing the dust), but led to a rather high uptime as already reported. Now I can shutdown the machine for the clean up and attaching it to a UPS in the next days - after more than 510 days!

19 April 2009

Kees Cook: recording from PulseAudio

Every PulseAudio "Sink" has a "Source" named "monitor". This lets you attach to a given Sink and chain more stuff to it, for example, recording the audio that is playing through PulseAudio at any given moment. This is very handy for creating, for example, PubQuiz-style clips of songs, movies, etc. Here is a script to find the monitor for the most recently added Sink, record from it, and shove it through "sox" to get a WAV instead of raw sound data (requires recent sox, Pulse, etc):
#!/bin/bash
WAV="$1"
if [ -z "$WAV" ]; then
    echo "Usage: $0 OUTPUT.WAV" >&2
    exit 1
fi
rm -f "$WAV"
# Get sink monitor:
MONITOR=$(pactl list   grep -A1 '^\*\*\* Source #'   \
    grep '^Name: .*\.monitor$'   cut -d" " -f2   tail -n1)
# Record it raw, and convert to a wav
echo "Recording to $WAV ..."
echo "Close this window to stop"
parec -d "$MONITOR"   sox -t raw -r 44k -sLb 16 -c 2 - "$WAV"

Martin F. Krafft: Extending the X keyboard map with xkb

xmodmap has long been the only way to modify the keyboard map of the X server, short of the complex configuration daemon approaches used by the large desktop managers, like KDE and GNOME. But it has always been a hack: it modifies the X keyboard map and thus requires a baseline to work from, kind of like a patch needs the correct context to be applicable. Worse yet, xmodmap weirdness required me to invoke it twice to get the effect I wanted. When the recent upgrade to X.org 7.4 broke larger parts of my elaborate xmodmap configuration, I took the time to finally ditch xmodmap and implement my modifications as proper xkb configuration.

Background information I had tried before to use per-user xkb configuration, but could not find the answers I want. It was somewhat by chance that I found Doug Palmer s Unreliable Guide to XKB configuration at the same time that Julien Cristau and Matthew W. S. Bell provided me the necessary hints on the #xorg/irc.freenode.org IRC channel to get me started. The other resource worth mentioning is Ivan Pascal s collection of XKB documents, which were instrumental in my gaining an understanding of xkb. And just as I am writing this document, Debian s X Strike Force have published their Input Hotplug Guide, which is a nice complement to this very document you are reading right now, since it focuses on auto-configuration of xkb with HAL. The default xkb configuration comes with a lot of flexibility, and often you don t need anything else. But when you do, then this is how to do it:

Installing a new keyboard map The most basic way to install a new keyboard map is using xkbcomp, which can also be used to dump the currently installed map into a file. So, to get a bit of an idea of what we ll be dealing with, please run the following commands:
xkbcomp $DISPLAY xkb.dump
editor xkb.dump
xkbcomp xkb.dump $DISPLAY

The file is complex and large, and it completely went against my aesthetics to simply edit it to have xkb work according to my needs. I sought a way in which I could use as much as possible of the default configuration, and only place self-contained additional snippets in place to do the things I wanted done differently. setxkbmap and rule files Thus began my voyage into the domain of rule files. But before we dive into those, let s take a look at setxkbmap. Despite the trivial invocation of e.g. setxkbmap us to install a standard US-American keyboard map, the command also takes arguments. More specifically, it allows you to specify the following high-level parameters, which determine the sequence of events between key press and an application receiving a KeyPress event:
  • Model: the keyboard model, which defines which keys are where
  • Layout: the keyboard layout, which defines what the keys actually are
  • Variant: slight variantions in the layout
  • Options: configurable aspects of keyboard features and possibilities
Thus, with the following command line, I would select a US layout with international (dead) keys for my Thinkpad keyboard, and switch to an alternate symbol group with the windows keys (more on that later):
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch

In many cases, between all combinations of the aforementioned parameters, this is all you ever need. But I wanted more. If you append -print to the above command, it will print the keymap it would install, rather than installing it:
% setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch -print
xkb_keymap  
  xkb_keycodes    include "evdev+aliases(qwerty)"        ;
  xkb_types       include "complete"     ;
  xkb_compat      include "complete"     ;
  xkb_symbols     include "pc+us(intl)+inet(evdev)+group(win_switch)"    ;
  xkb_geometry    include "thinkpad(us)"         ;
 ;

There are two things to note:
  1. The -option grp:win_switch argument has been turned into an additional include group(win_switch) on the xkb_symbols line, just like the model, layout, and variant are responsible for other aspects in the output.
  2. The output seems related to what xkbcomp dumped into the xkb.dump file we created earlier. Upon closer inspection, it turns out that the dump file is simply a pre-processed version of the keyboard map, with include instructions exploded.
At this point, it became clear to me that this was the correct way forward, and I started to investigate those points in order. The translation from parameters to an xkb_keymap stanza by setxkbmap is actually governed by a rule file. A rule is nothing more than a set of criteria, and what setxkbmap should do in case they all match. On a Debian system, you can find this file in /usr/share/X11/xkb/rules/evdev, and /usr/share/X11/xkb/rules/evdev.lst is a listing of all available parameter values. The xkb_symbols include line in the above xkb_keymap output is the result of the following rules in the first file, which setxkbmap had matched (from top to bottom) and processed:
! model         layout              =       symbols
  [...]
  *             *                   =       pc+%l(%v)
! model                             =       symbols
  *                                 =       +inet(evdev)
! option                            =       symbols
  [...]
  grp:win_switch                    =       +group(win_switch)

It should now not be hard to deduce the xkb_symbols include line quoted above, starting from the setxkbmap command line. I ll reproduce both for you for convenience:
setxkbmap -model thinkpad -layout us -variant intl -option grp:win_switch
xkb_symbols     include "pc+us(intl)+inet(evdev)+group(win_switch)"    ;

A short note about the syntax here: group(win_switch) in the symbols column simply references the xkb_symbols stanza named win_switch in the symbols file group (/usr/share/X11/xkb/symbols/group). Thus, the rules file maps parameters to sets of snippets to include, and the output of setxkbmap applies those rules to create the xkb_keymap output, to be processed by xkbcomp (which setxkbmap invokes implicitly, unless the -print argument was given on invocation). It seems that for a criteria (option, model, layout, ) to be honoured, it has to appear in the corresponding listing file, evdev.lst in this case. There is also evdev.xml, but I couldn t figure out its role.

Attaching symbols to keys I ended up creating a symbols file of reasonable size, which I won t discuss here. Instead, let s solve the following two tasks for the purpose of this document:
  1. Make the Win-Hyphen key combination generate an en dash ( ), and Win-Shift-Hyphen an em dash ( ).
  2. Let the Caps Lock key generate Mod4, which can be used e.g. to control the window manager.
To approach these two tasks, let s create a symbols file in ~/.xkb/symbols/xkbtest and add two stanzas to it:
partial alphanumeric_keys
xkb_symbols "dashes"  
  key <AE11>  
    symbols[Group2] = [ endash, emdash ]
   ;
 ;
partial modifier_keys
xkb_symbols "caps_mod4"  
  replace key <CAPS>  
    [ VoidSymbol, VoidSymbol ]
   ;
  modifier_map Mod4   <CAPS>  ;
 ;

Now let me explain these in turn:
  1. We used the option grp:win_switch earlier, which told xkb that we would like to use the windows keys to switch to group 2. In the custom symbols file, we now simply define the symbols to be generated for each key, when the second group has been selected. Key <AE11> is the hyphen key. To find out the names of all the other keys on your keyboard, you can use the following command:
    xkbprint -label name $DISPLAY -   gv -orientation=seascape -
    
    
    I had to declare the stanza partial because it is not a complete keyboard map, but can only be used to augment/modify other maps. I also declared it alphanumeric_keys to tell xkb that I would be modifying alphanumeric keys inside it. If I also wanted to change modifier keys, I would also specify modifier_keys. The rest should be straight-forward. You can get the names of available symbols from keysymdef.h (/usr/include/X11/keysymdef.h on a Debian system, package x11proto-core-dev), stripping the XK_ prefix.
  2. The second stanza replaces the Caps Lock key definition and prevents it from generating symbols (VoidSymbol). The important aspect of the second stanza is the modifier_map instruction, which causes the key to generate the Mod4 modifier event, which I can later use to bind key combinations for my window manager (awesome).
The easiest way to verify those changes is to put the setxkbmap -print output of the keyboard map you would like to use as a baseline into ~/.xkb/keymap/xkbtest, and append snippets to be included to the xkb_symbols line, e.g.:
"pc+us(intl)+inet(evdev)+group(win_switch)+xkbtest(dashes)+xkbtest(caps_mod4)"

When you try to load this keyboard map with xkbcomp, it will fail because it cannot find the xkbtest symbol definition file. You have to let the tool know where to look, by appending a path to its search list (note the use of $HOME instead of ~, which the shell would not expand):
xkbcomp -I$HOME/.xkb ~/.xkb/keymap/xkbtest $DISPLAY

You can use xev to verify the results, or just type Win-Hyphen into a terminal; does it produce ? By the way, I found xev much more useful for such purposes when invoked as follows (thanks to Penny for the idea):
xev   sed -ne '/^KeyPress/,/^$/p'

Unfortunately, xev does not give any indication of which modifier symbols are generated. I have found no other way to verify the outcome, other than to tell my window manager to do something in response to e.g. Mod4-Enter, reloaded it, and then tried it out.

Rules again, and why I did not use them in the end Once I got this far, I proceeded to add option-to-symbol-snippet mappings to the rules file, and added each option to the listing file too. A few bugs [[!debbugs 524512 desc=later]], I finally had setxkbmap spit out the right xkb_keymap and could install the new keyboard map with xkbcomp, like so:
setxkbmap -I$HOME/.xkb [...] -print   xkbcomp -I$HOME/xkb - :0

I wrote a small script to automatically do that at the start of the X session and could have gone to play outside, if it hadn t been for the itch I felt due to the entire rule file stored in my configuration. I certainly did not like that, but I could also not find a way to extend a rule file with additional rules. When I looked at the aforementioned script again, it suddenly became obvious that I was going a far longer path than I had to. Even though the rule system is powerful and allows me to e.g. automatically include symbol maps to remap keys on my Thinkpad, based on the keyboard model I configured, the benefit (if any) did not justify the additional complexity. In the end, I simplified the script that loads the keyboard map, and defined a default xkb_keymap, as well as one for the Thinkpad, wich I identify by its fully-qualified hostname. If a specific file is available for a given host, it is used. Otherwise, the script uses the default.

17 April 2009

Matthew Garrett: Intel's commitment to open source driver support

Update: It turns out that the original PSB code is still on the moblin site - I just failed to notice that there's two pages of git repositories. I fail. Last update is from March 2008, though.

Edited again: An up-to-date version of the drm code is available as part of the Moblin kernel SRPM - repo.moblin.com isn't obviously linked off the Moblin website, but I should have thought to find and check their source repository.

By and large, working with Intel is a pleasure. In fact, a while ago I said so in a fairly obvious way. So it's a shame that the support for Poulsbo is still an absolute fucking mess with absolutely no obvious end in sight. Intel's Moblin group (the people who seem to be closest to having responsibility for providing any public Linux support for that hardware, though it's really not clear if they've got any more say in this than I do) have a kernel team that don't want or need a git tree despite having someone who's working on forward porting the driver to modern kernels - probably a job that requires more than one person given what a godawful fucking mess the last public release was, but it'd at least be nice to see what the current state is. Especially since you don't even seem to be able to get the last public release - moblin's git repositories have been rearranged and I can't find the psb-kmod one on their gitweb any more.

The most terribly depressing thing about this is that development is clearly still happening internally. Ubuntu is still being sent tarball releases by Intel, not that the code's getting any better judging by the type of diff involved. This is made even more entertaining by the bug in question being private, leaving no indication whatsoever about what bugs this code is meant to fix or why.

Closed development, random private tarball drops to partners without any public releases or changelogs, no indication of any upstream release, kernel developers entirely unrelated to the development of the driver trying to get code merged so people can actually use the hardware they bought? I'd expect this of some random far-east vendor with no experience in working with Linux, but not a company that's consistently one of the top ten contributors to the kernel and has frequently touted their level of support. There's no meaningful support for Poulsbo. There's just a thin cardboard cutout that's been carefully placed in front of a hole filled with users' hopes and money.

Seriously Intel. Sort it the fuck out.

12 April 2009

Benjamin Mako Hill: AttachCheck Revved

I finally got around to pushing out a new version of AttachCheck --- a trivial little program I wrote several years ago that tries to prevent people from having to send followup emails with subjects that include phrases like, "REALLY attached this time," by asking you for confirmation when you send an email that says you've attached something when it looks like you haven't. The release fixes a single bug that affected a few users --- thanks to Iain Murray who sent the patch in and apologies to him and others for taking a while to push it out. There's very little to AttachCheck and, if I remember correctly, it was the very first program I wrote in Python. I'm only mentioning this revision because it's been quite a few years since I last mentioned the program and, while the script doesn't do much, it continues to save me a little embarrassment and effort every other week or so.

10 April 2009

Jurij Smakov: Series of unfortunate events

While I did not think I would continue hacking on VoiDroid after achieving a semi-broken proof-of-concept implementation, it seems that I can t get out of it easily :-). I spent today trying to implement a solution which would allow the native C++ code to call into Java - for example, if we have a call in progress, and want to update the call status displayed to the user when it changes. The JNI library has some native callbacks which are getting called on call state changes, but getting the information to Java turned out to be a quite an experience, which, I thought, would be amusing to document, and, hopefully, save someone some quality bang-head-on-desk time. 1. Some of the threads calling native callbacks were initiated in Java (and thus known to JVM), and some where not, they were started in the native code. As JVM did not know anything about them, JNI calls would fail without too much explanation. Eventually I figured that out, and started attaching them as appropriate (JNI provides facilities for that). 2. Even after attaching, I could not resolve references to my classes from the native code, until I found a mysterious line in JNI docs: When a thread is attached to the VM, the context class loader is the bootstrap loader.. Basically, it means that when you attach a native thread, it is only capable of resolving built-in Java classes. 3. I later found a reference to a technique, suggesting to cache a good classloader from a Java-initiated thread, and later set the classloader for the attached thread to it. I implemented that, only to find out that Android s JVM is not your regular JVM, so things are slightly different here, and this technique does not work. 4. Luckily, there is an incredibly useful post describing the problem, and suggesting a workaround: caching an instance of needed class globally in native code, after calling NewGlobalRef on it, to avoid garbage collection on exit from a native function. I did that, however I only had access to the class, and Android JNI only supports calling NewGlobalRef on an object, so I had to create a dummy object in the native code (JNI_OnLoad() function, actually) and cache that. 5. That finally enabled me to call into Java from native, attached threads (w00t!). Unfortunately, you are not allowed to modify the UI (exactly what I wanted to do from my Java callback!) from a non-UI thread, which brought me right back to the drawing board. 6. As callbacks initiated from native code would not work for this reason, it was suggested to me to use a classic producer-consumer mechanism, getting Java call into the native library block until new information gets delivered by a native thread. Easily done with some pthread mutexes, and the goal is close, right? 7. Wrong! Now the problem was that if I started a UI Java thread, which would call the native function in a loop to retrieve the data (it would not actually spin, as the native function blocks until new data is available), then it would be declared not responding by the UI framework in a few seconds. I could start a non-UI thread for that, but you guessed it a non-UI thread is not allowed to update UI! Instead of resolving the problem we just shifted it into the Java land. 8. While I was simply thinking about stuffing the result into a class variable and passing it to the UI thread this way, this tutorial, showing how to display a progress indicator while doing some heavy calculations in the background, provided a more elegant idea. One can use Android s android.os.Handler class to pass messages (with arbitrary payload) between threads. That was the last piece of the puzzle, which allowed me to finally achieve a working implementation (some 10 hours or so later :-).

6 April 2009

Stefano Zacchiroli: ocaml 3.11 in testing

OCaml 3.11 has migrated to testing Quoting from Dato:
* St phane Glondu [Sat, 04 Apr 2009 14:01:35 +0200]:
> Adeodato Sim  a  crit :
> >> Please schedule the attached requests for the OCaml 3.11.0 transition.
> > Scheduled, with the glitches noted below. Please get back to us with the
> > needed wanna-build actions.
> All packages that needed recompilation or sourceful uploads for the
> OCaml 3.11.0 transition are now compiled and available in unstable.
> I guess migrating ocaml to testing can now be considered...
This is now done:
ocaml    3.11.0-5   testing
ocaml    3.11.0-5   unstable
Congratulations for making of this transition one of the less painful
I ve ever had to deal with, though I guess being a quite self-contained
set of packages and not having ties to other ongoing
transitions really
helped. ;-)
Thanks!,

IOW OCaml 3.11 has just migrated to Debian testing YAY \o/ Congrats and thanks to all the people who contributed. Special kudos go to the (not so) newbies of the Debian OCaml Task Force, and in particular to Stephane Glondu and Mehdi Dogguy: they have contributed work to a lot of packages and have also developed new tools which helped monitoring the transition effectively. Keep up the good work.

Next.

Previous.