(The much-delayed part 2 in an ongoing series: shiny things in telepathy-glib) Over a year ago I explained my first major round of work on telepathy-glib, with which it could be used to implement D-Bus services. I started to write this post shortly after, but then got distracted by a bee or something; recent discussion about having a D-Bus binding in GLib has finally prompted me to finish it. The second major round of work on telepathy-glib dealt with implementing client code, and went sufficiently well that telepathy-qt4 (currently under development) uses essentially the same model. Again, while telepathy-glib is intended for Telepathy implementors, I think TpProxy's ideas are generically useful, particularly for "large" D-Bus APIs.

Some background: dbus-glib and libtelepathy

The life-cycle of a DBusGProxy (PD clipart credit: karderio and Andy)

The API provided by dbus-glib for client code is, er, rather less than ideal. The basic object is DBusGProxy, which has a few problems. There's one per interface, rather than one per object; this leads to a maze of twisty GObjects, all suspiciously similar. Our old client library, libtelepathy, got round this by subclassing DBusGProxy to make helper objects called TpChan, TpConn etc., which were simultaneously the proxy for the "main" interface (Channel, Connection etc.), and a factory for proxies for the other interfaces (like Channel.Type.Text and Channel.Interface.Password). This still wasn't ideal, but it was a start. If you use a DBusGProxy in any way after it has emitted the destroy signal, this is considered to be invalid use of the API, and dbus-glib asserts all over the place. We'd rather it didn't do that - remote objects becoming invalid is a fact of life, and if you don't immediately discard all your references to those objects, remote operations on them should just give you a runtime error. You have to be prepared to handle such runtime errors already, because any IPC call can fail at any time. The main API for calling methods uses varargs and is rather subtle. Notably, the calling convention for variant parameters (of D-Bus signature 'v') is not consistent with the calling convention for any other container. For any other container, for instance a GPtrArray *, you pass in a GPtrArray ** pointing to a GPtrArray * variable containing NULL; on success, a pointer to a new GPtrArray is written into that variable. For variant parameters, though, you must pass in a GValue * pointing to a zero-filled GValue, and the result will be written into that GValue. dbus-binding-tool generates thin wrappers around this method-calling API which provide type-safety, although they can't directly specify a non-default timeout. As for signals, before binding to a signal, you have to tell dbus-glib about its signature. This tells dbus-glib how it should push the signal arguments onto the C stack (in GObject terminology: how it should marshal them), to match the signature of the signal-handler function. The documentation claims that this isn't necessary for objects supporting the Introspect method, but that's untrue - on the other hand, if the documented behaviour actually happened, I would consider this to be a critical bug in dbus-glib, since it would enable remote objects to crash dbus-glib clients (by causing signal arguments to be pushed onto the stack in a way that does not match the signature of the signal handler). Of course, the signal-adding function takes varargs, and has to be called exactly once per signal per object. If you don't, dbus-glib asserts when you connect to the signal; if you call it twice, dbus-glib asserts anyway.

TpProxy TpProxy is a GObject subclass vaguely inspired by Python's dbus.proxy.ProxyObject. It isn't a subclass of DBusGProxy, which lets us encapsulate dbus-glib almost completely - in fact, all the references to DBusGProxy are in a separate header, proxy-subclass.h, only used by subclasses of TpProxy (as opposed to normal library API users). TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. As explained below, I believe that this is the most natural object mapping for D-Bus interfaces; in practice it tends to lead to clearer code for us, since one remote object often maps nicely to one UI element (of course, if your objects only have one interface, this approach is no better or worse than dbus-glib's). TpProxy provides support for the following common D-Bus things:

• Calling methods
• Connecting to signals
• Objects becoming invalid
• Objects with optional interfaces
• Extensibility

It doesn't currently provide any particular help with Properties - the Properties interface is just another interface as far as we're concerned. We find that, in practice, that's not a problem (even though we use D-Bus properties extensively). In telepathy-qt4 we basically reinvented TpProxy in C++, as Tp::DBusProxy. I've put in some comments below illustrating how the general ideas we used in TpProxy translate into a different object mapping.

Calling methods Method calls on TpProxy are done through auto-generated wrapper functions, much like DBusGProxy. These look something like this:

typedef void (*tp_cli_channel_callback_for_close) (TpChannel *proxy,
    /* any 'out' arguments would go here - Close() doesn't have any */
    const GError *error, gpointer user_data,
    GObject *weak_object);
TpProxyPendingCall *tp_cli_channel_call_close (TpChannel *proxy,
    gint timeout_ms,
    /* any 'in' arguments would go here - Close() doesn't have any */
    tp_cli_channel_callback_for_close callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object);

Methods for some interfaces, like DBus.Properties, are available on TpProxy itself; others, like Telepathy's Channel.anything, are only available on a particular subclass, like TpChannel. Things to note here:

• It's an asynchronous call with a callback. General design principle: "this is IPC, get over it". We don't try to hide the fact that IPC is taking place by doing pseudo-blocking calls.
• The timeout is explicit (although you can always pass -1 to let libdbus pick a "reasonable" default for you). This is because the remote service might not respond, and API users should be at least vaguely aware of this: "this is IPC, get over it".
• The callback isn't a GCallback; it has an explicit signature, for some semblance of type-safety (it's not much, but it's better than nothing).
• The callback takes a GError, because any IPC call can fail for any reason.
• A TpProxyPendingCall * is returned. This is a pointer to an object that only exists until just after the callback is called (so you can ignore it if you don't want it), with a method you can call to cancel the method call (with hindsight this is not ideal, and if I was writing this API now, I'd use gio's GCancellable). "Cancel" is perhaps slightly misleading: the method call still takes place (it was already in libdbus's outgoing queue) but the result gets ignored and the callback isn't called. This is mostly so you an object can forget all the calls it was busy making at the time that the results become irrelevant (e.g. the object is destroyed).
• There is an extra weak_object argument, which is weakly referenced and is passed to the callback: in practice, most users of telepathy-glib use this as well as or instead of user_data. If the weak_object runs out of references, the callback is automatically cancelled, which means that in practice, explicit cancellation is almost never needed.
• The generated code has as its first argument a suitable subclass of TpProxy, in this case TpChannel (we tell the code generator about subclasses).
• You can't see it in the API, but the TpProxy gains an extra ref while a method call is in progress, so you never have to worry about what happens if the proxy gets unreffed during a method call.
• There's an explicit destructor for the user_data, which is called even if you "cancel" the method call.

In telepathy-qt4, these methods are on small generated helper classes similar to dbus-python's dbus.Interface, one per interface (we provide accessors for them on the non-generated DBusProxy subclass). Instead of using callbacks, we return a temporary QObject per call, which emits a Qt signal on success or failure (that's how all async calls in QtDBus work).

Connecting to signals Similarly, signals have some generated functions:

typedef void (*tp_cli_channel_signal_callback_closed) (TpChannel *proxy,
    /* any arguments would go here - Closed doesn't have any */
    gpointer user_data, GObject *weak_object);
TpProxySignalConnection *tp_cli_channel_connect_to_closed (TpChannel *proxy,
    tp_cli_channel_signal_callback_closed callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object,
    GError **error);

API notes for these:

• The callback doesn't take a GError any more, because signals can't fail; the weak_object and the user_data are the same as for method calls
• This isn't actually a GObject signal at all: those aren't particularly typesafe, and lack namespacing. This might not be very binding-friendly, we haven't tried doing that yet... it might be worth introducing a signal with details, like in dbus-glib, for which these functions are "C bindings".
• The returned TpProxySignalConnection lets you disconnect from the signal, just like the TpProxyPendingCall above; it can safely be ignored
• Similarly, if the weak_object dies, then the signal automatically disconnects
• It is possible for connecting to a signal to fail: this happens if the proxy doesn't actually have the requested interface. If this happens, it's graceful, not a crash. In practice, the flow of code is almost always such that error can safely be NULL because the interfaces are already known.

In telepathy-qt4 the per-interface generated helper classes emit Qt signals. In many cases, we end up proxying these signals by responding to them in the hand-written Tp::DBusProxy subclass by emitting a different signal, in order to give them a nicer representation.

Becoming invalid This is IPC, and any error can happen to you at any time. We found that it was useful to have a general concept of "this object is no longer useful", so we introduced the concept of an object becoming invalidated. The invalidation reason is stored on the object as a GError. If this GError is NULL (as it is initially), then the object is still expected to be valid; if it's non-NULL, the remote object has vanished and the local proxy no longer works. There are several ways a TpProxy can become invalid:

• its bus name falls off the bus (the process exited or crashed)
• application-specific reasons (usually triggered by a D-Bus signal in a subclass, like the Telepathy Channel's Closed signal)
• the GObject gets disposed

Method calls on an invalidated object always fail, with the invalidation reason as the error. This means that calling a method on a TpProxy should never crash your process. The method's callback has to be prepared to handle an error in any case, because this is Sparta^WIPC, so there's no loss in giving it another error. Connecting to signals on an invalidated object always fails, and any existing signal connections are disconnected when it becomes invalidated. This means that if a new remote object appears, and it happens to have the same object path, bus name etc. as the old one, you won't get its signals. A TpProxy can either be bound to a unique bus name or a well-known bus name. In the equivalent code in telepathy-qt4, we call this "stateful" or "stateless" - these names don't capture the intention perfectly, but they're the best we could come up with. A proxy for a stateful API like Telepathy's Channel should always bind to a unique name - when the unique name falls off the bus, the TpProxy becomes invalidated automatically, representing the fact that the Channel you had no longer exists, and nothing can be a perfect replacement for it (the VoIP call has already been terminated, you've already left the chatroom, or whatever - if you make another Channel, that's e.g. a distinct VoIP call). A proxy for a stateless API like Telepathy's ConnectionManager, where an exiting process can disappear, be service-activated again, and still have exactly the same API, should bind to the well-known name. Proxies for well-known names aren't invalidated when the process exits. Qt doesn't have a natural equivalent for GError, so the invalidation reasons in telepathy-qt4 are just a pair of strings, the namespaced name and the message - basically a libdbus DBusError. The principle is the same, though.

Optional interfaces As mentioned above, TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. I believe that this is the most natural object mapping for the D-Bus object model, because the D-Bus interfaces on a remote object can behave like any of these:

• classes and subclasses (Telepathy.Channel.Type.Text is a Telepathy.Channel)
• shared interfaces (some Telepathy.Channel objects, of many Types, implement Telepathy.Channel.Interface.Group; any D-Bus object can implement DBus.Properties)
• optional features (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.ChatStates, some do not)
• discoverable extensions (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.Messages, which is more or less Text 2.0)

The way in which interfaces are discovered is also variable. The "classic" D-Bus way to discover interfaces would be to call Introspect. Telepathy services still support that style of introspection, but we don't use it for anything beyond d-feet:

• Introspect returns a blob of XML, which you have to parse into something useful; if it was in a D-Bus data structure, you'd already have some sensible data structure.
• Finding out more information about the supported interfaces than their names isn't very useful: in a static language like C you already need to know the methods, signals, properties and their signatures in order to write and compile your code, and in a dynamic language like Python consumers of Introspect end up being remotely crashable by services with unexpected method signatures (this is one of dbus-python's big mistakes in my opinion).
• If a method that you want to call turns out to be missing, the worst that can happen (in a competently written service) is that it fails with a D-Bus error - and you have to handle D-Bus errors anyway, because this is IPC and anything could fail.
• Static bindings like dbus-glib have poor support for removing interfaces for which C code exists at runtime, whereas Telepathy needs features like an IM connection that might or might not support avatars (we don't get to find out until we've successfully connected to the server).

Instead, our older interfaces have a method on the "base class" (Channel, Connection etc.) called GetInterfaces, which just returns an array of strings. Newer interfaces (like Account), and older interfaces that have been ported (like Channel), have a property called Interfaces which is, again, an array of strings - this lets us combine the download of the interfaces list with downloading other basic information in a GetAll call. To cater for all these ways to use interfaces, the TpProxy base class has very basic support for interface discovery - you can ask it whether it supports an interface, and you can tell it that it does, in fact, support an interface. Asking which interfaces are supported is directly useful for library users; it's also used as a check by the generated method-call and signal-connection stubs, which return a canned error (Telepathy.Error.NotImplemented in Telepathy's case) if the object isn't known to have the interface. Telling TpProxy that it does support an interface is intended for use by subclasses, and might have been protected if we were writing C++: what happens in practice is that a concrete subclass like TpChannel knows how to discover the fully supported interfaces for this particular object, does so, and calls methods on the TpProxy to tell it which interfaces can work.

Extensibility The API stubs used by TpProxy to call methods and bind to signals are generated from XML documents containing an augmented form of D-Bus introspection data. These XML documents contain various Telepathy-specific extensions, but none of the extensions are language-specific - telepathy-glib, telepathy-python and telepathy-qt4 all operate from exactly the same XML specification. telepathy-qt4's different object-mapping did require us to tighten up some rules that were previously only conventions, but even so, the format is the same. I think this is vitally important - it just doesn't scale to have one set of language-specific annotations in a D-Bus API for each language that will have bindings. One thing that we make heavy use of is what I call "Ugly_Case", which is camel-case with underscores at word boundaries. This gives us a simple, unambiguous and foolproof rule to use in code generators whenever we generate any of the popular conventions for identifiers:

• CamelCase: delete underscores
• javaCamelCase: force everything before the first underscore to lower case, then delete underscores
• lower_case: force everything to lower case, retain underscores
• UPPER_CASE: force everything to upper case, retain underscores

dbus-glib, by contrast, applies a complex and subtly buggy algorithm to the CamelCase D-Bus names, which results in our SendDTMF method being mapped into GLib as send_dt_mf, and requires that telepathy-glib's Python reimplementation of dbus-binding-tool uses a bug-for-bug compatible implementation (we ended up reimplementing dbus-binding-tool in order to generate the TpProxy-based bindings). Our code generation tools are still rather ad-hoc, because so is our spec format, but for draft interfaces it's possible to copy them into individual projects and use them to generate additional methods for TpProxy or any subclass. This is how we implement unfinished APIs like geolocation in Empathy, for instance. The Interfaces or GetInterfaces hook described above is entirely usable for these extension interfaces, and in fact that's how they're set up. In telepathy-qt4, the generated method stubs are genuine C++ methods, so we can't just append methods to an existing class: this is why we have one helper class per interface, so individual clients can generate a helper class for their particular version of an unfinished API, and instantiate an object of this class attached to a particular Tp::DBusProxy. When an interface becomes stable, it can be added to the set of interfaces for which code is generated in telepathy-glib or telepathy-qt4, at which point any client or service that was already using the final draft of that interface can easily be ported to the library version using sed (the API remains the same).

Becoming ready The concept of being "ready" does not directly exist in TpProxy, but is implemented in TpChannel, TpConnection and TpConnectionManager. The idea is that a freshly created proxy object isn't really fully usable yet - you have to connect to basic signals and recover the initial state of the remote object, as well as checking which extension interfaces are supported. In many well-designed D-Bus APIs you can do this initial setup with a few signal connections and a DBus.Properties.GetAll call. After doing that initial setup (which has to be done asynchronously, because it's IPC), the TpProxy subclass has a local cache of the remote object's state (not necessarily in the same format as the representation on the bus), which can be accessed synchronously at any time, and will be updated in response to change-notification signals. TpChannel and subclasses like it have explicit support for checking for readiness, and having a callback called when the object is ready or invalidated (whichever happens first). The "ready" property also supports the GObject notify signal. telepathy-qt4 took this concept and ran with it - there is library support for objects that can become ready, including a mixin. If I had as much time to work on telepathy-glib as I wish I had, this would be one of the first telepathy-qt4 features to be added to telepathy-glib.

Features becoming ready Going beyond that, in an extensible framework like Telepathy it's probable that not every client understands (or wants to understand) every feature of every object, so it's highly inefficient for every proxy object to subscribe to all the change notification signals and cache all the remote state on the off-chance that the library user wants them. In the API for TpContact (which is not actually a TpProxy for various reasons) we introduced the concept of optional features; telepathy-qt4 extended this idea throughout the library. The idea is that each library user knows which of the optional features are "interesting" to it, and makes a single asynchronous call (which expands into a series of D-Bus calls inside the library) to download the state for core functionality plus all of the selected features, and subscribe to change notification for all of those. This is another telepathy-qt4 thing that telepathy-glib still needs to catch up with; until someone works out how to make the clone() syscall apply to programmers, I fear all libraries are doomed to lag behind how their designers want them to look :-) (license: CC-BY-3.0 / LGPL-2.1+)

For those who haven't heard, Dreamwidth is a new journal hosting site using the LiveJournal code base (with improvements) but a different business model and general attitude towards their community. They're trying to be much more transparent and open about everything they're doing, they're trying to run the business so that it's focused on building a community than making a profit, and they're trying to do much better at being an active open source project than LiveJournal has managed. They also run all of their servers on Ubuntu, so they feel like part of the broader Debian community. I haven't had a lot of time to help, but I think their ideals look great and I really would like to see them succeed. Dreamwidth just launched open beta, which means you can purchase a paid account but they're using an invite code model for free accounts. Since they just launched, I have invite codes. I think one of the things that will make Dreamwidth successful is to have a broad-based community, and I'd also like to see Debian and Ubuntu folks involved in their community, in a position to help with improvements to infrastructure packages they use, and help with the open source development of the underlying code base. I've therefore got a few invite codes reserved for Debian and Ubuntu folks on basically a first-come, first-served basis. If you're active in Debian or Ubuntu, want to try Dreamwidth, and think there's a chance you'd actively use it, send me e-mail and I'll send you an invite. For more information about Dreamwidth, see their wiki.

What seems like eons ago, I ordered one of those SheevaPlug Development Kits. After about six weeks, they got around to shipping it, and then, yesterday, after I assume a tortoise had piloted it across the country on broken tricycle, it was finally delivered to me. I booted it up, observed that there was some kind of Ubuntu thing on it, and set myself to correct that problem. Within an hour I had managed to lock myself out. Here is something I should have read beforehand. Thanks to Martin Michlmayr, it is now running Debian and allowing me to log in. Here are some steps to follow if you would like to boot Debian off of a USB stick plugged into your Sheevaplug:

• Grab the tarball from http://people.debian.org/~tbm/sheevaplug/lenny/
• Make an ext3 filesystem on the first partition of your USB stick (ext2 will not work with the current image)
• Mount the USB stick's filesystem, unpack the tarball onto it, then unmount
• Plug the USB stick into the SheevaPlug's full-sized USB port
• Plug the SheevaPlug in, and use minicom or your favorite terminal program to console in over the mini-USB port (don't forget that you need to use the ftdi_sio module instead of usbserial; 115200,N,8,1, no flow control)
• Abort the autoboot, and type the following commands:

  setenv mainlineLinux yes
  setenv arcNumber 2097
  saveenv
  reset
  setenv bootargs_root 'root=/dev/sda1 rootdelay=10'
  setenv bootcmd_usb 'usb start; ext2load usb 0:1 0x0800000 /boot/uInitrd; ext2load usb 0:1 0x400000 /boot/uImage'
  setenv bootcmd 'setenv bootargs $(console) $(bootargs_root); run bootcmd_usb; bootm 0x400000 0x0800000'
  saveenv
  run bootcmd
  

That should be all there is to it. Next I'm wondering if SDIO wireless cards work.

Now when there is already some content, it's time to announce planet phpMyAdmin. You will find there posts from phpMyAdmin contributors including Google Summer of Code students, so you might find there what interesting ideas they have. Please note that this blog does not have to contain only phpMyAdmin related posts, it's just for all what contributors do (as usual on other planets). It currently lives on http://pmaplanet.cihar.com/. PS: If you think your blog is missing there, please drop me a note.

Call for papers: JITP-2010
The Politics of Open Source
May 6-7, 2010 - Amherst, Massachusetts Full Paper Submission Due Date: January 10, 2010 Approach
Free/Libre and Open Source Software (FOSS) has made significant advances, both technically and organizationally, since its emergence in the mid-1980s. Over the last decade, it has moved from a software development approach involving mostly volunteers to a more complex ecology where firms, nonprofit organizations, government agencies and volunteers may be involved. Moreover, the production paradigm continues to expand to other areas of digital content (e.g., Creative Commons, Wikipedia, Connexions, etc.). In this conference we use the phrase open source to capture this broader phenomenon. The Program Committee encourages disciplinary and interdisciplinary approaches to the study of open source, broadly defined. Politics in the conference title, can have many interpretations. Political issues closely tied to the free and open source software movement(s) include: national government policies related to the adoption of open source technologies or questions related to interoperability and open standards, software patents, vendor lock-in, and copyright. These are central themes we expect may be discussed in this forum. In this context, we welcome international submissions since differences in the political perspective appear in international contexts. However, topics related to how the concept of openness has led to various interpretations, adaptations, and applications of open source in other domains, and political issues that surround these broader innovations, are also welcome. Specific topics might include, but are not limited to: General topics related to the politics of open source:
+ How open source software or its principles are changing politics
+ Emerging transparencies in software, systems and society
+ Open source in the developing world and other international contexts
+ The political economy of open source
+ Digital divides and open source Open source and the public sector:
+ Open source software and transparency in government
+ Government policies toward open source and open standards
+ Regulation and open source Open source and democracy:
+ Open source and democratic engagement
+ Open source voting systems
+ Activism, political mobilization and open source The expansion of open source into other domains:
+ Understanding how open source collaboration works and how it can be extended into other areas of collective action
+ Policy areas, such as the effects of free textbooks on education policy or the politics of One Laptop Per Child
+ The political implications of open source in other cultural domains Keynote Speakers
We are pleased to confirm Eric von Hippel (MIT) and Clay Johnson (Sunlight Labs) as the daily keynote speakers for JITP-2010. Paper Submissions
Authors are invited to prepare and submit to JITP a manuscript following one of the six submission formats by January 10, 2010. These formats include research papers, policy viewpoints, workbench notes, review essays, book reviews, and papers on teaching innovation. The goal is to produce a special issue, or double issue, of JITP with a wide variety of approaches to the broad theme of The Politics of Open Source.

How to Submit
Everything you need to know about how to prepare and submit a strong JITP paper is documented at http://www.jitp.net/. Papers will be put through an expedited blind peer review process by the Program Committee, and authors will be notified about a decision by March 10, 2010. A small number of papers will be accepted for presentation at the conference. Other paper authors will be invited to present a poster during the Friday evening reception. All posters must include a YouTube version of their research findings. Best Paper and Poster Cash Prizes
The author (or authors) of the best research paper will receive a single $1,000 prize. The creator (or creators) of the best poster/research presentation will also receive a single prize of $1,000. Program Committee
M.V. Lee Badgett, University of Massachusetts Amherst
Paul M.A. Baker, Georgia Institute of Technology
Deborah Bryant, Oregon State University Open Source Lab
Andrea Calderaro, European University Institute
Mark Cassell, Kent State University
Edward Cherlin, Earth Treasury
Gabriella Coleman, New York University
Doug Downham, University of Massachusetts Amherst
Robert English, Daystar Computing & University of Massachusetts Amherst
Joseph Feller, University College Cork
Jelena Karanovic, Rutgers University
Dave Karpf, University of Pennsylvania/Miller Center for Public Affairs
Jeremy Hunsinger, Virginia Tech
Andrea Kavanaugh, Virginia Tech
Jose Marichal, California Lutheran University
Jens Hardings Perl, Pontificia Universidad Cat lica de Chile
Charlie Schweik, University of Massachusetts Amherst, co-chair
Stuart Shulman, University of Massachusetts Amherst, co-chair
Megan Squire, Elon University
Krishna Ravi Srinivas, Research Information System For Developing Nations
Louis Suarez-Potts, Sun Microsystems, Inc. & OpenOffice.org
Anas Tawileh, Cardiff University & Meedan.net

Call for papers: JITP-2010
The Politics of Open Source
May 6-7, 2010 - Amherst, Massachusetts Full Paper Submission Due Date: January 10, 2010 Approach
Free/Libre and Open Source Software (FOSS) has made significant advances, both technically and organizationally, since its emergence in the mid-1980s. Over the last decade, it has moved from a software development approach involving mostly volunteers to a more complex ecology where firms, nonprofit organizations, government agencies and volunteers may be involved. Moreover, the production paradigm continues to expand to other areas of digital content (e.g., Creative Commons, Wikipedia, Connexions, etc.). In this conference we use the phrase open source to capture this broader phenomenon. The Program Committee encourages disciplinary and interdisciplinary approaches to the study of open source, broadly defined. Politics in the conference title, can have many interpretations. Political issues closely tied to the free and open source software movement(s) include: national government policies related to the adoption of open source technologies or questions related to interoperability and open standards, software patents, vendor lock-in, and copyright. These are central themes we expect may be discussed in this forum. In this context, we welcome international submissions since differences in the political perspective appear in international contexts. However, topics related to how the concept of openness has led to various interpretations, adaptations, and applications of open source in other domains, and political issues that surround these broader innovations, are also welcome. Specific topics might include, but are not limited to: General topics related to the politics of open source:
+ How open source software or its principles are changing politics
+ Emerging transparencies in software, systems and society
+ Open source in the developing world and other international contexts
+ The political economy of open source
+ Digital divides and open source Open source and the public sector:
+ Open source software and transparency in government
+ Government policies toward open source and open standards
+ Regulation and open source Open source and democracy:
+ Open source and democratic engagement
+ Open source voting systems
+ Activism, political mobilization and open source The expansion of open source into other domains:
+ Understanding how open source collaboration works and how it can be extended into other areas of collective action
+ Policy areas, such as the effects of free textbooks on education policy or the politics of One Laptop Per Child
+ The political implications of open source in other cultural domains Keynote Speakers
We are pleased to confirm Eric von Hippel (MIT) and Clay Johnson (Sunlight Labs) as the daily keynote speakers for JITP-2010. Paper Submissions
Authors are invited to prepare and submit to JITP a manuscript following one of the six submission formats by January 10, 2010. These formats include research papers, policy viewpoints, workbench notes, review essays, book reviews, and papers on teaching innovation. The goal is to produce a special issue, or double issue, of JITP with a wide variety of approaches to the broad theme of The Politics of Open Source.

How to Submit
Everything you need to know about how to prepare and submit a strong JITP paper is documented at http://www.jitp.net/. Papers will be put through an expedited blind peer review process by the Program Committee, and authors will be notified about a decision by March 10, 2010. A small number of papers will be accepted for presentation at the conference. Other paper authors will be invited to present a poster during the Friday evening reception. All posters must include a YouTube version of their research findings. Best Paper and Poster Cash Prizes
The author (or authors) of the best research paper will receive a single $1,000 prize. The creator (or creators) of the best poster/research presentation will also receive a single prize of $1,000. Program Committee
M.V. Lee Badgett, University of Massachusetts Amherst
Paul M.A. Baker, Georgia Institute of Technology
Deborah Bryant, Oregon State University Open Source Lab
Andrea Calderaro, European University Institute
Mark Cassell, Kent State University
Edward Cherlin, Earth Treasury
Gabriella Coleman, New York University
Doug Downham, University of Massachusetts Amherst
Robert English, Daystar Computing & University of Massachusetts Amherst
Joseph Feller, University College Cork
Jelena Karanovic, Rutgers University
Dave Karpf, University of Pennsylvania/Miller Center for Public Affairs
Jeremy Hunsinger, Virginia Tech
Andrea Kavanaugh, Virginia Tech
Jose Marichal, California Lutheran University
Jens Hardings Perl, Pontificia Universidad Cat lica de Chile
Charlie Schweik, University of Massachusetts Amherst, co-chair
Stuart Shulman, University of Massachusetts Amherst, co-chair
Megan Squire, Elon University
Krishna Ravi Srinivas, Research Information System For Developing Nations
Louis Suarez-Potts, Sun Microsystems, Inc. & OpenOffice.org
Anas Tawileh, Cardiff University & Meedan.net

I know there are DD s and other FLOSS folks in Madrid, so don t miss Chris Kelty s talk on Free Software and Open Biology on May 5th!

I know there are DD s and other FLOSS folks in Madrid, so don t miss Chris Kelty s talk on Free Software and Open Biology on May 5th!

previously i threw together a small hook to make it a bit easier to avoid duplicate work while maintaining packages, as well as easily keep the BTS up to date with relevant information. now i've commonized the code just a bit and have a second hook which can be used to maintain certian (what i believe to be) good practices for keeping packages in git. admittedly, it's a bit more "stick" than "carrot" with respect to streamlining workflows, but i feel the justifications and the resulting benefits are worth it. so the hook does basically two things, either of which can be customized and/or disabled. prevent "non-debian" changes on a "debian" branch assuming that there are seperate branches for "debian" packaging and for "upstream" development, this hook prevents upstream-style changes on the debian branches. that is to say changes to files outside ./debian are not permitted on a "debian" branch, unless of course you're merging from an upstream branch. instead, changes to the source in a debian branch should be managed by quilt-style "feature patches", or a more advanced branch topology using some kind of feature branches (topgit or similar). prevent changelog modifications from being mixed in with other commits this one might be a bit more controversial for some. basically, the idea is that:

• stuff is fixed in a commit
• there is a log message for this commit, which contains a description of the fix
• the changelog update is redundant information to this log
• it's impossible to merge/cherry-pick/revert the fix later if the changelog gets tangled into the commit.
• there are nice tools (git dch) for managing debian/changelog updates anyway.

therefore, this hook "declines" commits which modify debian/changelog, unless it is the only file being changed. using this new hook note this is the same repo as the other hook, so if you already have that set up you can skip the clone and instead just pull in the changes. to set it up:

REPO_PATH=/path/to/your/repo.git
HOOK_REPO_PATH=/somewhere/you/want/to/put/it
git clone git://git.debian.org/users/seanius/vcs-hooks/git-hooks.git $HOOK_REPO_PATH
ln -sf $HOOK_REPO_PATH/debian/git-hooks/pre-receive-fileset-fascism.py $REPO_PATH/hooks/pre-receive

the config options for controlling this hook (these are git config options just like the other hook):

# hooks.debianbranches (default: 'debian-.*')
#    a regular expression which indicates which branches are "debian" branches.
#    in the context of this hook such branches are not allowed to have changes
#    in files outside of the ./debian directory.
# hooks.sacredchangelog (default: True)
#    if set to True, debian/changelog can not be changed in a commit that
#    also modifies other files.  this helps ensure changes that are easily
#    merged/cherry-picked/reverted.

examples of the hook in action rejecting a commit that has an entangled changelog:

rangda[~/debian/php] git push                                  :)
Counting objects: 9, done.
Delta compression using 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 442 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
fileset-fascism rejecting commit 2fab3269b3d9daedd7013a576483a11ccb4cb86a
    debian/changelog must be changed seperately from other files.
    changed files in this commit:
        debian/changelog
        debian/control
error: hooks/pre-receive exited with error code 1
To ssh://git.debian.org/git/pkg-php/php.git
 ! [remote rejected] debian-sid -> debian-sid (pre-receive hook declined)
error: failed to push some refs to 'ssh://git.debian.org/git/pkg-php/php.git'

rejecting a commit that has non ./debian changes:

rangda[~/debian/php] git push                                  :)
Counting objects: 5, done.
Delta compression using 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 309 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
fileset-fascism rejecting commit d96669c0780e33b250af81404e263fc181fe0547
    non ./debian changes on a debian branch in this commit.
error: hooks/pre-receive exited with error code 1
To ssh://git.debian.org/git/pkg-php/php.git
 ! [remote rejected] debian-sid -> debian-sid (pre-receive hook declined)
error: failed to push some refs to 'ssh://git.debian.org/git/pkg-php/php.git'

as always, comments/feedback/suggestions/etc welcome

Here are the Winter Camp interviews in Ogg format

Here are the Winter Camp interviews in Ogg format

I have a little job that runs on my web server and tells me what URIs people have tried to fetch over the last 24 hours which were not successful. One of these URIs was for a page that used to contain my first ever Linux device driver. That device driver is now obsolete and so there was little point in tracking down the code and putting it back in place, but I did think there was a story to tell, so today I wrote down what happened with this driver at the URI. Thanks to the magic of ikiwiki's "inline" directive, here it is in the log: This is a story about my first real attempt to contribute to the Linux kernel. The Davicom DM9601 is a cheap USB NIC that I picked up on ebay at some point in late 2006. Suprisingly, at the time, there was no driver for it in the Linux kernel. I'd had some experience hacking on the kernel whilst working for IBM and I've always wanted to get more involved, so I thought this would be a good opportunity. I searched around and managed to find a vendor-written driver for the 2.4.x kernel series, but nothing newer. I felt that the best approach would be to start porting this driver to 2.6, improving it along the way. I got to work, and after several hours I had something that would compile against 2.6 series headers, at least, so I put my progress online and wrote a short log post about it (dm9601). A really invaluable resource whilst doing this was LWN.net's "Porting device drivers to the 2.6 kernel". Once I had the code up there, kernel hacker Arnd Bergmann got in touch with me. He said, quite frankly, that my driver had a long way to go, but that he would be willing to help mentor me and guide me in improving it so that it would make it into mainline. This was fantastic news: this is exactly what people need when trying to get up to speed with the customs of something as complex as the kernel development community. Arnd sent me a long list of things I needed to fix. Whilst I had my code online I had several other people get in touch with me and provide things like device IDs for their NICs that they needed to add in order to get it to work for their hardware. Several weeks later, after implementing a good chunk of Arnd's suggestions, I had a driver that worked, at least a little bit, with 2.6, so I got back in touch with Arnd. Bad news, for me at least: Peter Korsgaard, another kernel hacker, had implemented a dm9601 driver from scratch over a weekend. Apparently he was at a friend's house, his friend had this device, he noticed there wasn't a driver in the kernel already and sat down to write it as a fun project. From scratch. In one weekend. This was a bit of a blow. I couldn't imagine having the skill and knowledge required to be able to pump out a driver like that, but I knew the first step on the road was what I was doing, and that road was now closed. I couldn't really blame Peter, I could only really feel quite jealous at his prowess, so that was the end of that. His driver was much leaner, much tidier, fitted the kernel programming style properly, and was thus accepted. However, for my hardware at least, it didn't actually work, and he didn't have access to the device after that one weekend, so he wasn't in a position to fix it. This situation lasted about 4 months from 2007-02-23, when Peter's driver was accepted, until about 2007-06-27 with this patch. From this whole experience, I did manage to get one small patch into the kernel: one which added the device IDs for my NIC and those of the people who had got in touch with me.

This is a continuation from before. I am digressing a little in this post. One of the things I want to get out of this exercise is to learn more about Ontologies and Ontology editors, and on the principle that you can never learn something unless you build something with it (aka bone knowledge), so this is gathering my thoughts to get started on creating an Ontology for package building. Perhaps this has been done before, and better, but I ll probably learn more trying to create my own. Also, I am playing around with code, an odd melange of my package building porcelain, and gitpkg, and other ideas bruited on IRC, and I don t want to blog about something that would be embarrassing in the long run if some of the concepts I have milling around turn out to not meet the challenge of first contact with reality. I want to create a ontology related to packaging software. It should be general enough to cater to the needs any packaging effort in a distribution agnostic and version control agnostic manner. It should enable us to talk about packaging schemes and mechanisms, compare different methods, and perhaps to work towards a common interchange mechanism good enough for people to share the efforts spent in packaging software. The ontology should be able to describe common practices in packaging, concepts of upstream sources, versioning, commits, package versions, and other meta-data related to packages. I am doing this ontology primarily for myself, but I hope this might be useful for other folks involved in packaging software. So, here follow a set of concepts related to packaging software, people who like pretty pictures can click on the thumbnail on the right:

• software is a general term used to describe a collection of computer programs, procedures and documentation that perform some tasks on a computer system.
• software is what we are trying to package
• software has names
• software may exist as
  • source code
  • executable code
  • packaged code
• source code is any collection of statements or declarations written in some human-readable computer programming language.
• source code is usually held in one or more text files (blobs).
• A large collection of source code files may be organized into a directory tree, in which case it may also be known as a source tree.
• The source code may be converted into an executable format by a compiler, or executed on the fly from the human readable form with the aid of an interpreter.
• executable format is the form software must be in in order to be run. Running means to cause a computer to perform indicated tasks according to encoded instructions.
• software source code has one or more lines of development. Some Common specific lines of development for the software to be packaged are:
  • upstream line of development
  • feature branch is a line of development related to a new feature under development. Often the goal is to merge the feature branches into the upstream line of development
  • usually, all feature branches are merged into the integration branch, and the package is created from the integration branch.
  • integration branch is the line of development of software that is to be packaged
• some software lines of development have releases
• releases have release dates
• some releases have release versions
• source code may be stored in a version control repository, and maintain history.
• Trees are a collection of blobs and other trees (directories and sub-directories). A tree object describes the state of a directory hierarchy at a particular given time.
• Blobs are simply chunks of binary data - they are the contents of files.
• a tree can be converted into an archive and back
• In git, directories are represented by tree object. They refer to blobs that have the contents of files (file name, access mode, etc is all stored in the tree), and to other trees for sub-directories.
• Commits (or changesets ) mark points in the history of a line of development, and references to parent commits.
• A commit refers to a tree that represents the state of the files at the time of the commit.
• HEAD is the most recent commit in a line of development or branch.
• A working directory is a directory that corresponds, but might not be identical, to a commit in the version control repository
• Commits from the version control system can be checked out into the working directory
• uncommitted changes are changes in the working directory that make it different from the corresponding commit. Some call the working directory to be in a dirty state.
• uncommited changes be checked in into the version control system, creating a new commit
• The working directory may contain a ignore file
• ignore file contains the names of files in the working directory that should be ignored by the version control system.
• In git, a commit may also contains references to parent commits.
  • If there is more than one parent commit, then the commit is a merge
  • If there are no parent commits, it is an initial commit
• references, or heads, or branches, are movable references to a commit. On a fresh commit, the head or branch reference is moved to the new commit.
• lines of development are usually stored as a branch in the version control repository.
• A new branch may be created by branching from an existing branch
• a patch is a file that contains difference listings between two trees.
• A patch file can be used to transform (patch) one tree into another (tree).
• A quilt series is a method of representing an integration branch as a collection of a series of patches. These patches can be applied in sequence to the upstream branch to produce the integration branch.
• A tag is a named reference to a specific commit, and is not normally moved to point to a different commit.
• A package is an archive format of software created to be installed by a package management system or a self-sufficient installer, derived by transforming a tree associated with an integration branch.
• packages have package names
• package names are related to upstream software names
• packages have package versions
• package versions may have
  • an upstream version component
  • a distribution or packaging specific component
• package versions are related to upstream software versions
• helper packages provide libraries and other support facilities to help compile an integration branch ultimately yielding a package

My Openmoko FreeRunner suffered from the infamous hardware bug that causes bad buzzing for my dialog partner, making it relatively useless for me. Last week, though, a collegue of my brother fixed it for me (by applying some soldering-skills), and indeed, the audio quality is above bearable now! I plan to get Debian on the FreeRunner up to date again with the other developments in the community now. Or at least soon, depending on how much time I ll be able to spare.

This year it looks like Google Summer of Code will bring me much more work than it has brought in past years. Currently I have four students on the track:

• Mobile phones synchronisation for OpenSUSE. This will hopefully bring improvements to both Gammu and Conduit and the outcome should be working solution for at least some phones.
• Import improvements for phpMyAdmin. I would say this has been the best GSoC proposal I have seen so far, so let's see how it will progress.
• Changes tracking in phpMyAdmin. At the end we do have two projects on similar topic, but I want to push one of students more to synchronisation direction.

Let's see what will this bring to participating projects, but I hope all projects will end up at least with partial success and will bring great additions to free software.

At work I've been promoting the use of pkg-config and modules to solve the problems of avoiding hard-coded paths and an environment where we've multiple compilers. In summary, "module load intel-cc" to load the intel compiler, "module load netcdf-intel" to, which among other things appends /ichec/packages/netcdf/4.0-intel/pkgconfig to $PKG_CONFIG_PATH, and then:

NCDF_INCS :=  pkg-config netcdf --cflags 
NCDF_LIBS :=  pkg-config netcdf --libs 

in the application code. Replace "intel-cc" and "netcdf-intel" with "gcc" and "netcdf-gcc", and it builds with gcc. This would work better if upstream supplied .pc files, which means the next stage in world domination is to send patches to do just that. But it's not that easy, apparently. netcdf (for example) supplies two libraries, libnetcdf.so and libnetcff.so, with the second including code only needed for Fortran. So, for gcc I have the following netcdf file:

prefix=/ichec/packages/netcdf/4.0-gcc
exec_prefix=$ prefix 
libdir=$ prefix /lib
includedir=$ prefix /include
Name: netcdf
Description: netCDF libraries, include files and development tools (gcc version)
Version: 4.0
Libs: -L$ libdir  -Wl,--as-needed -lnetcdf   -lnetcdff 
Cflags: -I$ includedir 

While for Intel I have:

prefix=/ichec/packages/netcdf/4.0-intel
# Add fortran libs 
forlibs=/ichec/packages/intel/fce/11.0.081/lib/intel64
exec_prefix=$ prefix 
libdir=$ prefix /lib
includedir=$ prefix /include
Name: netcdf
Description: netCDF libraries, include files and development tools
Version: 4.0
Libs: -L$ libdir  -lnetcdf -L$ forlibs   -lnetcdff -lifcore
Cflags: -I$ includedir 

The main problem is that --as-needed is not understood by non GNU-ld linkers, and must be conditionally removed somehow. Any ideas ? (There is a second wrinkle of needing to add additional libraries for Intel Fortran here, but I'm sure I can remove that with the addition of more .pc files.). A second issue is that --as-needed can break otherwise working pkg-config usages. Thanks to galtgendo at PhP Bugs for this example:

alastair@ailm:~$ cc -o testx  -Wl,--as-needed  pkg-config glib-2.0 --cflags  test.c  pkg-config glib-2.0 --libs 

works, but:

alastair@ailm:~$ cc -o testx  -Wl,--as-needed  pkg-config glib-2.0 --cflags    pkg-config glib-2.0 --libs  test.c
/tmp/ccsUEwtk.o: In function  main':
test.c:(.text+0x20): undefined reference to  g_print'
collect2: ld returned 1 exit status

The problem being that --as-needed removes libraries as unnecessary before the linker sees the test.o code. Tags programming, debian, work, pkg-config, shared-libs, lazyweb

Alan Toner has written an overview of the Pirate Bay convictions. Well worth a read if you are interested in these developments. And of course, anon can t stay away from a good party so they are making some of their own waves as well.

Yay, newsbeuter 2.0 was just released! I am way too lazy to summarize all the new features and bugfixes, It's just to many of them. Here is the changelog:

Added more flexible dialog handling
Improved position handling in article list (fixes #112; thanks to Isaac Good)
Fixed a lot of bugs (#102, #111, #117, #130, #131).
Added ability to specify a list of OPML URLs when using OPML as URL source.
Added config option "keep-articles-days" to optionally keep articles only for a limited number of days.
Added config option "bookmark-interactive" to indicate that the configured bookmarking command is interactive.
Don't display authentication information in URLs (fixes #121).
Replaced mrss with new RSS/Atom parser.
Added ability to search for text from the article view.
Added basic support for Yahoo Media RSS.
Made article view pager configurable.
Improved HTML rendering of links and underlined and bold text.
Added ":source" commandline command to (re)load configuration files.
Implemented "pipe-to" key to pipe articles to external commands.
Implemented backtick evaluation for configuration files.
Extended filter language with "between" operator.
Added "age" attribute for articles to filter them for relative age (in days).
Extended "set" commandline command to toggle boolean variables and reset configuration variables of all types to their default.
Added ability to configure local files as feeds.
Added a "random-unread" key to go to a random unread article.
When opening articles from a search result dialog, make search phrase stand out in article view.
Persist commandline and search history.
Implemented commandline completion.
Improved help dialog so that it now shows unbound functions.
Added ability to sort feed list and article list by interactively choosing the sort method.
Improved and extended conditional HTTP download handling.

The full release annoucement by ak is available on: http://newsbeuter.wordpress.com/2009/04/21/newsbeuter-20-released/

Also just uploaded the package to Debian unstable.