Search Results: "pm"

13 November 2008

Christian Perrier: One more step towards the release

The D-I team finally released Debian Installer RC1 for lenny. That wasn't an easy process and it revealed some miscommunication and rampant conflicts inside the team. That will needed to be solved for squeeze development but, as of now, we (the very few of us still working on D-I development...) focus on preparing an RC2 release. That release is meant to be out ASAP. It is mostly meant to deal with recent kernel updates and definitely not add new features. Even bug fixing will probably be limited to what's considered major blockers. That's another step towards the overall release of lenny, I think. Now we should probably all focus on remaining RC bugs, which count seems to have stopped going down during last weeks, which is IMHO worrying. But, of course, let our good release team deal with all this first (which probably involves dropping some crappy packages, imho).

Biella Coleman: Steal this Film

For tomorrow’s class we are going to be discussing piracy and we will do so with the help of Alan Toner one of the filmmakers behind Steal this Film. We will be reading some of his work (and this ethnographic look at the life of a p2p file sharer provides a refreshing look at what it actually looks like), as well as the work of Matt Mason among others. We will also be reading an oldie but goodie that I could not find easily online and given that it is a class on piracy, I thought I would honor the class (and the piece) engage in a little transgression. If find the piracy exhilarating, daunting, questionable, or reproachable, come to NYU Free Culture’s screening of Steal this Film part 2, this Sunday Nov 16th at 7 PM. Alan will also be there to field questions and kick off the debate. I am looking forward to the post-movie discussion. update: Here is a flyer with lots more information

12 November 2008

Obey Arthur Liu: Aptitude 0.5.0 (aka Aptitude-gtk) released

Long time no post. Anyway, I have some good news. The Gtk code for Aptitude has been merged some time ago into the main development trunk and we now have a release in Experimental.

For those that don’t know about it, here’s what it’s all about : “The new frontend is is an effort to bring some of the design principles of the curses frontend to a GUI environment, while also exploiting the unique features a GUI gives us and exploring ways to deal with changes in the environment in the nine years since aptitude was first designed.” I had a very good time this summer working on Aptitude with Daniel Burrows in the Google Summer of Code program and I’m very glad we now have a real release. This version is by no means final or perfect but it’s a good start. Head for the blog post from Daniel for some other informations : [Daniel Burrows]

11 November 2008

Daniel Burrows: aptitude 0.5.0 released

I've released version 0.5.0 of the aptitude package manager (release notes). Once the mirrors have synced, you should be able to download it from Debian's experimental distribution, or manually from the aptitude package page.

[Edit] This release was uploaded to experimental because it is not a final release. aptitude follows the Linux version numbering scheme in which an odd second digit indicates an unstable development version. DO NOT INSTALL THIS VERSION IF YOU WANT STABLE SOFTWARE; IT IS INCOMPLETE, INADEQUATE, AND PROBABLY INCORRECT. To reply to some questions that several people have asked me:

The final release of aptitude 0.6.0 will be split into two packages, one containing only the traditional text interface and named aptitude-curses, and one containing the GUI binary named aptitude-gtk, with both of them providing the aptitude binary through Debian's alternatives mechanism. This release wasn't split because the code to drop GTK+ support was buggy and I discovered this fact halfway through assembling the release. It's fixed now.
To run the traditional curses interface, pass the --no-gui command-line parameter or set the option Aptitude::Start-Gui to false in /etc/apt/apt.conf. Unless you want to test that the curses frontend isn't broken, though, there won't be much for you in this release: the only change affecting it is Xapian support, and that currently breaks incremental search. (see the note above about this being an unstable release)

Also, another feature that I suspect might be bothersome is the behavior of updating the package lists and downloading changelogs on startup. There's no switch for changelogs yet, but you can disable the package list update by setting Aptitude::Update-On-Startup to false.

The main change in this release is the new GTK+ frontend, designed and implemented by Obey Arthur Liu with funding from the 2008 edition of Google's Summer of Code.

The new frontend is is an effort to bring some of the design principles of the curses frontend to a GUI environment, while also exploiting the unique features a GUI gives us and exploring ways to deal with changes in the environment in the nine years since aptitude was first designed. For instance, it is no longer reasonable for the user to actually read the entire package list (when I first installed Debian, you could read through the whole package list in a single sitting). So instead of basing the interface around a list of all the packages, we based it around the ability to search for the packages you're interested in. This version of aptitude also introduces Xapian searching:

daniel@emurlahn:~$ aptitude search "apt package manager"
i   apt                                  - Advanced front-end for dpkg
p   apt-dater                            - terminal-based remote package update manager
p   apt-dater-dbg                        - terminal-based remote package update manager (d
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
p   createrepo                           - generates the metadata necessary for a RPM pack
p   gnome-apt                            - graphical package manager
p   smartpm                              - An alternative package manager that works with
p   smartpm-core                         - An alternative package manager that works with
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality

You might ask why I didn't search for just package manager. The reason is simple: aptitude doesn't yet sort by relevance, and that second search gave me screenfuls of packages whose description contains both package and manager, including addressmanager.app (a PIM for GNUstep), compiz-gtk (a piece of eye candy), and wterm (an X terminal emulator). However, it is worth noting that Xapian searches are fully integrated into the aptitude search language, so they can be combined with non-Xapian search terms in just the way you would expect:

daniel@emurlahn:~$ aptitude search "?installed apt package manager"
i   apt                                  - Advanced front-end for dpkg
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality

So, all the basic functionality that the GUI version needs is in place; the next step is to start polishing it and filling in the gaps that are left. And, of course, suggestions and bug reports are welcome, so we know where the rough spots and the gaps are. Please send your comments to aptitude-devel@lists.alioth.debian.org so that they are part of the public record and so that people besides me can read them. :-)

Happy package managing!

Biella Coleman: Steal this Film

John Goerzen: Review: Silicon Mechanics

After some hilariously frightening reactions from Dell support to simple problems, and HP becoming aggressively competitive on price, we've been using HP servers for a few years now. The hardware is good, and the support, while reasonable, always... pauses... when I mention that we're running Debian. I try not to let it slip if I don't have to.

We put in some HP blades a couple of years ago, and I was annoyed to discover that they have discontinued that enclosure and all the blades in it. I decided this was a good time to look at their newer options, as well as at other companies.

Back in July, I had noticed a Silicon Mechanics booth at OSCon. I noticed their slogan "experts included." That sounds great; we've got software experts here, but not hardware experts, and I'd enjoy dealing with a company that knows more about their hardware than I do. I went up to their booth and asked what they'd say about us running Debian on their hardware. "That would be just fine." "So you'd fully support it when I'm running Debian?" "Sure." "What about management software - do you have any of that which I'd find annoying to port to Debian?" "Our servers don't need any management software other than what comes with your kernel." Good answers.

So, when it came time for us to decide what to do about getting a new server in here, I figured I'd call up Silicon Mechanics and see what they'd recommend. They put me on a conference call with a sales rep and an IT engineer, and wound up recommending a 1U server for us to start with, and an iSCSI storage device to address some of the storage needs we have (both for that server and others). I had heard of iSCSI only vaguely, and asked how it worked, and what the performance would be like compared to our 2Gb FC SAN. I got back intelligent (and correct) answers.

They probably spent 2 hours with me on the phone before we placed an order. I was incredibly happy with their service, level of expertise, and helpfulness. They even did a webinar to demo the management interface on the storage unit for me.

Today, the 1U server arrived. I unboxed it and set it on my desk to configure. First item: set an IP address for the IPMI card. That's the device that lets me connect to it over a web browser and interact with the console, power cycle it, etc. as if I was there. I set an IP, but somehow couldn't seem to figure out the username and password for the web interface.

So I called Silicon Mechanics support at the number that was included on the fridge magnet (!) that came with the shipment. Phone rang once. Then a live, capable American answered. No menus, no fuss. I asked my question. He apologized, saying, "I should know that, but I'll have to look it up... hold on just a bit." I had my answer about 90 seconds later. He offered to send me the full docs for the IPMI card if I wanted as well.

So I've been very impressed with them so far. From what I've heard, their iSCSI enclosure ought to be quite something as well. They even helped us spec out a switch that supports trunking for use with it.

I'll give them a "highly recommended".

10 November 2008

Riku Voipio: When Linux doesn't mean freedom

I actually think it's a bit of an insult if people think of Motorola's EZX or MAGX (and now Android) phones as "Linux phones". Because all the freedoms of Linux (writing native applications against native Linux APIs that Linux developers know and love, being able to do Linux [kernel] development) are stripped.

-
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">Harald Welte

This is something that has worried me too for quite a while. For years we have now had Linux phones available in form or another (well, mostly in far away countries you happen not to be at). Yet with the exception of openmoko none of them allow native application development. Yet all the same time you can go to the nearest phone shop and grab HTC windows CE phone or a Nokia Symbian phone, which actually give you the freedom to writing and running your own software on them.

Now we have the situation that HTC's most locked device* is their only Linux device - The Android G1.

Incidentally, you can install Debian/armel on T-mobile G1, but only until Google engineers manage to fix the hole that gives you r00t.

*The only one that doesn't allow native app development.

David Welton: Generating Rails Fixtures From Large Datasets

I've been working with Tomaso Minelli doing some Rails work, and one thing we needed to do was set up some tests. The situation we are in is that we already have a large set of data to work with in our development environment, but getting that data into .yml files is no easy task. Since we have a fairly large number of interdependent tables, dumping one of them alone is pretty useless, unless you manage to get the other tables it links to, and so on and so forth. Dumping all the data would make testing quite slow, and really is too much to be useful. Here's the rough solution we arrived at:


@fixtures =  
@seen =  
# Do something with a fixture.  In this case, we store it for later.
def handle_fixture(record)
  @fixtures[record.class.table_name]  = []
  @fixtures[record.class.table_name] << record
end
# Get some sort of unique way of identifiying a record - class name and id.
def unique_record(record)
  return record.class.to_s + " " + record.id.to_s
end
# Grab all the relations of a class and add them to the queue.
def add_fixture(record)
  ur = unique_record(record)
  if @seen[ur]
    puts "LOOP!"
    return
  end
  @seen[ur] = true
  puts "Fixture for # record.class.name  - ID # record.id "
  handle_fixture(record)
  record.class.reflections.each do  k, r 
    begin
      if r.macro == :has_many  
          r.macro == :has_and_belongs_to_many
        related = record.send(r.name).send('find', :all, :limit => STARTING_LIMIT)
      else
        related = record.send(r.name)
      end
      related = [related] unless related.is_a?(Array)
      related.each do  related_record 
        if related_record
          puts "    New Record # related_record.class.name  # related_record.id "
          @todo << related_record
        end
      end
    rescue => e
      puts "    ERROR: # e :\n# e.backtrace.inspect "
    end
  end
end
@todo = []
namespace :db do
  namespace :fixtures do
    desc 'Create YAML test fixtures starting from a single model using reflections'
    task :modelbase => :environment do
      STARTING_CLASS = Foobar
      STARTING_FIND  = :all
      STARTING_LIMIT = 1
      tmp = 0
      totale = STARTING_CLASS.find(STARTING_FIND, :limit => STARTING_LIMIT).size
      STARTING_CLASS.find(STARTING_FIND, :limit => STARTING_LIMIT).each do  m 
        tmp += 1
        puts "# tmp  su # totale : # m.id "
        @todo << m
      end
      while true
        if @todo.length == 0
          break
        end
        add_fixture(@todo.pop)
      end
      puts "\n\n# '='*80 \nScan Finished\n# '='*80 \n\n"
      @fixtures.each do  name, fixture 
        if fixture.size > 0
          i = "000"
          filename = "# RAILS_ROOT /test/fixtures/# name .yml"
          puts "Create # filename  for class # fixture.class.name "
          FileUtils.mkdir(File.dirname(filename)) unless Dir[File.dirname(filename)].size > 0
          File.open("# RAILS_ROOT /test/fixtures/# name .yml", 'w') do  file 
            file.write fixture.inject( )    hash, record 
              hash["# record.class.table_name _# i.succ! "] = record.attributes
              hash
             .to_yaml
          end
        else
          puts "Fixture for # name  is EMPTY"
        end
      end
    end
  end
end

It's written in a somewhat quick and dirty way, but it gets the job done. What it does: It takes a single class - hopefully a well connected one - fetches a few of them, and goes from there, exploring all the other objects that the initial ones are connected to, and then dumping them in .yml files in the fixtures/ directory. It could certainly use more work:

Additional starter classes, for ones that aren't connected to others.
Intelligent examination of data and data sets...
For example you could check to see what classes aren't connected to the others and use them as starter classes.
If you wanted to waste a lot of cycles, you could also try and aim for a representative set of examples. For an object that has_many something elses, you could try and find one with zero, one with one, and one with a lot, to cover various cases. That could quickly get out of hand for largish data sets though.
I'm sure there are various other tweaks, knobs, and levers that could be added to work more efficiently. I'd love to see them!

Biella Coleman: Epic Lulz will be had (aka: I am giving a talk)

Screenshot-coleman-scientology-slides - OpenOffice.org Impress -1, originally uploaded by the biella.

“Old and New Net Wars over Free Speech, Freedom and Secrecy or How to Understand the Lulz battle against the C0$ Gabriella Coleman, Assistant Professor, New York University Thursday November 13: noon - 2:00pm
Columbia University Communications Colloquium
270B IAB (International Affairs Building update Open the public! You may need ID to get into the building but I am not sure. This is the case with NYU…

Biella Coleman: Epic Lulz will be had (aka: I am giving a talk)

Screenshot-coleman-scientology-slides - OpenOffice.org Impress -1, originally uploaded by the biella.

Matthew Garrett: Android

As a brief introduction to this - I first read through the Android code when interviewing with Google for an opening with the Android kernel team. I didn't get the job, and while I don't think anything that follows here is as a result of residual bitterness you might want to take some of it with a pinch of salt.

Anyway. Android is a Linux kernel with a heavily customised user layer that bears little resemblance to any traditional Linux system. I write in C because using pointer arithmetic lets people know that you're virile so this Java-like thing is of little interest to me and I'm going to ignore it and just look at the kernel, because after all that's what I'm paid to be good at.

The short summary of the rest of this post is that I'm not planning on giving up my iphone yet.

The Android kernel source lives in a git tree here. It can pretty much be logically split into two parts - the port to Qualcomm's MSM7xxx series chips and the special Android customisations. A bunch of the MSM7xxx port code has been merged through the ARM tree and is now upstream, and Brian Swetland seems to have been fairly active in looking after that. Full marks to Google there. This code is handy outside the Android world and benefits anyone wanting to run Linux on similar devices.

The Android-specific code is more... interesting.

As I mentioned, the Android application layer isn't Unix in any real sense. The kernel reflects this. It ranges from pragmatic (if hacky) approaches like encoding security policy and capabilities in group IDs that are hardcoded into the kernel through to implementing an in-kernel IPC mechanism (apparently related to the OpenBinder implementation from Palm, but judging by the copyrights a complete rewrite). To an extent, I'm fine with this. Something like Binder is pretty clearly not going upstream, so the fact that it engages in bizarre design decisions like sticking chunks of its interface in /proc is pretty irrelevant. What's more interesting is the code that should be generalisable.

I work on power management, so I'm always interested in what kind of power management functionality and interfaces people want. Plumbers included a nice discussion with someone from an embedded company I can't remember, culminating in us deciding that the existing cpufreq interface did what they wanted and so no new interfaces needed to be defined. Google was going to be an interesting case of a large company hiring people both from the embedded world and also the existing Linux development community and then producing an embedded device that was intended to compete with the very best existing platforms. I had high hopes that this combination of factors would result in the Linux community as a whole having a better idea what the constraints and requirements for high-quality power management in the embedded world were, rather than us ending up with another pile of vendor code sitting on an FTP site somewhere in Taiwan that implements its power management by passing tokenised dead mice through a wormhole.

To a certain extent, my hopes were fulfilled. We got a git server in California.

Android contains something called the android_power driver (ignore the references to CSMI, which is a piece of OMAP-specific hardware for communicating with the baseband - I'm not sure what they're doing in there). As far as I can tell, this is an interface that handles the device being locked and unlocked, and associated powering down of certain bits of hardware. Except it's shit. Devices register device handlers and a level where they wish to be suspended. There's no direct concept of intra-device dependencies so you end up with stuff like:

android_early_suspend_t early_suspend;
android_early_suspend_t slightly_earlier_suspend;

to deal with the fact that the MSM framebuffer driver depends on the MDDI driver having brought the link bank up and so needs its suspend method called before the MDDI driver but its resume method to be called after, and the only way to handle that is to register two methods - the "slightly earlier" one which has a suspend method but no resume, and the "early" one which has a resume method but no suspend one.

Of course, this also means that all of your device runtime power management policy ends up in the kernel. Userspace indicates what state it wants to go to and the kernel decides what's going to get powered down. This kind of coarse grained approach means that as your hardware setup becomes more complex you hit combinatorial explosion. Expressing all the useful combinations of hardware state simply becomes impractical if all you're exposing is a single variable. What would be more useful is the ability for userland to interact with individual pieces of hardware.

The amusing thing is that in many cases Linux already has this. Take a look at the backlight and LCD class drivers, for instance. They provide a trivial mechanism for userspace to indicate its desires and then modify the device power state. It's true that there are other pieces of hardware that don't currently have interfaces to provide this kind of information. And that's where cooperation with the existing community comes in. We've already successfully fleshed out interfaces for runtime power management for several hardware classes, with the main thing blocking us being a lack of awareness of what the use cases for the remaining classes are. But linux-pm has seen nobody from the Android team, and so we end up with a lump of code solving a problem that shouldn't exist.

When Robert Love gave his presentation on Android at Lugradio Live in SF back in April, he talked about how one of the reasons that Google weren't releasing the source for their userland stack until they shipped phones was to prevent it being seen as throwing unfinished code over the wall and then ignoring the community. It's unfortunate that this is almost exactly what's happened with their kernel. Right now the fact that Android is based on Linux is doing almost nothing to benefit the larger Linux community. What could have been a valuable opportunity for us to gain understanding of an interesting problem has instead ended up as yet another embedded vendor kernel, despite all the assurances I got from various people at Google.

Disappointing.

9 November 2008

Christine Spang: Switching to Colemak

Way back in mid-late May, I switched keyboard layouts from standard US QWERTY to Colemak. I’d played around with the layout a bit before that and thought it seemed pretty solid, but it hadn’t been a good time to switch then. (I did, however, snag the idea of remapping Caps Lock to Backspace from that early trial though, and to this day that’s probably the most annoying thing about typing on QWERTY, because I associate Caps Lock with Backspace on both layouts now. Anyway.) This time, however, I strategically timed it to coincide with the end of classes, so I had no assignments due and could make a fool out of myself during the first few days of my summer job. Smashing success. It only took a couple days to get past the point of extremely frustrating slowness. Two weeks in I was up to 40wpm, which was about the point where I stopped noticing that I’d taken a backwards step in typing speed, as it didn’t really hinder anything I did on a regular basis. By the end of the summer I was back up to the same speed as I’d been prior to switching (around 75wpm). I didn’t really do any typing lessons after the first few days, as I found just doing whatever I would normally do on a computer much more interesting. The biggest hurdle after the initial adaptation of actually being able to type on the layout was, for a command-line lover like myself, dealing with “hjkl” not being a convenient way of navigating around in console apps like mutt, vim, aptitude, and less. After a bit of experimentation, I stole an idea from this page and started adapting my dotfiles to make these programs use ne for jk and st for hl, and that seems to be working out well. The worst key that this clobbers in general is n, but it’s pretty easy to remap that to QWERTY’s n, which is easy to remember and is the hardly-used k on the Colemak layout. For any interested others, I’ve thrown the relevant dotfile snippets up here. I’d been putting off blogging this because I wanted to finish setting up dotfiles/homedirs-in-git la Joey Hess beforehand, but I figure I’ll never actually get it done if I keep using that excuse. For mutt, the given snippet goes in your .muttrc (or whatever you source keybindings into your muttrc from). The same goes for vim. Aptitude keybindings go in .aptitude/config. For less, it’s a little more arcane: first you stick your keybindings in .lesskey, and then you run the command lesskey. I also remapped screen’s escape key to ctrl-t. Why not Dvorak? Dvorak seems to be the most well-known QWERTY alternative around, so why didn’t I switch to that if I wanted something better? I did try Dvorak first, but the incredibly uncomfortable combination of ls on the right pinky was a complete deal-breaker. I also found it very slow-going to try to adapt to Dvorak after typing on QWERTY for years. The Colemak FAQ has some more notes on Dvorak deficiencies that it addresses. Overall, I’ve found typing on Colemak to be remarkably lacking in anything that particularly bothers me. I still type on QWERTY a little, but mostly just for login prompts on Athena workstations and the desktop machine in my room which my roommate sometimes uses. I have to glance down at the keyboard when typing on QWERTY to get it back in my head, but generally I only need it for a few minutes anyway and it’s no big deal. As long as I glance down occasionally I can still maintain a speed level well above that of frustration. On the plus side, this means that if I look at the keyboard when typing normally in Colemak, I get screwed up because I start thinking in QWERTY. So it’s good touch-typing reinforcement in general.

Simon McVittie: Implementing D-Bus services with telepathy-glib

(Part 1 of an ongoing series: shiny things in telepathy-glib)

16:46 < epmf> tp-glib is made of magic
—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Introduction to telepathy-glib telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-) My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September. This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months. The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!

Some background: service-side code generation

The secret Collabora code-generation process

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong. dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time. Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks. Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:

copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
pray that the diff process hadn't randomly exchanged functions' implementations
update the resulting code in src so it worked
check the whole mess back into darcs
hope that nobody else had made changes that would result in darcs conflicts

If you haven't yet gathered, this was a bit of a nightmare. Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.

How telepathy-glib fixes code generation telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib:

commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Fr cinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200
    Interface support for bindings.

Instead of running dbus-binding-tool or creating GObjects with D-Bus interfaces, we put the entire Telepathy spec through glib-ginterface-gen.py, a distant descendant of Gabble's gengobject.py. For each interface in the spec, we generate a GInterface that mirrors the D-Bus interface. Any GObject that implements the GInterface automatically gets the D-Bus interface if it's exported onto D-Bus. There are a few non-obvious refinements, though:

The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns. For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened. This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done). For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.
The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.
If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.
While we're generating code anyway, we generate some static inline wrapper functions which wrap dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().
Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.

Not all interfaces are stable enough to be included in telepathy-glib's stable API and ABI, so some of our other projects include a copy of the telepathy-glib code-generation tools, and generate their own "mini-telepathy-glib" (traditionally in a /extensions/ directory) for their implementation-specific (drafts of) interfaces. This isn't yet as polished as it ought to be (mainly because we don't want to freeze the augmented-introspection-XML format that we write the Telepathy spec in, because it's rather ad-hoc and hackish in places) but it works quite well in practice. To see what this looks like in practice, have a look at the examples in telepathy-glib, or at a Telepathy connection manager like telepathy-gabble.

Mixins and base classes Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces. There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does. The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.

8 November 2008

Steve McIntyre: 3rd Free Software Congress in Valencia

I spent much of this week in Alicante. It was wonderful to have a few days away from the beginnings of Winter in Cambridge - Alicante's in South West Spain, right on the Mediterranean. It's a lovely city, although it was very quiet due to the time of year. I was invited down to Alicante by organisers of the Congr s de Programari Lliure, Comunitat Valenciana, to present a talk about Debian. That I did, and as always my slides and photos are available in case people are interested. I wasn't too sure about what level to aim the talk, so I deliberately went for a general introduction to Debian. That seems to have been the correct level - nobody threw anything at me, and nobody was visibly falling asleep. *grin* It's also the first time I've ever given a talk with official translators, which is a weird experience! The main topic for the conference was the new release of Lliurex, Valencia's own GNU/Linux distribution targeted primarily at use in schools. The conference was massive; I was told there were about 1500 attendees in all, most of whom were teachers who wanted to come and learn more about Free Software. There was a good spread of talks and workshops, covering topics from development through to deployment in the classroom. The level of enthusiasm for Free Software here was immense - lots of people were keen to learn more about it and get involved. I was told that out of the 15 regions/states of Spain, 11 of them have now started their own distributions to use in schools! Extremadura were the first region to experiment with Free Software like this, and their success with LinEx was a clear inspiration to their neighbours elsewhere in Spain. As well as the obvious cost advantages of moving from proprietary to Free Software, the local governments also love the freedom to use and modify their software however they like - including the ability to provide their own translations.

I met up with a lot of cool people at the conference too. The local education minister (Alejandro Font de Mora) was keen to talk with all the speakers and very enthusiastic about the future of Free Software in his schools. As is increasingly common at FS events around the world, there were of course quite a few Debian people around too. Luciano and Miriam were also giving talks, and I bumped into Rene and Miguel and a whole bunch of others whose names I'm too crap to remember - sorry! I also spoke a lot with Kurt Gramlich, the mega-enthusiastic Skolelinux developer who was also happy to push Debian to everybody he spoke to *grin*. Jon "Maddog" Hall had some excellent ideas that I'm sure we'll all hear more about soon, and his talk was excellent. He did a very good job of explaining how Free Software makes such a difference, including how it helps people to make money. He also explained that due to travel he had missed Halloween at home and he wanted to share the experience of scary masks with the people at the conference:

I had a great time in Alicante, and I'm very grateful for the invitation to go there. I wish that the UK was as far as advanced as Spain in the use of Free Software in education - they've done a huge amount of good work in this area and I'm glad we can help them.

6 November 2008

David Moreno Garza: Harlem and the elections

A lot has been said about the elections already. I just want to share my personal experiences about it. During the weekend, Tom invited us to his place in lower Harlem (we live upper Harlem) on 123rd St and Adam Clayton Powell Jr Ave, for Tuesday night. As Tuesday arrived, we didn’t see too many people too excited about it. Living in Harlem, we were pretty used to black people supporting Obama, so getting closer and closer to the elections, we didn’t feel people anxious. On Tuesday, we commuted to work and some of our co-workers got to work late because they have been standing in lines for a couple of hours to vote in polls. The whole day was about talking on how much that day was representing for the US and the world, guessing how they results were going to end up, etc. Later at night, Greg, Phil, Raquel and I headed to Tom’s place for his watch-the-results party. His fridge was full of Blue Moon beer, but he also had some Red Stripe

Since I like to go light on politics, I preferred some other American variantion, Coors Light

Vermont, Connecticut and a few other New England states were claimed by the Democrats. Matty, a friend of Tom attending the party brought US maps with the electoral votes on each one for people to color them as they were called. It was pretty fun. At some point we forgot about the maps since it was pretty clear that Obama was gonna win. But when they called him at around 11 pm (and I was starting to feel nicely drunk), the whole neighborhood and people in the streets just went completely nuts. It’s something I hadn’t experienced at all in my life, it was like if Mexico would have won the f tbol World Cup (which yes, it will never happen anytime soon :P), but nicer. People went out to the Harlem State Office Building square were cars were honking, and a huge amount of people gathered to celebrate, play brass music, cheer “Yes We Can”. Ghetto guys, white hippies, curious latinos, Africans, African Americans, Puerto Ricans, just a lot, a lot of people, all kinds of them, just a clusterfuck. They projected Obama’s speech from Chicago and people went nuts. It was just, fucking amazing. The images they projected from Harlem to the outside world was in that same square we were in. Our Flickr set for the celebration here.

Martin-Éric Racine: Available for a new job

A couple of years ago, I started what has probably been the best job I've held so far, as a Business Development Manager for Artec Group where I focused on marketing and selling the ThinCan LTSP client. During that time, I used a combination of Guerrilla Marketing techniques to spread the word about this cool Estonian embedded platform in its LTSP client configuration and defined an extremely tight market-oriented focus that saw this relatively obscure electronic design start-up rise to the forefront of the LTSP hardware market and also brought the company commendable notoriety in the Coreboot community, thanks to its Programmable LPC Dongle, a low-cost ROM emulator that can be used to bootstrap various embedded hardware and test firmware images. Looking back at close to 3 years with Artec (if we include the stint I did at Linutop), I most fondly remember how I managed to bring the Estonian and Turkish nations closer together in collaborating on Innovation. On my first visit to Turkey, I had noticed that very little is know about Estonia in Turkey, even though both countries have a collaboration that dates back to the 1920's, so my tactic was to introduce Estonia's achievements as a whole, to establish confidence in Artec's offering. It worked: Among other things, my idea to organize an Estonian ICT Conference during the visit of Estonian PM Andrus Ansip in Turkey created a lot of extremely positive buzz that resulted in Estonia being mentioned as a trendy country in a prime time Turkish TV series. During this time, I've also made a lot of contacts among the Estonian and Turkish business communities and, more surprisingly, among the Estonian diplomacy who, much to my amazement, welcomed my ideas for better promoting Estonian interests abroad with great enthusiasm. Come December 2008, my time at Artec will become a page in my employment history. While the Thincan remains one of Artec's key products, current marketing and sales resources are focused on new markets and new technologies that completely fall outside my core expertise, which is why the company decided to let me go. I thus welcome offers from interested parties in Coreboot or LTSP markets. Other refreshing offers from outside the box - such as in business diplomacy - are equally welcome, as I've been pondering a career change for a long time.

Eddy Petrișor: Coordinating localization via git

Note: this is quite long and might not be interesting for people not involved with /in either of: debian i18n, debian l10n, git, shell scripting.

For some time now I have been the de-facto coordinator of the Romanian localization team. During this time I was faced multiple times with problems related to motivation of the team members, setting goals for a release, coordination of the changes inside the team, integration of new translators in the team, loosing team members, my lack of time on some occasions and other problems.

I thought a lot about how to improve on those points and I was never satisfied with the answers I got. What I think was the worst problem in the team was the inability to set clear goals, especially during lenny's development cycle.

During sarge's and etch's development cycles we were in an infancy state and setting as the goal to have a fully translated installation process was really enough to keep people motivated. For sarge we missed by a small margin, but the translations were of poor quality, while for etch we worked more on improving the translations.

But during lenny it was bad, really bad. My free time had been shrinking for a while, starting with etch's release, and we were unable to set clear goal, since for etch we managed to have the installation process fully translated and a few other translations. There was no way for us to reach 100% translations during lenny's development cycle, so setting that as a goal was really unrealistic. Percents by themselves don't mean anything for people, and as long as there's no substance to those numbers, there's no motivation to reach for one arbitrary percent.

I tried to set as a goal translating the packages installed by default in a new installation, but that hit the eternal question How do we know easily which are those packages?. This remained sometimes unanswered or got an unsatisfying answer. Also, there was a goal to have correct diacritics for Romanian in lenny, to have aspell-ro that uses the correct diacritics.

I even got to a point that I, myself, lost my motivation and set myself a personal goal of overrunning the level the translations that the language just above Romanian had in the po-debconf l10n statistics (ranking between languages). This was nice way for me to keep myself motivated, but I had my reserves in making this motivation public out of fear of being misinterpreted, because, by a strange coincidence, that next language was Hungarian, and in Romania's history there was some friction between Hungaria and Romania, while there are still some tensions with the Hungarian minority in Romania, in areas where they represent the local majority.

I managed to reach my personal goal, but this wasn't addressing the big picture.

So, sometime around the start of this year I started thinking about ways to coordinate the Romanian localization team in order to have:

a clear goal at any given time
a way to always be able to change that goal as we go
a way to sync with eventual calls for translations, or the current sid translations
stats immediately available
automated checks for spelling, correct diacritics usage and other checks that might be useful (e.g. translation completeness)
an easy way to assign somebody else as a language coordinator (I would appreciate some help or I might even consider stepping down)
easy integration of new translators (by providing immediate answer to the question "What can I do to help with the translations?")

For short, a tool that would allow the team to work more efficiently while having the possibility to set clear goals in order to keep people motivated.

So after some pondering, I thought that creating a repository with the translations and the helper tools that would do the funky sync, checks, stats would be the best way to do that. So I started hacking on that somewhere around July-August and I published the result, but without much publicity, since it is still incomplete.

Some of the technical details are still in a haze, but I have a general idea and I got some basic functionality.

Today I decided that I should announce this semi-officially though my blog, maybe I get some input, ideas, or even contributions (I really should write a TODO).

I give you the Debian L10n Romanian coordination repository.

This is a git repository that has some tools to facilitate translation coordination and the translations that are current in the distro for the team.

Can be cloned with:

git clone git://git.debian.org/git/users/eddyp-guest/debian-ro-repo.git

or, if you're behind a restrictive firewall:

git clone http://git.debian.org/git/users/eddyp-guest/debian-ro-repo.git

Currently the work flow for updating a translation is as follows:

source _bin/polibs (. _bin/polibs)
cd foo
po_refresh
complete the translation
po_rearrange "ro.po"
git add "ro.po" && git commit -m "updated translation for foo"
send the translation ("git format-patch origin" and send the patches by mail, or, alternatively, just "git push")

Features:

provides a po_refresh function that can import material directly from http://i18n.debian.net/material, but can also allow manual imports (template.pot from a call from translation)

for a new translation: source _bin/polibs (. _bin/polibs), make a directory with the name of the source package, cd into it, and run po_refresh

po_rearrange - beautify and unify the layout of PO files (facilitating compact and sane diff-ing for PO files)
po_merge uses compendium, if present

Planned features:

sync translations/templates from package VCS-es (Vcs-* headers and debcheckout should be the means to the end)
po_rearrange should be called as a pre-commit hook; should either reject the commit if the po file was not rearranged, or automatically rearranged before the commit
generate stats
add commands for "what's outdated", "what needs review", "submit translation", and maybe "reserve translation for offline use"
conflict merges should be done via po_merge (.gitattributes is key here)
support other file types (?) - does this make any sense?
periodic and automated sync with sid for all translations

Problems:

security - running tests automatically from files within the repo doesn't seem too wise, but looks like the only way to get automated testing on any translator machine; maybe keeping the code in a submodule might address this issue?
entry level translators still have a hard time - UI sucks now; there should be a wrapper command that should use the library functions and should provide a useful help
is git too difficult ? - git backend usage maybe should be cloaked?
still in development/alpha stage - I still haven't figured some of the issues
central repo or really distributed - should there be a central git repo where the coordinator(s) do the pushes? it seems the central repo with a small pushers team for new translators (which can't commit directly) might actually facilitate interactions between experienced and new translators to instruct/bring up to speed the rookies

I was hoping that the release notes for lenny would facilitate from this infrastructure, but unfortunately I was lately in a really inactive period wrt Debian.

Questions, suggestions, ideas are welcome.

4 November 2008

Ross Burton: OfflineIMAP, ConsoleKit, GNOME Keyring

Over the weekend I finally got fed up with Evolution struggling to connect to work's "IMAP" server (Exchange 2007), and switched to using OfflineIMAP to sync the mail to a local Maildir. This as expected worked pretty well, and I'm now hidden from the nasty lag on the server. However, I've had to write my top secret Intel password into .offlineimaprc, which sucks. Then I had a cunning plan... GNOME Keyring will store passwords in a pretty secure manner, so somehow I need to fetch the password from there. A quick look at the OfflineIMAP manual revealed that I can write Python functions which return the password, so I should be abe to hook into the keyring from OfflineIMAP. This should be fairly simple:

import gobject, gnomekeyring
# The keyring needs to know the application name
if gobject.get_application_name() is None:
  gobject.set_application_name("offlineimap")
def keyring(user, host):
  keys = gnomekeyring.find_network_password_sync(user=user, server=host, protocol="imap")
  # First one will do nicely thanks
  return keys[0]["password"]
...
remotepasseval = keyring("rburton", "imapmail.intel.com")

After writing a small tool to add the key to the keyring, to my surprise this worked first time. I bounced with glee, but ten minutes later I had error messages from OfflineIMAP running from cron in my inbox... GNOME Keyring uses an environment variable to find the daemon, which isn't set in a cron environment. GNOME Keyring will fall back to using DBus to find the daemon, but the DBus session bus environment variable isn't set. DBus will fall back to reading the session bus address from the X root window, but DISPLAY isn't set so that doesn't work either... EPIC FAIL. But, I thought, I upgraded to Network Manager 0.7 last week which bought in ConsoleKit. If I ask ConsoleKit for my sessions I should be able to find a session with has an X connection, then I can set DISPLAY appropriately and then the chain described above will work, and I'll have my password. Shockingly, this worked first time too:

import dbus, os
if not os.getenv("DISPLAY"):
  # Get the ConsoleKit manager
  bus = dbus.SystemBus()
  manager_obj = bus.get_object('org.freedesktop.ConsoleKit', '/org/freedesktop/ConsoleKit/Manager')
  manager = dbus.Interface(manager_obj, 'org.freedesktop.ConsoleKit.Manager')
  
  # For each of my sessions..
  for ssid in manager.GetSessionsForUnixUser(os.getuid()):
    obj = bus.get_object('org.freedesktop.ConsoleKit', ssid)
    session = dbus.Interface(obj, 'org.freedesktop.ConsoleKit.Session')
    # Get the X11 display name
    dpy = session.GetX11Display()
    if dpy:
      # If we have a display, set the environment variable
      os.putenv("DISPLAY", dpy);
      break

(man, I really with python-dbus had a better syntax for getting objects with a specific interface) So there you go, integrating OfflineIMAP with the GNOME Keyring via ConsoleKit and DBus. Surprisingly this was pretty easy to do, thanks to DBus and the magic provided by ConsoleKit it is 100% hack free.

MJ Ray: Library Open Source Webinars (FYI/RFC)

Today is the first of the Library Open Source Webinars run by WiLS. It takes place in a few hours (2pm CST, which I think is 14:00:00-0600, or 20:00:00Z) and costs USD25 per session or USD100 for all six. Future sessions will cover Drupal and Evergreen - sadly not Koha, as far as I can see. I’m told that “standard browser, standard phone” is all that’s needed to attend live, but recordings are Windows Media Video (WMV) files, which I think isn’t great for an Open Source Webinar. I’d be interested to hear from anyone who attends, including whether it played nicely with your software.

MJ Ray: Library Open Source Webinars (FYI/RFC)

Next.

Previous.