There will be a general meeting on the #koha IRC channel on Wednesday, 6 May 2009 at 19:00 UTC with a nominal finish time of 21:00. To convert to your timezone, try date -d @1241636400 (thanks to SPI (Joerg?) for that idea). The meeting will include Koha 3.2/tip status and timeline, Koha 3.0.x status, Koha bugs database, KohaCon 2009 report. The full agenda is on the wiki. Connection details are shown on the wiki front page but are basically irc://irc.katipo.co.nz:6667/#koha Hope to see you there, or a summary should be linked from the comments here later.

I have just uploaded Embedded GLIBC (EGLIBC) into the archive (it is currently waiting in the NEW queue), which will soon replace the GNU C Library (GLIBC). The EGLIBC is a variant of the GLIBC which stays source and binary compatible with the original GLIBC. While primarily targeted for embedded architectures, it has some really nice points:

• More friendly upstream (especially with regard to embedded architectures): Encourage cooperation, communication, civility, and respect among developers (as opposed to this).
• Stable branch with fixes for important bugs (a real one, not like the GLIBC one which is left unchanged).
• Better support for embedded architectures.
• Support for different shells (GLIBC only supports bash).
• Support for building with -Os.
• Configurable components (do we really need NIS or RPC support in debian-installer?).
• Better testsuite for optimized or biarch packages.

We do not use some of these features yet, but this upload is a first step. From the user point of view, the package names are unchanged (except the source package and the binary package containing the sources) so no transition is needed.

Since Lenny was the first Debian release to feature CUPS under its new package naming strategy, I started going through 'rdepends' results to see which packages in Squeeze still present dependencies for *cupsys* packages. Much to my amazement, there's quite many. If you are the maintainer of a package that still has those dependencies, please fix them ASAP. Alternately, if you're aware of any favorite package that does, please do not hesitate at filing a patch to help the maintainer update their debian/control. Comes June 2009, transitional CUPS packages will be removed from our debian/control.

I have mentioned the flashrom utility in my blog in the past. This is a small command line tool which allows you to update your BIOS/coreboot/firmware chips without opening the computer and without any special boot procedures. Yesterday, flashrom 0.9 was finally released. Here's a short passage from the release announcement:

After nine years of development and constant improvement, we have added support for every BIOS flash ROM technology present on x86 mainboards and every flash ROM chip we ever saw in the wild. Highlights of flashrom include:

• Parallel, LPC, FWH and SPI flash interfaces.
• 157 flash chip families and half a dozen variants of each family.
• Flash chip package agnostic. DIP32, PLCC32, DIP8, SO8/SOIC8, TSOP32, TSOP40 and more have all been verified to work.
• 75 different chipsets, some with multiple flash controllers.
• Special mainboard enabling code for dozens of nonstandard mainboards.
• No physical access needed. root access is sufficient.
• No bootable floppy disk, bootable CD-ROM or other media needed.
• No keyboard or monitor needed. Simply reflash remotely via SSH.
• No instant reboot needed. Reflash your ROM in a running system, verify it, be happy. The new firmware will be present next time you boot.
• Crossflashing and hotflashing is possible as long as the flash chips are electrically and logically compatible (same protocol). Great for recovery.
• Scriptability. Reflash a whole pool of identical machines at the same time from the command line. It is recommended to check flashrom output and error codes.
• Speed. flashrom is much faster than vendor flash tools.
• Supports Linux, FreeBSD, DragonFly BSD, Solaris, Mac OS X. Please refer to the README for build instructions.

Please note that rewriting your flash chip can be dangerous and flashrom developers make no guarantees whatsoever. That said, many users have successfully replaced proprietary tools such as awdflash, amiflash and afudos with flashrom.

Download: flashrom-0.9.0.tar.gz
SVN: svn co svn://coreboot.org/flashrom/trunk flashrom
Debian: apt-get install flashrom Do yourself a favor and try flashrom next time you want to upgrade your BIOS. No more floppies or bootable CD-ROMs with DOS/Windows binaries or similar crap. Run flashrom conveniently from the Linux command line, or even via SSH or serial console if you want...

TIME: Monday, May 18, 7:00pm VENUE: Courant Institute
251 Mercer Street (Warren Weaver Hall)
Room 109 TITLE: The Struggle to Build a Digital Republic ABSTRACT: David Bollier will speak about the themes of his new book, Viral
Spiral: How the Commoners Built a Digital Republic of Their Own (New
Press). The book is the first comprehensive history of the free
culture movement and sharing economy that is empowering ordinary
people, disrupting markets and changing politics and culture. Bollier
will talk about the rise of free and open source software, Creative
Commons licenses, the new forms of non-market creativity (Wikipedia,
blogs, remix music, videos) as well as fascinating innovations in open
science, open education and open business models. More about the
book can be found at the website www.viralspiral.cc. More about
Bollier can be found at www.bollier.org. Bio
David Bollier is a leading American activist, author, blogger and
proponent of free culture on the Internet and the commons. He is an
editor of Onthecommons.org and Senior Fellow at the USC Annenberg
School for Communication. Bollier is also co-founder of Public
Knowledge, a Washington, D.C., organization that advocates for the
public s stake in the Internet and copyright law, and the author of
Silent Theft, Brand Name Bullies, and four other books. He lives in
Amherst, Massachusetts. VENUE: Courant Institute
251 Mercer Street (Warren Weaver Hall)
Room 109

Apparently Comcast is planning on axing its analog cables at some undetermined point in the future. That will make my MythTV setup more complicated, and no doubt require a more expensive cable plan and yet more equipment. My setup is already a tad ridiculous.

Providing city services only to users of specific proprietary software is bad public policy. I just discovered that New York City's 2009 Summer Youth Employment Program requires Internet Explorer in order to apply online. Even downloading the pdf version of the application to print out from the site is impossible under non-IE browsers because the actual pdf link is wrapped in some IE-only javascript. And even if you could download the PDF directly, any additional "online information to help you select a SYEP provider" is inaccessible for the same reason. I just called 311 and filed an official complaint against the NYC Dept. of Youth and Community Development (DYCD), who sponsor the program. I also called the number on the application page (1-800-246-4646), and spoke with "Karen" from the DYCD, who explained that the site was a New York City web site, and that it had been created by DoITT (the Department of Information Technology and Telecommunications) at the request of the DYCD, but that the DYCD didn't program it directly. She seemed to misunderstand the tech behind the situation, saying "we can't bring it to a higher level (like Firefox) because then it wouldn't work for everyone". I was happy that she understood that Firefox was a concern here, but the point is not to build the site "higher" to Firefox, but to use standard technology that all browsers can access for a public site. Karen also seemed to think the situation was acceptable because the city youth "can always use IE on local public library computers" to access the site. Note that the applications involve submitting very detailed information (SSN, health insurance, family income, criminal record, selective service registration #, etc), which are things that i would personally be unwilling to submit over a shared public computer if i had any other choice. Furthermore, this crappy implementation decision encourages the NYPL to continue to spend limited resources on proprietary software to an out-of-state monopolist to run their computer labs, which is money that could be better spent locally (or even spent on books or something similarly quaint and library-like). I'm frustrated. This is 2009. The application process for public services here should not require any proprietary technology, but it uses it gratuitously. This excludes legitimate citizens, and encourages Microsoft in its ongoing pursuit of monopoly status. Both of these are bad things. I submitted feedback on the DYCD customer survey web site, and submitted two 150-word-limited(!) complaints to Commissioner Mulgrav of the DYCD and Commissioner Cosgrave of DoITT. I'm sure they'd be interested in hearing from other people about this. Is this kind of proprietary lock-in what we should expect from a Mayor who cuts budgets city-wide except for IT? Where is all that money going? What is the city getting out of it?Tags: policy

It took me a while because of travel and other projects, but I finally released a tar ball containing Debian lenny for the SheevaPlug along with some installation instructions. It should be really easy to install Debian this way, but I'll also make sure that SheevaPlug support will be in the Debian installer for the next release of Debian. For now, check out the tar ball I prepared!

Cupt is experimental re-implementation of APT suite from scratch using Perl .
It consists of Perl modules and console front-end to them.

Why?

- to finally avoid some bugs in APT design;
- to introduce some useful features;
- to make an extensible and readable codebase;


What infrastructure does Cupt use?

It uses the same APT infrastructure, e.g. index files, deb cache archive
files, configuration files. It understands some of widely used APT options.


What useful features has Cupt already?

- full-case strict dependency problem resolver;
- command-line and APT-like option name checker;
- case-sensitive search;
- pinning by source package name;
- pinning by package groups using shell-like patterns;
- configurable 'depends' and 'rdepends' subcommands;
- 'satisfy' subcommand.

What features will Cupt have in future?

See incomplete roadmap.


Why is it 'experimental'?

Because not all important functionality is implemented yet:
- 'update' action;
- cooperating with debconf;
- working with source packages;
- translated package descriptions;


Why Perl?

- I like Perl
- code conciseness
- code extensibility
- several useful libraries available


Can I use Cupt along with APT?

Yes, you can mix apt-get/apt-cache/aptitude/etc. with cupt without bad consequences.


Can I build a Debian package for Cupt?

Yes, go to repository, checkout it and build the Debian package as usual.


What's the status of Cupt in Debian?

It's currently in the NEW queue, but don't rush to try root-required actions because it's still experimental.


Who am I?

The man who had been contributing to the APT.

dmonitor now has a webpage. I've been running it for a night now, watching alerts come and go via manual firewall rules so I'm pretty confident it works reliably and in a way that avoids transient failures. The only obvious failure case is if each monitoring node loses the links to each of the others. (Solution there is to have a sufficiently large number of them! Hence the reason the configuration is file/directory based. rsync for the win!) Still I will leave it there for now. The only things missing are better instructions, and more service checking plugins.. Now to go paint some more ... adding new wall decorations has inspired me a little! ObFilm: Heathers

Older (up to the *60 series) Thinkpads have a slightly odd mixer setup. There's a mixer stage that sits between the software controllable mixer and the speakers. This is controlled by the firmware in response to the volume keys being hit and can't be prevented by the OS. This is unfortunate, since users are quite enthusiastic about seeing an on-screen display when they hit various keys and if we do the obvious thing and map the volume keys to KEY_VOLUMEUP and KEY_VOLUMEDOWN then they get the OSD but also get their software-controlled mixer changed at the same time as their firmware-controlled mixer. This causes boundless confusion.

Lorne Applebaum recently sent a patch that added an ALSA control to represent the Thinkpad mixer. This means that we can now read and control the Thinkpad-specific hardware without userspace needing any special knowledge. I've gone a little further and sent a followup patch that sends ALSA notifications on button presses. Lennart suggested that it would also be helpful to export dB values and wrote a nice app that measures them - that means that Pulseaudio will be able to set the mixer volume correctly by default.

The only missing part of the puzzle now is getting the OSD back. We still can't map the keys to KEY_VOLUMEUP and KEY_VOLUMEDOWN because that'll still result in two mixers being changed at once. We can't add new keys to indicate that it's a notification event rather than a command, since X can't deal with keycodes greater than 255 (actually 243, since it adds 8 to every kernel keycode for historical hilarity) and we've run out of space. We can't hook off the ALSA notifications because that'll result in the OSD popping up in response to someone dragging a slider in a volume control application. The easiest thing might be to add another ALSA notification that's only triggered if the notification was generated in the kernel and then have userspace listen for that.

Newer Thinkpads appear to have done away with this mixer setup entirely and just send standard volume events. This is a Good Thing.

In other news, I'm off on holiday to Australia for a couple of weeks so may be a bit slow in replying to things.

As some of you may know, Arthur Liu did the initial work on a GTK+ interface for aptitude last summer as part of the Google Summer of Code program. Since that program ended, I've been working to complete the GTK+ interface, along with some unrelated but similarly large changes to other parts of aptitude. It's been a while since I've written anything about aptitude-gtk, though, mostly because I set it aside for a few months to improve the dependency solver. While I'll probably be doing more work on the backend over the summer, I also hope to get the GTK+ interface to a point where I can feel comfortable including it in a stable release of aptitude. There are two things that I want to be in place for the GTK+ interface before I release it:

1. The interface should be functionally complete, meaning that you should be able to do anything you can do in the other interfaces.
2. The interface should be reasonably straightforward to use and nice to look at. This is a never-ending task, but there are some pieces of the current interface that are particularly ugly or awkward that I'd like to fix before release.

I've gone through the major areas of the current UI and marked them up with notes about things that I think need to be done. This isn't a definitive or exhaustive list, but it should give some idea of where aptitude is and where it's going.

---

Dashboard Tab The first thing you see when you start aptitude is the dashboard. This is a display that gives you immediate access to some of the most common things you'll want to do: searching for a package, viewing the available upgrades, and upgrading as many packages as aptitude can figure out how to upgrade without removing anything. This is one of the most complete parts of the interface, but there are a few minor aesthetic things that I'd like to take care of. [UPDATE] Michael Johnson writes:

I agree with most of the notes on this screen. However, I'd like to see the Selected Package changelog stay. I often look more than one upgrade back. The summary doesn't show more than what's being upgraded. However, I do like the idea of package description tab. The search box should definately always be available. I found it annoying when I had to click back to a different tab to get it. One thing I definately miss from the console version of aptitude is the list of recommended and suggested packages. I don't let aptitude auto-install recommended packages, but I like to know what they are.

---

Packages tab This is the view that's shown after you search for a package by name, keyword, or by other criteria. It's also fairly complete, but there are more things that could be added (mostly enhancements). Related to this is the dialog box for editing which columns are visible:

---

Dependency solver tab The dependency solver got a lot of love in the 0.5.2.1 release, and it's now basically functional. There are some minor layout issues to be fixed, and if I have time I'd like to add more ways of controlling the resolver's behavior.

---

Package information tab The package information page is probably the weakest part of the GTK+ aptitude interface; it wouldn't be too much of a stretch to say that this tab is holding back a stable release of aptitude-gtk. The big problem I have with it is that I feel that the information tab should open up with an overview of the status of the package. What version is installed, what's available, what archive is it in, who's the maintainer, how big is it, etc. Instead, aptitude-gtk currently shows you a big tree of all the package's dependencies. While that's useful, it should be something the user chooses from. I also find this tab to be visually unappealing, but I'm not entirely sure how to fix that. [UPDATE] Gunnar Wolf writes:

As you have mentioned in your blog post (quote this mail if you so wish) that you want to make the package information screen "sexier": I think it would be natural to integrate screenshots (from http://screenshots.debian.net/) into it. Of course, it would be worth measuring whether it imposes too high a load into that system (maybe it should only get a screenshot if the user explicitly asks for it) - but surely would be a net win.

I think this is a great idea and I'm definitely planning to incorporate this feature. [UPDATE] Michael Johnson writes:

One of the things I really like about aptitude-gtk over console-aptitude is how obvious it makes some things. The version display on this tab is one of those things. Maybe it could be more obvious, I don't know. I agree, the dependency list is not the most important thing. And it's missing the reverse dependencies. I use that more than the dependencies. As far as the actual display of the list, I can't think of anything besides a tree. Although the current implementation does feel quite clunky. What I'd more like to see first is the list of files in the package. But I can see how other people would have different priorties. So it should be really easy to change the default view.

---

Preferences tab One of the reasons that it is a stretch to say that the information tab is holding aptitude-gtk back is the preferences tab, which is a huge pile of not-yet-implemented-ness. In fact, it's not even designed, let alone implemented. Releasing a GUI program with no way to alter the settings strikes me as ... silly.

---

Preview tab The preview tab is a bit weaker than the packages tab, in my opinion, but much stronger than the information tab. Like the information tab, I find it visually unappealing, but I'm not entirely sure how to improve it. I'm currently leaning towards the paned browser idea. [UPDATE] Michael Johnson writes:

I've never been comfortable using this tab. It never made much sense to me. I think the last one or two times I used aptitude-gtk it started making sense, but that seems to be too long a time. I think it's the mass of text at each top level entry. The console version has brief descriptions with more info in the details pane when the entry is selected. Perhaps healthy use of tool tips or an info pane would help with this. The paned browser idea sounds promising. I think it will be easier for new users to follow. And it provides a natural way to provide more information on what each section is about.

---

That's all folks!

My Desktop

I use a number of computers in my daily life, but the machine I use the most often is my "desktop box". This is one of a pair of machines sat side by side on my desk. One machine is desktop (Sid) and one is the backup host (Lenny). The backup machine is used by random visitors to my flat, and otherwise just runs backups for my remote machines (www.steve.org.uk, www.debian-administration.org, etc) every 4/6/24 hours. I apply updates to both boxes regularly but my desktop machine tends to have lots of browsers open, and terminals. I rarely restart it, or logout. So the recent updates to X, hal, udev, mostly pass me by - I can go months without logging out and restarting the system. On Saturday the desktop machine died with a OOM condition when I wrote some bad recursive code for indexing a collection of mailboxes. Oops. When it came back I was greeted with a new login window, and all the fonts look great. Now in the past the fonts looked OK, but now? They look great. I cannot pin down what has changed precisely, but everything looks so smooth and sexy. So, a long entry, the summary is "I restarted my machine after a few months of being logged in, and now it looks better".

Distributed Monitoring?

Random conversation with Alex about monitoring yesterday made me curious to see if anybody has put together a useful distributed monitoring system? Assume you have a network with Nagios, or similar, monitoring it. If your link between the monitoring box and the hosts being checked is flaky, unreliable, or has blips you will see false positives. We've all been there and seen that. So, what is the solution? There are two "obvious" ones:

• Move the monitoring as close to the services as possible.
• Monitor from multiple points.

Moving the monitoring closer to the services does reduce the risk of false positives, but introduces its own problems. (i.e. You could be monitoring your cluster, and it could be saying "MySQL up", "Web up", but your ISP could have disconnected you - and you're not connected to the outside world. Oops. The solution there is to test external connectivity too, but that re-introduces the flakyness problem if your link is lossy.) Distributed monitoring brings up its own issues, but seems like a sane way to go. I wrote a simple prototype which has the ability to run as a standalone tester, or a CGI script under Apache. The intention is that you run it upon >3 nodes. If the monitoring detects a service is unavailable it queries the other monitoring nodes to see if they also report a failure - if they do it alerts, if not it assumes the failure is due to a "local" issue. There is a lot of scope for doing it properly, which seems to be Alex's plan, having the nodes run in a mesh and communicate amongst each other "Hey I'm node #1 - I cannot see service X on host Y - Is down for you too?" - but the simple version of having the script just do a wget on the CGI-version on the other nodes is probably "good enough". I really don't track the state of the art in this field, just struggle to battle nagios into submission. Do there exist systems like this already? (sample code is here, sample remote-status checks are here and here. Each node will alert if >=2 nodes see a failure. Otherwise silence is golden.)

ObFilm: X-Men Origins: Wolverine

I ve just bught an NVidia video card from Computers and Parts Land (CPL) [1]. I asked for a PCIe*8 card but was assured that a PCIe*16 card would work. However when I got it home I discovered that it would not fit, the size difference was obvious enough that I didn t even need to unwrap the new card. According to the Wikipedia page for PCIe (PCI Express) [2] it is quite legal to have a motherboard implement a slower version of PCIe but have the full slot width, so a motherboard could have PCIe*1 implemented but have a socket that will fit a PCIe*16 card. The motherboard and the card will negotiate the capabilities that they both support. But there is no requirement to have a larger socket than required to use the capabilities of the motherboard. The CPL salesman assured me that any PCIe*16 card will fit into any PCIe*8 socket. I presume that he is used to white-box gear manufactured by companies that can save a few cents on the sockets if they make them all the same size. The Dell PowerEdge T105 that I own has PCIe*8 sockets that do not fit PCIe*16 cards. So now I have to return the card, and the CPL web site doesn t even give the hours that the store is open, so I have to call them. Sigh. Update: I returned the video card, first I had a debate with the CPL guy who had sold it to me regarding whether such PCIe sockets exist. He suggested that I had mistaken a PCI slot for a PCIe slot and was not convinced until I showed him a picture I had on my phone. He then claimed that Dell made sub-standard machines for not using a PCIe*16 socket for PCIe*8 connectors - so it was Dell s fault not his fault for delivering me a PCIe*16 board when I asked for a PCIe*8. He told me that no-one had ever made a PCIe*8 video card. Then he told me that I was lucky to not have a re-stocking fee (they sold me a device that was not what I asked for, I returned it in perfect condition on the same day and I was lucky to not pay them for the privilege). They gave me a credit note for the value of it (not a refund). I recommend never doing business with CPL if you can avoid it. I will make exactly one more purchase from them, that will be to use the credit note.

• [1] http://www.cpl.net.au/


• [2] http://en.wikipedia.org/wiki/PCIE

Bruce Schneier has written about the foolish actions of Justice Antonin Scalia [1]. Antonin made some comments opposing the need for greater privacy protection, most people could get away with doing that, but when a Supreme Court Justice does so it gets some attention. In response to this Fordham University law professor Joel Reidenberg assigned his class a project to discover private information on Antonin using public sources. The class produced a dossier of such information which was then offered to Antonin [2], but which was not published. Now anyone who knows anything about how the world works would just accept this. Among other things Antonin now knows what is publicly available and can take steps to remove some public data according to his own desires. But being apparently unaware of the Streisand effect [3] Antonin went on to say the following: It is not a rare phenomenon that what is legal may also be quite irresponsible. That appears in the First Amendment context all the time. What can be said often should not be said. Prof. Reidenberg s exercise is an example of perfectly legal, abominably poor judgment. Since he was not teaching a course in judgment, I presume he felt no responsibility to display any. This is of course essentially issuing a challenge to the entire Internet to discover the information that the Fordham students discovered. Of course doing so would not be fun unless it was published. The meme of 2009 has yet to be defined, it might be discovering and widely publishing personal information about Antonin. Already one of the comments in Bruce Schneier s blog suggests that activists should do such research on all senior figures in the US government to encourage them to take privacy more seriously. I expect that the first reaction of the legislative branch to such practices would be to enact special laws to protect their own privacy while still allowing large corporations (the organisations that pay for the election campaigns) to do whatever they want to ordinary people. It s an interesting situation, I predict that Antonin will regard this as one of the biggest mistakes he s ever made. I m sure that there are many more LULZ to come from this.

• [1] http://www.schneier.com/blog/archives/2009/05/googling_justic.html


• [2] http://www.concurringopinions.com/archives/2009/04/justice_scalias_2.html

<:LI>[3] http://en.wikipedia.org/wiki/Streisand_effect

Wednesday 1st

At the G20 protests, Londoner Ian Tomlinson collapses and is attended to by the emergency services; Metropolitan Police claim "during this time a number of missiles - believed to be bottles - were being thrown at them". Tomlinson dies that evening.

Thursday 2nd

The Evening Standard reports that "police were bombarded with bricks, bottles and planks of wood" as they helped Tomlinson.

Saturday 4th

Postmortem of Tomlinson finds he died of a heart attack.

Monday 6th

The EC Data Retention Regulations 2009 comes into force requiring internet service providers to retain details of emails, internet phone calls and other web traffic.

Tuesday 7th

The Guardian publishes a video of a police officer assualting Tomlinson on 1st April.

Wednesday 8th

Twelve Pakistani students are arrested on suspicion of planning terror attacks over the Easter holiday weekend.

Thursday 9th

The Guardian publishes a second video of Tomlinson's assualt, contributing that no missiles were thrown.

Monday 13th

Police arrest 114 climate protestors on conspiracy to cause criminal damage at Nottingham power plant. [ref]

Tuesday 14th

A video of a seperate assualt at the G20 protests is published.

Thursday 16th

Police claim tourist's photography of transport-related buildings is "strictly forbidden" and demand the photos are deleted. [ref]

Friday 17th

A second postmortem of Iam Tomlinson finds the cause of death to be an abdominal haemorrhage. A policeman is interviewed under caution on suspicion of manslaughter. [ref]

Saturday 18th

The Times reveals MP Damian Green private emails were searched by the police using the keywords "Shami Chakrabarti" (director of Liberty) after his arrest last year.

Monday 20th

The Guardian reveals police intelligence was given to E.On before the Nottingham climate demonstration, including movements of protesters and their meetings.

Tuesday 21st

Another video of the Tomlinson assault is revealed showing his head hitting the pavement. The Independent Police Complaints Commission fails to obtain an injunction to prevent broadcast.

Wednesday 22nd

Nine Pakistani students arrested during anti-terror raids are released after 13 days without charge and are passed to UK Border Agency custody for deportation. [ref]

Monday 27th

Home Secretary Jacqui Smith announces that ISPs and telecoms companies will be asked to record usage "to track murderers and paedophiles" without a warrant. [ref]

Tuesday 28th

Emails are revealed of the Home Office asking Phorm, the behavioural targeted advertising service, whether they are falling outside the law. Home Office claim Phorm would be legal if consent is obtained and that "providers of goods and services need only ensure they are compliant with relevant legislation". [ref]

Wednesday 29th

The Guardian documents the friends of the 7/7 bombers were denied bail on secret evidence - "the appellant asked the judge the question: why are you sending me to prison? To which the judge replied: 'I cannot tell you that.'"

Spring has arrived in Europe and I'm looking forward to spending more time outdoors. While I was pondering some plans, I thought it would be nice to have a GPS. I've never used a GPS before but it seems it will allow me to combine outdoors activities such as walking and hiking with geek things. In particular, I'm interested in tracking where I'm going (what route, what distance, what speed, etc), geotagging photos and possibly contributing to OpenStreetMap. Since last week, I have a Qstarz BT-Q1000X (Travel Recorder X), a GPS logger and receiver, and I've been playing around with it a bit. This device uses the new MTK II chip, which gives good accuracy and long battery time. I've created a new gps category in which I'll share my experience using a GPS in general and specifically the Qstarz BT-Q1000X. I've also created a Qstarz BT-Q1000X and Debian page to describe how to use this GPS device on Linux and I'll add more information to the page as I try out different applications.

I got a summer internship at Google in Z rich, Switzerland, so I ll be moving there this summer. Any DD working there ? I already went to Z rich once last year and it was quite a cool place. I briefly met Cate and I know that there are other DDs there (hi Madduck!). I look forward to meeting more of you. Now there are two issues I d like help with, dear Z richer Lazyweb, housing and flight to DebConf. Can I haz apartment Looking for an apartment in Z rich seems very very very hard.. So far I only found a few apartment/hotel geared towards rich expats staying for few months (the kind like Citadines). It s ungodly expensive (like 2200+ CHF a month for one room), but sure available and well placed.. Here s what I need:

• from beginning of June to end of August (I have a friend willing to let me crash in so middle of June is okay)
• either one room (for myself) or up to 2 or 3 (I d be flatsharing then)
• about or less than ~1500 CHF for one room, or less than ~2000 CHF for two and ~3000 CHF for three, you got the idea
• reasonably close transport-wise from Enge, 20-30 minutes by public transport or, even better bike, would be cool

I don t have much more criterias, as long as I can find something. We re about a few dozens Google interns all (desperately) looking for accomodation in Z rich. We re all good neighbours, don t party all night and all have very good references. Please contact me if you \<know someone who\>* know something, I d be very grateful. Can I haz DebConf I m going to DebConf this year in Spain, along with all the Google Summer of Code students who can make it. I m going to reserve my flight in the next few days. I think I ll only go to DebConf proper, arriving on the 23rd and leaving on the 31st of July.

• Here s my tentative travel plan:
  • Swissair flight 2026, 23rd July, Z rich 12:25 - Madrid 14:45
  • Renfe train, 23rd July, Madrid 16:40 - C ceres 20:02
  • DebConf!
  • Renfe train, 31st July, C ceres 5:08 - Madrid 9:43

• Either:
  • Visit Madrid
  • Swissair flight 2033, 2nd August, Madrid 19:45 - Z rich 21:55
• Or:
  • Air France flight 1301, 2nd August, Madrid 12:40 - Paris 14:45
  • Visit home
  • SNCF train TGV Lyria 9217, 2nd August, Paris 17:54 - Z rich 22:26

I m not decided yet on the last point. So, any of you coming to DebConf from Z rich ?

(The much-delayed part 2 in an ongoing series: shiny things in telepathy-glib) Over a year ago I explained my first major round of work on telepathy-glib, with which it could be used to implement D-Bus services. I started to write this post shortly after, but then got distracted by a bee or something; recent discussion about having a D-Bus binding in GLib has finally prompted me to finish it. The second major round of work on telepathy-glib dealt with implementing client code, and went sufficiently well that telepathy-qt4 (currently under development) uses essentially the same model. Again, while telepathy-glib is intended for Telepathy implementors, I think TpProxy's ideas are generically useful, particularly for "large" D-Bus APIs.

Some background: dbus-glib and libtelepathy

The life-cycle of a DBusGProxy (PD clipart credit: karderio and Andy)

The API provided by dbus-glib for client code is, er, rather less than ideal. The basic object is DBusGProxy, which has a few problems. There's one per interface, rather than one per object; this leads to a maze of twisty GObjects, all suspiciously similar. Our old client library, libtelepathy, got round this by subclassing DBusGProxy to make helper objects called TpChan, TpConn etc., which were simultaneously the proxy for the "main" interface (Channel, Connection etc.), and a factory for proxies for the other interfaces (like Channel.Type.Text and Channel.Interface.Password). This still wasn't ideal, but it was a start. If you use a DBusGProxy in any way after it has emitted the destroy signal, this is considered to be invalid use of the API, and dbus-glib asserts all over the place. We'd rather it didn't do that - remote objects becoming invalid is a fact of life, and if you don't immediately discard all your references to those objects, remote operations on them should just give you a runtime error. You have to be prepared to handle such runtime errors already, because any IPC call can fail at any time. The main API for calling methods uses varargs and is rather subtle. Notably, the calling convention for variant parameters (of D-Bus signature 'v') is not consistent with the calling convention for any other container. For any other container, for instance a GPtrArray *, you pass in a GPtrArray ** pointing to a GPtrArray * variable containing NULL; on success, a pointer to a new GPtrArray is written into that variable. For variant parameters, though, you must pass in a GValue * pointing to a zero-filled GValue, and the result will be written into that GValue. dbus-binding-tool generates thin wrappers around this method-calling API which provide type-safety, although they can't directly specify a non-default timeout. As for signals, before binding to a signal, you have to tell dbus-glib about its signature. This tells dbus-glib how it should push the signal arguments onto the C stack (in GObject terminology: how it should marshal them), to match the signature of the signal-handler function. The documentation claims that this isn't necessary for objects supporting the Introspect method, but that's untrue - on the other hand, if the documented behaviour actually happened, I would consider this to be a critical bug in dbus-glib, since it would enable remote objects to crash dbus-glib clients (by causing signal arguments to be pushed onto the stack in a way that does not match the signature of the signal handler). Of course, the signal-adding function takes varargs, and has to be called exactly once per signal per object. If you don't, dbus-glib asserts when you connect to the signal; if you call it twice, dbus-glib asserts anyway.

TpProxy TpProxy is a GObject subclass vaguely inspired by Python's dbus.proxy.ProxyObject. It isn't a subclass of DBusGProxy, which lets us encapsulate dbus-glib almost completely - in fact, all the references to DBusGProxy are in a separate header, proxy-subclass.h, only used by subclasses of TpProxy (as opposed to normal library API users). TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. As explained below, I believe that this is the most natural object mapping for D-Bus interfaces; in practice it tends to lead to clearer code for us, since one remote object often maps nicely to one UI element (of course, if your objects only have one interface, this approach is no better or worse than dbus-glib's). TpProxy provides support for the following common D-Bus things:

• Calling methods
• Connecting to signals
• Objects becoming invalid
• Objects with optional interfaces
• Extensibility

It doesn't currently provide any particular help with Properties - the Properties interface is just another interface as far as we're concerned. We find that, in practice, that's not a problem (even though we use D-Bus properties extensively). In telepathy-qt4 we basically reinvented TpProxy in C++, as Tp::DBusProxy. I've put in some comments below illustrating how the general ideas we used in TpProxy translate into a different object mapping.

Calling methods Method calls on TpProxy are done through auto-generated wrapper functions, much like DBusGProxy. These look something like this:

typedef void (*tp_cli_channel_callback_for_close) (TpChannel *proxy,
    /* any 'out' arguments would go here - Close() doesn't have any */
    const GError *error, gpointer user_data,
    GObject *weak_object);
TpProxyPendingCall *tp_cli_channel_call_close (TpChannel *proxy,
    gint timeout_ms,
    /* any 'in' arguments would go here - Close() doesn't have any */
    tp_cli_channel_callback_for_close callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object);

Methods for some interfaces, like DBus.Properties, are available on TpProxy itself; others, like Telepathy's Channel.anything, are only available on a particular subclass, like TpChannel. Things to note here:

• It's an asynchronous call with a callback. General design principle: "this is IPC, get over it". We don't try to hide the fact that IPC is taking place by doing pseudo-blocking calls.
• The timeout is explicit (although you can always pass -1 to let libdbus pick a "reasonable" default for you). This is because the remote service might not respond, and API users should be at least vaguely aware of this: "this is IPC, get over it".
• The callback isn't a GCallback; it has an explicit signature, for some semblance of type-safety (it's not much, but it's better than nothing).
• The callback takes a GError, because any IPC call can fail for any reason.
• A TpProxyPendingCall * is returned. This is a pointer to an object that only exists until just after the callback is called (so you can ignore it if you don't want it), with a method you can call to cancel the method call (with hindsight this is not ideal, and if I was writing this API now, I'd use gio's GCancellable). "Cancel" is perhaps slightly misleading: the method call still takes place (it was already in libdbus's outgoing queue) but the result gets ignored and the callback isn't called. This is mostly so you an object can forget all the calls it was busy making at the time that the results become irrelevant (e.g. the object is destroyed).
• There is an extra weak_object argument, which is weakly referenced and is passed to the callback: in practice, most users of telepathy-glib use this as well as or instead of user_data. If the weak_object runs out of references, the callback is automatically cancelled, which means that in practice, explicit cancellation is almost never needed.
• The generated code has as its first argument a suitable subclass of TpProxy, in this case TpChannel (we tell the code generator about subclasses).
• You can't see it in the API, but the TpProxy gains an extra ref while a method call is in progress, so you never have to worry about what happens if the proxy gets unreffed during a method call.
• There's an explicit destructor for the user_data, which is called even if you "cancel" the method call.

In telepathy-qt4, these methods are on small generated helper classes similar to dbus-python's dbus.Interface, one per interface (we provide accessors for them on the non-generated DBusProxy subclass). Instead of using callbacks, we return a temporary QObject per call, which emits a Qt signal on success or failure (that's how all async calls in QtDBus work).

Connecting to signals Similarly, signals have some generated functions:

typedef void (*tp_cli_channel_signal_callback_closed) (TpChannel *proxy,
    /* any arguments would go here - Closed doesn't have any */
    gpointer user_data, GObject *weak_object);
TpProxySignalConnection *tp_cli_channel_connect_to_closed (TpChannel *proxy,
    tp_cli_channel_signal_callback_closed callback,
    gpointer user_data,
    GDestroyNotify destroy,
    GObject *weak_object,
    GError **error);

API notes for these:

• The callback doesn't take a GError any more, because signals can't fail; the weak_object and the user_data are the same as for method calls
• This isn't actually a GObject signal at all: those aren't particularly typesafe, and lack namespacing. This might not be very binding-friendly, we haven't tried doing that yet... it might be worth introducing a signal with details, like in dbus-glib, for which these functions are "C bindings".
• The returned TpProxySignalConnection lets you disconnect from the signal, just like the TpProxyPendingCall above; it can safely be ignored
• Similarly, if the weak_object dies, then the signal automatically disconnects
• It is possible for connecting to a signal to fail: this happens if the proxy doesn't actually have the requested interface. If this happens, it's graceful, not a crash. In practice, the flow of code is almost always such that error can safely be NULL because the interfaces are already known.

In telepathy-qt4 the per-interface generated helper classes emit Qt signals. In many cases, we end up proxying these signals by responding to them in the hand-written Tp::DBusProxy subclass by emitting a different signal, in order to give them a nicer representation.

Becoming invalid This is IPC, and any error can happen to you at any time. We found that it was useful to have a general concept of "this object is no longer useful", so we introduced the concept of an object becoming invalidated. The invalidation reason is stored on the object as a GError. If this GError is NULL (as it is initially), then the object is still expected to be valid; if it's non-NULL, the remote object has vanished and the local proxy no longer works. There are several ways a TpProxy can become invalid:

• its bus name falls off the bus (the process exited or crashed)
• application-specific reasons (usually triggered by a D-Bus signal in a subclass, like the Telepathy Channel's Closed signal)
• the GObject gets disposed

Method calls on an invalidated object always fail, with the invalidation reason as the error. This means that calling a method on a TpProxy should never crash your process. The method's callback has to be prepared to handle an error in any case, because this is Sparta^WIPC, so there's no loss in giving it another error. Connecting to signals on an invalidated object always fails, and any existing signal connections are disconnected when it becomes invalidated. This means that if a new remote object appears, and it happens to have the same object path, bus name etc. as the old one, you won't get its signals. A TpProxy can either be bound to a unique bus name or a well-known bus name. In the equivalent code in telepathy-qt4, we call this "stateful" or "stateless" - these names don't capture the intention perfectly, but they're the best we could come up with. A proxy for a stateful API like Telepathy's Channel should always bind to a unique name - when the unique name falls off the bus, the TpProxy becomes invalidated automatically, representing the fact that the Channel you had no longer exists, and nothing can be a perfect replacement for it (the VoIP call has already been terminated, you've already left the chatroom, or whatever - if you make another Channel, that's e.g. a distinct VoIP call). A proxy for a stateless API like Telepathy's ConnectionManager, where an exiting process can disappear, be service-activated again, and still have exactly the same API, should bind to the well-known name. Proxies for well-known names aren't invalidated when the process exits. Qt doesn't have a natural equivalent for GError, so the invalidation reasons in telepathy-qt4 are just a pair of strings, the namespaced name and the message - basically a libdbus DBusError. The principle is the same, though.

Optional interfaces As mentioned above, TpProxy instances are per-remote-object (per-object-path) like in dbus-python, rather than per-interface like in dbus-glib or QtDBus. I believe that this is the most natural object mapping for the D-Bus object model, because the D-Bus interfaces on a remote object can behave like any of these:

• classes and subclasses (Telepathy.Channel.Type.Text is a Telepathy.Channel)
• shared interfaces (some Telepathy.Channel objects, of many Types, implement Telepathy.Channel.Interface.Group; any D-Bus object can implement DBus.Properties)
• optional features (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.ChatStates, some do not)
• discoverable extensions (some Telepathy.Channel.Type.Text objects implement Telepathy.Channel.Interface.Messages, which is more or less Text 2.0)

The way in which interfaces are discovered is also variable. The "classic" D-Bus way to discover interfaces would be to call Introspect. Telepathy services still support that style of introspection, but we don't use it for anything beyond d-feet:

• Introspect returns a blob of XML, which you have to parse into something useful; if it was in a D-Bus data structure, you'd already have some sensible data structure.
• Finding out more information about the supported interfaces than their names isn't very useful: in a static language like C you already need to know the methods, signals, properties and their signatures in order to write and compile your code, and in a dynamic language like Python consumers of Introspect end up being remotely crashable by services with unexpected method signatures (this is one of dbus-python's big mistakes in my opinion).
• If a method that you want to call turns out to be missing, the worst that can happen (in a competently written service) is that it fails with a D-Bus error - and you have to handle D-Bus errors anyway, because this is IPC and anything could fail.
• Static bindings like dbus-glib have poor support for removing interfaces for which C code exists at runtime, whereas Telepathy needs features like an IM connection that might or might not support avatars (we don't get to find out until we've successfully connected to the server).

Instead, our older interfaces have a method on the "base class" (Channel, Connection etc.) called GetInterfaces, which just returns an array of strings. Newer interfaces (like Account), and older interfaces that have been ported (like Channel), have a property called Interfaces which is, again, an array of strings - this lets us combine the download of the interfaces list with downloading other basic information in a GetAll call. To cater for all these ways to use interfaces, the TpProxy base class has very basic support for interface discovery - you can ask it whether it supports an interface, and you can tell it that it does, in fact, support an interface. Asking which interfaces are supported is directly useful for library users; it's also used as a check by the generated method-call and signal-connection stubs, which return a canned error (Telepathy.Error.NotImplemented in Telepathy's case) if the object isn't known to have the interface. Telling TpProxy that it does support an interface is intended for use by subclasses, and might have been protected if we were writing C++: what happens in practice is that a concrete subclass like TpChannel knows how to discover the fully supported interfaces for this particular object, does so, and calls methods on the TpProxy to tell it which interfaces can work.

Extensibility The API stubs used by TpProxy to call methods and bind to signals are generated from XML documents containing an augmented form of D-Bus introspection data. These XML documents contain various Telepathy-specific extensions, but none of the extensions are language-specific - telepathy-glib, telepathy-python and telepathy-qt4 all operate from exactly the same XML specification. telepathy-qt4's different object-mapping did require us to tighten up some rules that were previously only conventions, but even so, the format is the same. I think this is vitally important - it just doesn't scale to have one set of language-specific annotations in a D-Bus API for each language that will have bindings. One thing that we make heavy use of is what I call "Ugly_Case", which is camel-case with underscores at word boundaries. This gives us a simple, unambiguous and foolproof rule to use in code generators whenever we generate any of the popular conventions for identifiers:

• CamelCase: delete underscores
• javaCamelCase: force everything before the first underscore to lower case, then delete underscores
• lower_case: force everything to lower case, retain underscores
• UPPER_CASE: force everything to upper case, retain underscores

dbus-glib, by contrast, applies a complex and subtly buggy algorithm to the CamelCase D-Bus names, which results in our SendDTMF method being mapped into GLib as send_dt_mf, and requires that telepathy-glib's Python reimplementation of dbus-binding-tool uses a bug-for-bug compatible implementation (we ended up reimplementing dbus-binding-tool in order to generate the TpProxy-based bindings). Our code generation tools are still rather ad-hoc, because so is our spec format, but for draft interfaces it's possible to copy them into individual projects and use them to generate additional methods for TpProxy or any subclass. This is how we implement unfinished APIs like geolocation in Empathy, for instance. The Interfaces or GetInterfaces hook described above is entirely usable for these extension interfaces, and in fact that's how they're set up. In telepathy-qt4, the generated method stubs are genuine C++ methods, so we can't just append methods to an existing class: this is why we have one helper class per interface, so individual clients can generate a helper class for their particular version of an unfinished API, and instantiate an object of this class attached to a particular Tp::DBusProxy. When an interface becomes stable, it can be added to the set of interfaces for which code is generated in telepathy-glib or telepathy-qt4, at which point any client or service that was already using the final draft of that interface can easily be ported to the library version using sed (the API remains the same).

Becoming ready The concept of being "ready" does not directly exist in TpProxy, but is implemented in TpChannel, TpConnection and TpConnectionManager. The idea is that a freshly created proxy object isn't really fully usable yet - you have to connect to basic signals and recover the initial state of the remote object, as well as checking which extension interfaces are supported. In many well-designed D-Bus APIs you can do this initial setup with a few signal connections and a DBus.Properties.GetAll call. After doing that initial setup (which has to be done asynchronously, because it's IPC), the TpProxy subclass has a local cache of the remote object's state (not necessarily in the same format as the representation on the bus), which can be accessed synchronously at any time, and will be updated in response to change-notification signals. TpChannel and subclasses like it have explicit support for checking for readiness, and having a callback called when the object is ready or invalidated (whichever happens first). The "ready" property also supports the GObject notify signal. telepathy-qt4 took this concept and ran with it - there is library support for objects that can become ready, including a mixin. If I had as much time to work on telepathy-glib as I wish I had, this would be one of the first telepathy-qt4 features to be added to telepathy-glib.

Features becoming ready Going beyond that, in an extensible framework like Telepathy it's probable that not every client understands (or wants to understand) every feature of every object, so it's highly inefficient for every proxy object to subscribe to all the change notification signals and cache all the remote state on the off-chance that the library user wants them. In the API for TpContact (which is not actually a TpProxy for various reasons) we introduced the concept of optional features; telepathy-qt4 extended this idea throughout the library. The idea is that each library user knows which of the optional features are "interesting" to it, and makes a single asynchronous call (which expands into a series of D-Bus calls inside the library) to download the state for core functionality plus all of the selected features, and subscribe to change notification for all of those. This is another telepathy-qt4 thing that telepathy-glib still needs to catch up with; until someone works out how to make the clone() syscall apply to programmers, I fear all libraries are doomed to lag behind how their designers want them to look :-) (license: CC-BY-3.0 / LGPL-2.1+)