The Age has an interesting article about how Apple supposedly killed the Netbook [1]. It s one of many articles with a similar spin on the news that the last two companies making Netbooks are going to cease production. The main point of these articles is that Apple decided that Netbooks were crap and killed the market for them by producing tablets and light laptops that squeeze them out of the market. Is the Macbook Air a Netbook? According to the Wikipedia page the Macbook Air [2] weighs 1080g for the 11 version and 1340g for the 13 version. According to Wikipedia the EeePC 701 (the first EeePC) weighs 922g and the last EeePC weighs 1460g [3]. The last EeePC produced is heavier than ANY Macbook Air while the first (and lightest) EeePC is only 158g lighter than the 11 Macbook Air. The 11 Macbook Air is 300*192*17mm (979cm^3) in size while the EeePC 701 is 225*165*35mm (1299cm^3) and the biggest EeePC was 266*191*38mm (1931cm^3). So the 11 Macbook Air is 13% wider than the widest EeePC but takes less volume than any EeePC. The 13 Macbook Air is 325*227*17mm (1254cm^3) which is still less volume than any EeePC. The Wikipedia page about Netbooks defines them as being small, lightweight, legacy-free (in terms of hardware not software) and cheap [4]. The Macbook Air clearly meets all the criteria apart from price. The Apple US web site offers the version of the 11 Macbook Air with 64G of storage for $999 with free shipping, for comparison the EeePC 701 was on sale in stores for $500 in 2008. The CPI adjusted price for the EeePC 701 would be at least $550 in today s money. The Macbook is a bit less than twice as expensive as the EeePC was, but that s more of an issue of Apple being expensive a few years ago companies like HP were also selling Netbooks that were more expensive than the EeePC. Unless having an awful keyboard is a criteria for being a Netbook I think that the Macbook Air meets the criteria. As an aside, a relative recently asked me for advice on a device that is like a Macbook Air but cheaper. Does anyone know of a good option? Is Netbook Production Ceasing? Officeworks currently sells an ASUS Notebook that has a 11.6 display and weighs 1.3kg for $398, it s got a metal body that looks a bit like a Macbook Air (which is the latest fashion and is good for heat dissipation). That s not advertised as a Netbook or a Eee product but it s cheap, lighter than the heaviest EeePC, and not much bigger than an EeePC. It seems that the general prices of laptops other than Apple products (which have always had higher prices) have been dropping a lot recently. There are lots of good options if you want a laptop that costs $500 or less. Even Thinkpads (one of the most expensive and best designed ranges of laptops) are well below $1000. Do the Articles about Netbooks Make Sense? The claims being made are that Apple skipped Netbooks because they couldn t make a good profit. This disregards the fact that the iPhone and iPad (which are very profitable) are in the high end of the price range that was occupied by Netbooks. While Apple does make a good deal of money from the iPhone App Market it would be possible to make a Netbook with a lower production price than an iPhone because making things smaller requires more engineering work and often more expensive parts. This also disregards the fact that there are a range devices which work as an iPad case with keyboard, an iPad with such a keyboard meets most criteria for being a Netbook, so Apple is one iPad keyboard device away from selling Netbooks. It s interesting to note that I haven t yet seen an article about the profits from Netbooks which didn t make an issue of the MS-Windows license fees. The first Netbooks only ran Linux but later on they switched to Windows, that had to make a big impact on profits. An article about Netbooks which just assumes that everyone has to pay a MS license fee is missing too much of the Netbook history to be useful. I wonder if anyone could make products that are as profitable as the iPhone and Macbook Air if they had to pay for MS license fees and design their hardware to work with MS software (as opposed to Apple who can change their software to allow a cheaper hardware design). The articles also claim that Netbooks give a bad user experience. When I bought my EeePC 701 it was the fastest system I owned for loading OpenOffice, SSD random read speeds were really good (writes sucked but that didn t matter so much). The keyboard on an EeePC 701 is not nearly as good as a full size laptop but it is also a lot better than using a tablet, I ve used both a 10 Android tablet and an EeePC as a ssh client and there is no comparison. When I m going somewhere that requires random sysadmin work (or other serious typing) and I can t carry much weight then I still take my EeePC 701 and I don t consider taking a tablet. The low resolution of the screen is a major issue, but it s about the same as a Macbook Air so that s not an advantage for Apple. I knew some people who used an EeePC 701 for the majority of their work, I couldn t do that but obviously some people have different requirements. I now use my phone for many tasks that I used to do on my EeePC (even light sysadmin work) so my EeePC sometimes goes unused for months. But it s still an important part of my collection of computers. It works well for what it does and I don t feel any need to buy a replacement. When it wears out I ll probably buy something similar to an 11 Macbook Air to replace it unless there s a good option of a tablet with a detachable keyboard. My plans for computer ownership for the near future are based on a reasonably large Android phone (currently a Samsung Galaxy S3 but maybe a Galaxy Note 2 or similar next year), a small laptop or large tablet with hardware keyboard (currently an EeePC 701), a large laptop (currently a Thinkpad T61), and a workstation (currently a NEC system with an Intel E4600 CPU and a Dell U2711 27 monitor). A reasonably small and light system with a hardware keyboard and solid state storage is an important part of my computer needs. If tablet computers with hardware keyboards replace traditional Netbooks that s not really killing Netbooks but introducing a new version of the same thing. But a good way of getting web hits on an article is to claim that a once popular product is dead.

• [1] http://tinyurl.com/ajohhp3
• [2] http://en.wikipedia.org/wiki/Macbook_air
• [3] http://en.wikipedia.org/wiki/Eeepc
• [4] http://en.wikipedia.org/wiki/Netbook

For some time, I have been concerned about the amount of data I store on Google. This post aims to show alternatives to the service. This is primarily for two reasons. The first is simple. I would like to avoid a single point of failure, and the risk that I could be suspended for a number of months with no recourse. The second if more political. I'm worried about the way that certain governments can subpoena information which they may not be able to collect directly. Now, don't get me wrong. Google is good at transparency. They also provide a very useful service that works incredibly well. In fact, the quality of the integration and services provided is the primary reason I have not changed previously. Calendaring is something that I naturally feel shouldn't be hard, but somehow is. Google Calendaring service is rather good, and far outstrips anything open source I've found so far. However, I've managed to migrate to Zoho which offers a very similar level of functionality. It's also very easy to extract your entire calendar from Google, which makes the transition easier. Reader is a service provided to subscribe and read RSS feeds. In the past, I've used Gregarius as a self hosted solution, but this seems to have disappeared recently. I briefly flirted with NetVibes, which works well once you turn off the frustrating 'widget' mode, but has completely failed to address the need to have native tablet apps. In the end, I settled for NewsBlur which also has the clever use of hotkeys for navigation which I found essential in Google's products. Picasa has stood me well for hosting photos for some time. And photo hosting is something I have quite a large amount of professional experience with :) Despite having previously been in competition with them, I can heartily recommend Flickr, even with it's dogmatic reliance to creating a Yahoo! email address for each account. Search still has some way to go to find a viable replacement. I've tried DuckDuckGo, but found it lacking. Much like the mistake that Apple seem to have made with the launch of Apple Maps, there seems to be a misunderstanding that search is actually HARD.
There are two camps when it comes to searching. You can be dumb, or you can be smart. By being dumb, the user can provide enough information to help narrow the results down to a single (or at least manageable) result set. This works fine, and is a traditional model. The opposite is to be smart. Simply get the results right, using a variety of metrics to work out what the user wanted in the first place. This is obviously more difficult, but infinitely more valuable, and is something that Google excels at. The problem lies when you fall in between these stalls. Unfortunately, it seems that there is no viable alternative to the big G at the moment.

I'm thankfully in a place where I can afford to give money to charity, and I've slowly built up a set of charities that I give money to. But I've never been that systematic about it. I've had a vague target for how much I want to give, I add new charities when they seem neat, I'll rarely drop a charity when it doesn't seem to be effective, and I probably spread my money out too much. Recently, I followed a chain of blog references and discovered GiveWell, a site that's devoted to doing more in-depth analysis of charities than the basic financial analysis done by sites like Charity Navigator. In particular, they do literature surveys to try to dig into the effectiveness of particular interventions and take a closer look at how effectively a charity can use new donations. After reading quite a bit on their site and some related sites, I sat down yesterday and did a comprehensive review of my goals and principles and then put together a new plan. Some of that may be interesting to other people, so I'll write up that process here. This will be somewhat US-centric, since charities vary a lot by tax regime, but I think some of the general principles will still apply elsewhere. (But most of my specific examples will be US charities.) First, a few basic principles about charitable giving:

1. Charity goals cannot be evaluated uniformly. Different types of organizations need different criteria. I've found that charities divide generally into ones that are trying to accomplish a specific task and ones that are trying to create political change. For the former, small and quiet organizations are often the best. For the latter, the charity has to get to a certain size before it's part of the conversation, and that involves a different set of tradeoffs.
2. When you care about efficiency (number of lives saved or quality of life improvement per dollar spent), don't give money to organizations you've heard about via means other than word of mouth or third-party evaluations. This is for a simple reason: if the charity has successfully made you aware of it, that means they're advertising themselves to you. Advertising generally costs quite a bit of money. That's money they're not spending on whatever they're trying to accomplish. Excellent examples are many of the health research and education charities, such as the American Heart Association and the American Lung Association, or the massive disaster relief organizations like the American Red Cross. These are not efficient organizations. If you're trying to get the most impact for your dollar, this isn't a good place to spend it. As a special case of this, please don't ever give money to any organization that produces TV commercials showing starving children in Africa. If the commercial makes you want to give money, that's great; just don't give it to the organization on TV. TV commercials are extremely expensive, and those organizations are notorious for being some of the least efficient charities in existence (in some cases bordering on scams). You'll be lucky if 50% of your money actually goes to Africa, and the money that is spent probably isn't spent wisely.
3. If your donation gets you a membership that comes with a glossy magazine, remember that means that the organization is spending your money producing that magazine. This is counted as program funding, not administrative costs, in sites like Charity Navigator because the charity probably counts the magazine as part of their educational mission. If you want to subscribe to the magazine and enjoy it, there's not necessarily anything wrong with this. But if you were giving them money to do something else, be aware that money is being used to produce the magazine (and the Christmas cards and the mailing labels and the tote bag) instead of doing things that you might consider program funding. For example, I stopped giving money to the Audubon Foundation because I want my environmental giving to go directly to protecting the environment, not towards printing calendars and making a glossy magazine full of bird pictures. As a counter-example, I consider the Southern Poverty Law Center to be one of the most effective US political charities, despite the fact that they produce a glossy magazine. That's because a large part of their mission is investigative reporting and research, and that magazine is how they publish that research. In this case, I both want to read the magazine and consider the magazine something that I'm happy to fund.
4. Efficiency isn't everything. For some types of charities, the charity has to be huge in order to be effective. This is particularly true of political charities. For example, the ACLU is not a particularly efficient charity and has several fund-raising practices that I dislike. However, what the ACLU does requires that they have extensive media access and respect from legislators and the judicial system. It cannot be done effectively by a smaller and more efficient organization. Similar principles apply to a lot of political and environmental charities, and to some international aid charities such as Doctors without Borders (M decins Sans Fronti res).

If you're looking to increase the efficiency and effectiveness of your giving according to objective measures, I highly recommend reading through GiveWell's site. They recommend charities that avoid all the pitfalls above. You've probably never heard of them because they don't spend their money on advertising and marketing. However, keep in mind that efficiency isn't everything. With those principles in mind, and after doing a lot of reading on GiveWell, I redid my charity plan by dividing my giving up into five categories with different goals. Services. There are some things that I use that I think should be paid for by tax dollars but aren't, and are instead funded by charitable donations. I don't consider this charity in the same sense as the rest of this list. Rather, I'm paying for what I use plus some extra so that other people who can't afford it can use the service for free. Efficiency doesn't matter as much because there's usually only one organization supporting the thing that I personally use. Examples include my local PBS (public broadcasting) station, the non-profit that works with city government to support the local library system, and the trust that helps maintain the state parks where I go on vacation. Education. This is a special category for me since I regularly give money to the community college I attended. I know the region and the college and have some idea of what they need and what matters to the students there, so I can make specific choices. I give all the money in this category to the same place. Free software. Here, I know that my donations won't be used as efficiently in terms of quality of life as they could be if sent somewhere else, but this is my community and I want to support it because it's my community. Here, as with education, I have a lot more data and can pick and choose places that I think can use my money. Examples here are the Ada Initiative and the Free Software Foundation. Politics. This is the hardest area for efficiency, since it's so difficult to measure effectiveness in any organization where much of the goal of the organization is to persuade politicians or the general public. I try to measure effectiveness by how many concrete actions the organization takes (rather than press releases), which is easiest for organizations like the ACLU or the EFF that file lawsuits or legal documents and requests. As mentioned above, I think the standout in this area is the Southern Poverty Law Center, but political charities are so targeted that I give to multiple charities to cover my range of interests. I've found environmental charities particularly difficult to evaluate, but the one that appears the most effective to me so far is the Environmental Defense Fund, so that's where I concentrate my environmental money. I used to give to small political charities but have given up on that as basically useless, at least for national politics. I think the charity has to be of a minimum size to get anyone in government to listen to them. Poverty and Health. This is where efficiency matters much more than size. GiveWell's analysis is very interesting here, and sadly shows that most interventions either don't work or at least can't be proven to work. Here, I abandoned nearly all of my traditional large charities and have gone almost entirely with GiveWell's recommendations, with two exceptions. I still cycle money through Kiva (a microfinance support charity) because I have a bit of an emotional attachment to it, although microfinance is looking increasingly suspect as an effective charity method and I may yet drop them. And I still give money to Doctors without Borders because, despite being less efficient, I think their size and history has won them an international credibility that lets them get into areas that smaller, more efficient charities wouldn't be able to help. One of GiveWell's recommendations is my new favorite charity: GiveDirectly. All aid charities run a serious risk of having a colonialist bent, where rich countries come into poor countries and build things or tell them how to do things following the priorities of the rich countries. (This is one of the reasons why I prefer medical charities, since they're less susceptible to this.) GiveDirectly identifies the poorest people in a region (using a very transparent process) and transfers money to them directly using the M-Pesa cell system to spend however they choose, with no strings attached other than some due diligence to protect against fraud. This is refreshingly non-paternalistic and makes me far more comfortable than typical aid projects. Originally, I was going to assign money into those buckets and then break the buckets down further, but I ended up not quite doing that. The services bucket is more driven by the services I use than by an even allocation. Of the rest, it goes approximately three shares to poverty and health, two to politics, and one each to education and free software. I think I've managed to eliminate most of the places where I'm giving money to multiple organizations that do the same thing, but I'm still spreading my money rather widely and could stand to focus it more. That's hard, though; there are a lot of different things that I care about, and political charities in particular aren't very interchangeable. I'm currently not giving locally very much, which might be worth changing. The easiest way to do so would be to find a good local food bank and start donating to them (giving cash, not food). For a more comprehensive list of charities I support, see my charities link page.

git-pbuilder is a shell script suitable for use as the builder command in git-buildpackage that simplifies using pbuilder, cowbuilder, or qemubuilder to build packages in a chroot. It's now part of the git-buildpackage Debian package, but I maintain it somewhat separately. This version supports a new environment variable, GIT_PBUILDER_OUTPUT_DIR, that can be set to override the default output location of ".." (the parent directory). It also changes the script to be a bash script and uses arrays to handle the options passed to the builder command so that shell metacharacters in the GIT_PBUILDER_OPTIONS environment variable will hopefully be handled correctly. As part of that change, it also tries to escape any options that will be passed to --debbuildopts to hopefully protect them from the additional layer of shell parsing that they'll be subject to. Thanks to Guido G nther and paul cannon for the patches. I hate having to make the script bash-specific, but trying to get the shell expansion right is otherwise a huge headache. That probably means that it's outlived the usefulness of the original implementation language (which was shell only because this started life as a quick hack). I should probably take the time to rewrite it in either Perl (which I know best) or Python (which is the language the rest of git-buildpackage is written in). But that's for another day. (Here's an excellent case in point for why one shouldn't embed language-specific extensions in the name of an executable that's part of a user interface or configuration interface.) You can get the latest version from my scripts page.

So we finally made it, and sent out an official announcement for apt.postgresql.org. This new repository hosts packages for all PostgreSQL server versions (at the moment 8.3, 8.4, 9.0, 9.1, 9.2) for several Debian/Ubuntu distributions (squeeze, wheezy, sid, precise) on two architectures (amd64, i386). Now add packages for extension modules on top of all these, and you get a really large amount of binaries from a small number of sources. Right now there's 1670 .deb files and 148 .dsc files, but the .dsc count includes variants that only differ in the version number per distribution (we attach .pgdg60+1 for squeeze packages, .pgdg70+1 for wheezy and so on), so the real number of different sources is rather something like 81, with 38 distinct source package names. Dimitri Fontaine, Magnus Hagander, and I have been working on this since I first presented the idea at PGconf.EU 2011 in Amsterdam. We now have a Jenkins server building all the packages, an archive server with the master repository, and a feed that syncs the repository to the postgresql.org FTP (well, mostly http) server. If you were previously using pgapt.debian.net, that's the same archive as on apt.postgresql.org (one rsync away). Please update your sources.list to point to apt.postgresql.org, I'll shut down the archive at that location at the end of January. Here's the Quickstart instructions from the Wiki page: Import the repository key from http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc:

wget -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc   sudo apt-key add -

Edit /etc/apt/sources.list.d/pgdg.list. The distributions are called codename-pgdg. In the example, replace squeeze with the actual distribution you are using:

deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main

Configure apt's package pinning to prefer the PGDG packages over the Debian ones in /etc/apt/preferences.d/pgdg.pref:

Package: *
Pin: release o=apt.postgresql.org
Pin-Priority: 500

Note: this will replace all your Debian/Ubuntu packages with available packages from the PGDG repository. If you do not want this, skip this step. Update the package lists, and install the pgdg-keyring package to automatically get repository key updates:

apt-get update
apt-get install pgdg-keyring

Not that I'm bored or anything, but I spent some more time on this since last month, and apparently some people are interested enough, so here's the recent news about wiigh2midi.

• First, I think it's got generic enough that the GH part is no longer warranted. I've looked for a suitable name, and the best I found was the simplest. From now on, this piece of code will be known simply as Wiimidi. Various searches on the web led me to believe that it's a traditional name that's been used a couple of times already by projects that stopped evolving years ago. So mine will either continue in that vein and maintain the tradition, or stay alive and keep the name. Time will tell.
• I ended up doing the Debian packaging. It's fairly minimal, and I don't currently plan to actually upload it (unless asked to), but it's there.
• I also added support for sending MIDI program change events in addition to notes. Why, you wonder? See next item.
• A new set of settings called program_changes in the sample configuration file, and a new rhythmbox-midi script. The goal is to control a Rhythmbox music player with your Wiimote (or possibly the attached drumkit).
• I also slapped a GPL-2+ license on it, which is consistent with the python-cwiid license.

Also, since the code moved (and it doesn't harm mentioning it again): Wiimidi is developed with Bazaar, and the public branch is at https://alioth.debian.org/~lolando/bzr/wiimidi/trunk/. So, to grab a copy:

 bzr checkout https://alioth.debian.org/~lolando/bzr/wiimidi/trunk/

Patches welcome, of course! Also, I'll be interested to hear about your applications. I've been told about a Wiimote-powered foot controller, which is exactly the kind of unpredictable results I was hoping to achieve by publishing my code. Keep it up, I want to hear about Wiimote-controlled robot dinosaurs next! Update: Wiimidi now has its dedicated page at Wiimidi.

Review: American Gods, by Neil Gaiman

Publisher:	HarperTorch
Copyright:	July 2001
Printing:	May 2002
ISBN:	0-380-78903-5
Format:	Mass market
Pages:	588

At the start of American Gods, Shadow is in prison. He's done three years of a sentence of six and is about to get released on parole. He has a wife and a job waiting for him and every hope of getting back to a good life. None of that happens. The day he's released, he finds out his wife died in a car accident. The job is gone too; it was an offer from his best friend, who died in the same accident. Numb, an ex-con, with no job prospects, he's recruited by a strange man named Wednesday, with the help of a leprechaun named Mad Sweeney, to drive Wednesday around the country, run errands for him, hurt people if absolutely necessary, and hold his vigil if he dies. And that's how Shadow is drawn into the conflict between old gods and young, between the gods that immigrants brought to America from the old world and the flashy new gods of technology and media that America is creating for itself. With native American gods and cultural legends watching around the edges. I first read American Gods shortly after it came out and before I started writing reviews, so this is a re-read. It's a book I'm glad I re-read before reviewing. The first time I read it, I was thoroughly engrossed and thought it was excellent. This time, I was entertained, but surprised at how slowly it moved, how disconnected it sometimes felt, and how little substance there was to the plot. Reading American Gods involves spending lots of time with Shadow, and I think one's reactions to this book will vary a lot based on how much one likes him as a character. Shadow is very withdrawn, very detached from the world, and quietly matter-of-fact in how he goes through his life. He's been through some major shocks and has been severed fairly thoroughly from society by three years in prison, and he doesn't reconnect easily. To some readers, such as myself, this comes across as taciturn competence and quiet observation, two things that I greatly enjoy reading about. I liked Shadow, liked reading between the lines for hints at his feelings, and sympathized a lot with his emotional state. But I can easily see an alternative reading that makes him a cipher or, worse, a blank spot in the book into which the reader has to insert themselves. One has to work at seeing Shadow's character and finding some sympathy for the degree to which he stays separate from the world. American Gods does have a plot, but it's the sort of plot that allows for endless digressions, background stories, and side meetings. In that, it reminded me less of Gaiman's other novels and more of several arcs of Sandman. I think it's worthwhile to think of this less as a novel and more as an extended ramble through the world as Gaiman sees it, full of mythology, stories, quirky people understanding the world in their own ways, and upwellings of the sacred and significant in the most unexpected of places. For example, the holy places of America in American Gods, analogous to the multiple small sacred places and old ruins in Europe, are roadside attractions, via a line of argument that makes much more sense than it has any right to and demonstrates a canny understanding of the expression of American entrepreneurial culture. As with much Gaiman, reading with Wikipedia or another mythological resource at hand is a good idea. Even if you have a good general knowledge of the most common mythology studied in the US, you probably won't recognize many of the figures Gaiman draws on: Czernobog, for example, or Wisakedjak. Part of the fun of this book is seeing Gaiman humanize them and embed them in modern culture and modern concerns while retaining their mythological edge and strangeness. As any reader of Sandman can tell you, this is one of Gaiman's strengths as a writer, and he does it well here. The bits with the undertakers Mr. Ibis and Mr. Jacquel (Thoth and Anubis) are absolutely brilliant and probably the best sections of the book (although Sam's exchange with the Men in Black is a strong contender).

"Back in my day, we had it all set up. You lined up when you died, and you'd answer for your evil deeds and for your good deeds, and if your evil deeds outweighed a feather, we'd feed your soul and your heart to Ammet, the Eater of Souls." "He must have eaten a lot of people." "Not as many as you'd think. It was a really heavy feather. We had it made special. You had to be pretty damn evil to tip the scales on that baby."

Like so much of Gaiman's work, this is the sort of book that contributed a half-dozen quotes to my quote file during my first reading. Gaiman always impresses me by not only managing to portray gods as human without losing the numinous edge, but sustaining that portrayal through very extended contact with the same god. And despite the interlude with the undertakers, the long interlude in small-town life in Lakeside, and the numerous other digressions and meanderings of a book that's not in much of a hurry to get anywhere, there is a plot and a definite conclusion, and Gaiman has some interesting twists in store at the end of that road. There's a lot of philosophical currents going on under the surface, not all of which cohere, but Gaiman does pull a lot of them together. The resolution can be a bit of an anticlimax, but I think it's a fitting one that's true to the nature of the story. I didn't enjoy this as much on the second reading as I did on the first. It didn't feel quite as coherent as I'd remembered it, and the meandering pace occasionally made the story feel stretched. But I do like Shadow, I love spending time in Gaiman's head and with his portrayals of deities, and American Gods is a treasure of great quotes. If you enjoy Gaiman's other work and aren't looking for a lot of plot and forward-moving energy, this is definitely recommended. Rating: 7 out of 10

I just finished to address the awesome debian crowd at the Mini Deb conf in paris. My presentation was about a few challenges we have ahead to bootstrap debian on a new architecture. Johannes Schauer and Wookey did a lot of work in the last few months particularly focusing on Linaro/Ubuntu. After Wheezy I think it is important to catch up with their work and integrate it in debian. The two main take away messages from my presentation :

• Add Multi Arch annotations to your packages. This is essential to cross compile packages automatically. We are still not able to cross compile a minimal debian system in debian because we still miss many multi-arch annotations. Experiments show that with these annotations, these packages will cross compile just fine. A lot of work has been done in this direction by Wookey.

• Debian should consider adding build profiles to build dependencies. Build Profiles are global build dependencies filters to create packages with a different set of functionalities. Build profiles are of the form Build-Depends: foo (>=0.1) [amd64] <!stage1 bootstrap> bar.

This week we just reached an important milestone toward a fully automatic bootstrap procedure. Hopefully we are going to tell you more about this work during fosdem 2013 My slides are attached.

Attachment	Size
debminiconf2012.pdf	271.56 KB

A little while ago Michele Cane filed Debian bug #689912 against qcontrol asking that the package install the correct configuration file for a TS-419P II, a newer device which has an LCD. It turns out that to support devices with LCDs properly we need to probe a GPIO line in order to figure out if an LCD is attached or if a serial console is in use (since the two are mutually exclusive on this hardware, when you attach a serial console you must toggle a jumper too). This actually turns out to be rather trivial since qcontrol's configuration files are actually simple Lua programs and the state of the GPIO lines can be accessed via sysfs and so we can read the state directly from the configuration file itself. While looking into this I happened to notice that the various board-specific configuration files are actually very similar and that the differences can all be detected by a combination of the Hardware field in /proc/cpuinfo and the GPIO lines. So I started looking into whether I could write a single consolidated qcontrol.conf file to replace the various variant specific files. It turns out to be quite possible, the resulting consolidated file has been posted to the bug log (see also my mail). The danger with configuration files which are also fully-fledged programming languages is that there is a temptation to write more and more complex configuration files with the danger that the primary purpose of the file (configuration) can be obscured. I think in this case it is a close call but on balance I think the downsides outweigh the advantages, so I've uploaded qcontrol 0.4.2+svn-r40-2 which simply adds LCD support to the relevant board specific configuration file. In process of doing so I seem to have fallen into a trap which seems to catch out many sbuild users at least once and uploaded a package with "UNRELEASED" still in the changelog. To try and save myself (and others) some blushes in the future I've filed Debian bug #693841 (wishlist) against dput to see if it can catch this case, I've also added to my prerelease checklist. As it happens my own TS-419P II arrived today. I intend to use it for second tier storage (mythtv recordings, music, mirrors) as well as light server duties (MTA, internal DNS etc) which should allow me to turn off the existing PC in the cupboard under the stairs which is getting a bit long in the tooth and starting to become a bit unreliable (I also suspect its costing a fortune to run).

When someone wants to give you access to a Cisco VPN, they might give you a .mobileconfig file. This is apparently used by MacOS and iOS to encapsulate the configuration parameters needed to connect to a VPN. You should be able to connect to it with open source software (such as NetworkManager and vpnc) as long as you have the right configuration. Some helpful soul has tried to give you that configuration, but it s wrapped up in an Apple-specific container. Here s how you rip it open and get the goodies. File format A .mobileconfig appears to contain:

1. Some binary garbage which is safe to ignore
2. An XML document containing the good bits, i.e.:
   1. The local identifier (i.e. IPsec group name)
   2. The remote address (i.e. IPsec gateway host)
   3. The shared secret (base64 encoded)
3. Some more binary garbage which is safe to ignore

and it looks like this:

<plist version="1.0">
<dict>
  <key>PayloadContent</key>
  <array>
    <dict>
      <key>IPSec</key>
      <dict>
        <key>AuthenticationMethod</key>
        <string>SharedSecret</string>
        <key>LocalIdentifier</key>
        <string>LOCAL_IDENTIFIER_HERE</string>
        <key>LocalIdentifierType</key>
        <string>KeyID</string>
        <key>RemoteAddress</key>
        <string>REMOTE_ADDRESS_HERE</string>
        <key>SharedSecret</key>
        <data>
        BASE64_ENCODED_SHARED_SECRET_HERE
        </data>
      </dict>
      <key>IPv4</key>
      <dict>
        <key>OverridePrimary</key>
        <integer>0</integer>
      </dict>
      <key>PayloadDescription</key>
      <string>...</string>
      <key>PayloadDisplayName</key>
      <string>...</string>
      <key>PayloadIdentifier</key>
      <string>...</string>
      <key>PayloadOrganization</key>
      <string>...</string>
      <key>PayloadType</key>
      <string>com.apple.vpn.managed</string>
      <key>PayloadUUID</key>
      <string>...</string>
      <key>PayloadVersion</key>
      <integer>1</integer>
      <key>Proxies</key>
      <dict>
        <key>HTTPEnable</key>
        <integer>0</integer>
        <key>HTTPSEnable</key>
        <integer>0</integer>
        <key>ProxyAutoConfigEnable</key>
        <integer>0</integer>
        <key>ProxyAutoDiscoveryEnable</key>
        <integer>0</integer>
      </dict>
      <key>UserDefinedName</key>
      <string>...</string>
      <key>VPNType</key>
      <string>IPSec</string>
    </dict>
  </array>
  <key>PayloadDescription</key>
  <string>...</string>
  <key>PayloadDisplayName</key>
  <string>...</string>
  <key>PayloadIdentifier</key>
  <string>...</string>
  <key>PayloadOrganization</key>
  <string>...</string>
  <key>PayloadRemovalDisallowed</key>
  <false/>
  <key>PayloadType</key>
  <string>Configuration</string>
  <key>PayloadUUID</key>
  <string>...</string>
  <key>PayloadVersion</key>
  <integer>1</integer>
</dict>
</plist>

The shared secret is base64-encoded, so you can decode it with: $ echo -n 'BASE64_ENCODED_SECRET_HERE' base64 -d Network Manager configuration

1. Make sure you have network-manager-vpnc installed
2. Click the Network Manager icon, select VPN Connections , Configure VPN
3. Create a Cisco-compatible (vpnc) connection
   
   

   Create a Cisco-compatible (vpnc) VPN connection

4. Configure the connection settings as follows:
   
   

   Configure the connection settings

   • Enter the remote address in the Gateway field
   • Enter the local identifier in the Group name field
   • Enter the shared secret in the Group password field
5. To connect, click the Network Manager icon, select VPN Connections , and select the connection you just configured

Good luck and enjoy!

Review: Set This House in Order, by Matt Ruff

Publisher:	Perennial
Copyright:	2003
Printing:	2004
ISBN:	0-06-095485-X
Format:	Trade paperback
Pages:	479

A textbook case history would say that Andy Gage has disassociative identity disorder (referred to here as the more common although medically inaccurate "multiple personality disorder" for reasons of clarity and to match casual speech, according to the author). However, that's a completely inadequate description. A better description would be that Andy Gage's body is inhabited by a large set of different personalities, who take turns being in control. The original Andy Gage is dead, shattered by horrific childhood abuse. Aaron became the dominant personality and gave structure to the internal geography of the mind they all share, with the help of extensive counselling. But Aaron was exhausted by being the public face and maintaining control of the shared body. Hence, Andrew. Andrew Gage is the first-person narrator of half of this book. He is twenty-eight, the same age as the body, but he's only existed for two years. He considers Aaron his father, since Aaron called him out of the lake inside their internal metal geography. His purpose is to run the body and be the public face of the family. He is by turns capable and naive, shy on personal experience, but able to get the help of the others. The one heard from most frequently is Adam, a rude and mischevious teenager who also has an uncanny ability to read other people, and who frequently watches through the body's eyes from what the family calls the "pulpit" and gives Andrew advice. Andrew works for Julie Sivik, a woman he met the day he quit his job at a big-box store because he couldn't explain his behavior to his boss without discussing being a multiple. Julie is a wholly impractical, slightly unethical, well-meaning force of chaos who serves as the thread that draws the plot of the book together. She's running a startup on ingenuity and borrowed money that is attempting to create a comprehensive and customizable virtual reality world. She's the one who finds Penny and hires her, in part for her skills, and in part to push Penny and Andrew together. Penny Driver has the same condition as Andrew, but Penny is only vaguely aware of it. The world for her is full of blackouts, lost time, waking up in strange and sometimes horrific situations, and attempting to structure her life based on notes left for herself that she didn't write. When she meets Andrew, she's barely making it through life; thankfully, one of her personalities is remarkably good at computers, and that has been enough to make a living despite the chaos. But Julie is the unintentional catalyst for her losing that job, the best job she's ever had, leaving her with no real alternative but to take the job with Julie's company. There are a few things that are important to say up-front about this book. First, neither I nor the author are actually multiples (or, put another way, neither of us have DID), and writing about atypical mental states is fraught with peril. It's very easy to misrepresent other people's experiences, and multiple personalities in particular are frequent material for dramas of all sorts. There's something people find inherently fascinating about the idea, and it's easy for that fascination to run away with reality and distort the lived experience. I recommend reading Ruff's FAQ about the book, which spells out what his goals are and what approach he took. I did some quick searches and didn't uncover a lot of anger or dislike of the book among either people who have DID or psychiatrists, and it seems true to the descriptions of internal reality I've heard from people who are multiples. But despite being more respectful of real experience than most fictional treatments of DID, this is still a work of fiction written by someone without first-hand experience, and it's worth keeping that in mind. Second, DID is an intensely controversial diagnosis among psychiatrists. There are, generally speaking, two major controversies. One is over whether or not DID can actually be caused by traumatic experiences such as childhood abuse, or whether it is a creation of psychiatrists, a mental model that they've imposed on their patients or convinced their patients to believe. Set This House in Order comes down firmly on the first side of that controversy: that DID can be created by abuse and is not a creation of psychotherapy. (So, I should note, does every multiple that I've ever talked to or read.) Second, there is significant controversy over treatment, with some psychiatrists believing that integration of the personalities into one is the only true treatment, and others believing that multiples can remain stable and functional as multiple personalities. Set This House in Order leans strongly towards the view that multiples can be stable as multiples and that while integration may be appropriate for some, it's not for all. Both of these viewpoints are presented very convincingly by the narrative in the book, and both are consistent with the beliefs of a lot of multiples, but it's worth remembering that these are active and often quite heated controversies and a novel is not evidence. It's good to be aware of both of those points, and to read Put This House in Order as something of a what-if, where much more is clear about DID than is actually clear in our world. But once one has done that, one is in for an excellent book. I loved this book from start to finish. (Well, okay, I felt some surprise at the nature of the ending, but it grew on me after further consideration.) It's simply a brilliant novel, with a little bit of everything: very good characterization, messy love affairs, the satisfying tension of someone overcoming great odds to put themselves together with the help of friends, a startlingly original perspective of the world, some excellent surprises, and some absolutely wonderful characters. Andrew is a good choice of narrator: he brings an earnest calm to a story that could otherwise be quite chaotic and provides a great vantage point for the reader. And I found Penny (and particularly some of her other souls) impossible not to like. Set This House in Order is a tense book in places, and it's a book where people's lives break down before they can get better. It's also a book about rather horrific abuse, and while nearly all of the specifics happen off-stage, imagination fills in some nasty details. But Ruff manages to write it with a structure and with enough calm assurance that I felt just reassured enough without taking away the tension. That's a tricky balance; few books do it better. And the supporting characters are a delight. One might think, with all the complexity happening for both Andrew and Penny, that Ruff wouldn't have room in the narrative for other characters, but one would be wrong. There are very few saints and demons here, just a lot of complex people, with their own problems and their own perspectives, who are generally trying to do the best they can from day to day. There are some clear-cut villains in the past Ruff wisely makes no attempt to add shades of grey to abusers but even there one can see some clear signs of untreated severe mental illness and chaotic lives rather than mustache-twirling pre-meditated villainy. This is, in short, a book full of people, in all their messy complications, which is what makes it so delightful. I'm not going to say anything more about the plot, or even the various personalities, since discovering the direction of the book was so much of the fun. I will say that, despite the subject matter and some of the events, the book is surprisingly gentle. One of the things about it that I loved the most was the emotional tone Ruff gave to being a multiple: the sense of a squabbling but tight-knit family looking out for each other. I can't judge the reality of this impression, but it makes for a wonderful story. Highly recommended. The best book I've read so far this year. Rating: 10 out of 10

pyBit - cross-platform package building using AMQP


Message queues provide a simple way to create a distributed, cross-platform buildd toolkit to build packages using a collection of buildds, direct from various VCS clients. pyBit is intended to support rapidly evolving software collections and can support multiple VCS frontends and multiple build backends. Cross building is expected to be supported for some backends. The initial backend uses dpkg for Debian with subversion providing the source and sbuild doing the actual build.

pyBit includes support for cancelling selected builds and using multiple buildd clients per architecture, per platform and per suite.

Hooks are available or in development for subversion and git, other VCS hooks can be added. A RESTful web API provides live build reports and can generate build jobs for specific packages using particular VCS branches on selected architectures to support re-building packages at any point in the development process. Build history is stored using postgresql.

Other buildd systems can rebuild long lists of packages or build lots of binary packages from relatively slow moving source packages. PyBit exists to handle much more rapid software development across a wide range of platforms, VCS inputs and architectures. Buildd clients which are under-used can be tasked with building multiple suites or adding cross-build support. Buildd clients which are over-utilised are easily identified and adding new machines to an existing architecture / platform / suite pool should be easy. Hook activation automatically cancels any ongoing build for the same architecture, platform and suite to avoid wasting time on an interim version.

The emphasis in pyBit is to have fast builds with redundant clients, reliable reporting using a flexible and intuitive frontend. To this end, there is no need for a source package to be uploaded. Depending on the VCS hook in use, builds can happen every time a particular file is changed (e.g. debian/changelog for a native Debian package) or at every push (for a distributed VCS) or whatever is appropriate for a particular software collection. Builds are checked for available build-dependencies and packages re-queued if the build-dependencies are not yet met.

In the longer term, it may well be possible to use more than one server / database combination to support more builds and more platforms.

So far, we've got to a working model and tagged 0.1.0 as the first downloadable release. There is a lot more to do, especially adding more documentation, more VCS hooks, support for more VCS methods on the buildd clients and more buildd client scripts for platforms other than Debian. (The git hook and git source client are expected to let pyBit be self-buildable but a certain amount of configuration will be required for the server and each client which makes it not-quite self-hosting.)

pyBit concentrates on preparing collections of binary packages which can be used to build others, rather than trying to rebuild everything every time - this allows more rapid upstream software development and encourages modular, re-usable software. pyBit will also support rebuilding specific versions, architectures, suites or platforms via the RESTful web API. Access to this frontend can be controlled through any of the standard methods.

Components are loosely coupled via JSON encoded messages sent using rabbitMQ and curl. A new client can be added at any time and it will simply pick up buildd jobs from the relevant queue.

Development has now moved to Github and if anyone wants to look at new clients and new hooks, just contact the team by email or via IRC (#pybit on irc.oftc.net). Current development is based on Debian Squeeze 6.0.6 with backports and also on Debian Wheezy - more testing is welcome. Patches are also very welcome, pyBit is licensed under GPL2+.

There is no intrinsic reason why pyBit could not support any buildd platform capable of taking source from a known location and building a set of binaries. Packages are added to the queues whenever the hook is activated, so adding new packages to the collection is simply a case of triggering the existing hook.

Another interesting challenge for pyBit would be to trigger a hook on a new source code repository and just let it work through every package until everything is built. That probably won't work with the current 0.1.0 code but if that is what you'd like to work on, join the team.

Since October 2011 my flatmate and I where quite busy realizing a little pet project of ours called The Art of Asking. The ultimate goal is to visualize the world s opinion in an intuitive fashion and make it easy for everyone to play around with the data. The idea behind The Art of Asking is that users submit interesting questions which are answered by users around the world. But instead of showing only the boring result, we want to provide interesting insights and statistics about the answers given. For now the users can see the results of the question visualized by geographical regions. For example on the page for the question How are you today? you can see the interactive map with the pie chart. The map shows the average/dominating answer for each continent encoded by color, and the pie chart the distribution of the different answers for that region. If you move the mouse over a continent the pie chart updates and shows the distribution of answers for that continent. You can also click to zoom into the map to see the same for countries and regions. This allows you to investigate how the answers are distributed around the world. This is already quite nice and fancy to play around with, but of course we want much more. Right now we re working on a feature which will users allow to combine two arbitrary questions and see how the answers are related. This doesn t sound like much, but it is very addictive to crawl through the list of questions and find interesting correlations. Here is a little plot how it could look like (the data is from the actual data base of answers). For each possible combination of answers it shows the percentage of people who answered in that combination. Of course those plots only make sense when enough users answered for both questions you want to compare which is right now not very often the case, so I guess we ll roll out that feature sometime later when we have more data. But we have lots of ideas and are working on further ways to investigate the answers. Obviously one low hanging fruit for example would be the distribution of the answers over time. We ve been working for over a year now on this project in our spare time and we built it more or less from scratch. I wrote a small WSGI framework in Python and on top of that the WSGI application which runs the site. We use MongoDB for the storage of the data, uWSGI and nginx for the server, Jinja2 and Bootstrap for the HTML and D3.js for the visualization of the data, where Maci did a wonderful job realizing the interactive map and charts. We re running this site since July 2012 now and are already quite satisfied with the number of users, and the quality of the questions. But of course we could always use more (especially more answers). So if you want to try it out, go to theartofasking.com and fill out the blank spots on the map! We re happy about every answer and question we can get and are eager to hear your suggestions.

You certainly know Picasa Albums Web and Flickr, two commercial picture hosting services. You probably know a few free software projects that allow people to keep control of their own pictures, such as Gallery, Piwigo, Zenphoto or PhotoShow. GNU MediaGoblin is a similar project, but with some interesting characteristics and features:

• it has a funny name and mascot;
• it is part of the GNU project;
• it is written in Python;
• it does not need a database servers since can use an SQLite database;
• it is designed with a specific care for free culture, allowing to indicate the license of a picture and to provide its source format as an attachment for instance.
• it supports audio, video and 3D models with HTML5 features rather than the dreadful Flash;
• it is a work in progress and is asking for financial contribution.

If you find GNU MediaGoblin as promising as I do, please consider donating!

It's not difficult to write a UEFI application under Linux. You'll need three things:

• An installed copy of the gnu-efi library and headers
  
• A copy of the UEFI specification from here
  
• Something running UEFI (OVMF is a good option if you don't have any UEFI hardware)

So, let's write a trivial UEFI application. Here's one from the gnu-efi source tree:

#include <efi.h>
#include <efilib.h>
EFI_STATUS
efi_main (EFI_HANDLE image, EFI_SYSTEM_TABLE *systab)
 
        SIMPLE_TEXT_OUTPUT_INTERFACE *conout;
        conout = systab->ConOut;
        InitializeLib(image, systab);
        uefi_call_wrapper(conout->OutputString, 2, conout, L"Hello World!\n\r");
        return EFI_SUCCESS;
 

The includes are fairly obvious. efi.h gives you UEFI functions defined by the specification, and efilib.h gives you functions provided by gnu-efi. The runtime will call efi_main() rather than main(), so that's our main function. It returns an EFI_STATUS which will in turn be returned to whatever executed the binary. The EFI_HANDLE image is a pointer to the firmware's context for this application, which is used in certain calls. The EFI_SYSTEM_TABLE *systab is a pointer to the UEFI system table, which in turn points to various other tables.

So far so simple. Now things get interesting. The heart of the UEFI programming model is a set of protocols that provide interfaces. Each of these interfaces is typically a table of function pointers. In this case, we're going to use the simple text protocol to print some text. First, we get a pointer to the simple text output table. This is always referenced from the system table, so we can simply assign it. InitializeLib() just initialises some internal gnu-efi functions - it's not technically required if we're purely calling native UEFI functions as we are here, but it's good style since forgetting it results in weird crashes.

Anyway. We now have conout, a pointer to a SIMPLE_TEXT_OUTPUT_INTERFACE structure. This is described in section 11.4 of the UEFI spec, but looks like this:

typedef struct _EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL  
    EFI_TEXT_RESET Reset;
    EFI_TEXT_STRING OutputString;
    EFI_TEXT_TEST_STRING TestString;
    EFI_TEXT_QUERY_MODE QueryMode;
    EFI_TEXT_SET_MODE SetMode;
    EFI_TEXT_SET_ATTRIBUTE SetAttribute;
    EFI_TEXT_CLEAR_SCREEN ClearScreen;
    EFI_TEXT_SET_CURSOR_POSITION SetCursorPosition;
    EFI_TEXT_ENABLE_CURSOR EnableCursor;
    SIMPLE_TEXT_OUTPUT_MODE *Mode;
  EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL;

The majority of these are functions - the only exception is the SIMPLE_TEXT_OUTPUT_MODE pointer, which points to the current mode. In an ideal world you'd be able to just call these directly, but sadly UEFI and Linux calling conventions are different and we need some thunking. That's where the uefi_call_function() call comes in. This takes a UEFI function as its first argument (in this case, conout->OutputString), the number of arguments (2, in this case) and then the arguments. Checking the spec for OutputString, we see this:

typedef
EFI_STATUS
(EFIAPI *EFI_TEXT_STRING) (
    IN EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL *This,
    IN CHAR16 *String
);

The first argument is a pointer to the specific instance of the simple text output protocol. This permits multiple instances of the same protocol to exist on a single machine, without having to duplicate all of the code. The second argument is simply a UCS-2 string. Our conout pointer is a pointer to the protocol, so we pass that as the first argument. For the second argument, we pass L"Hello World!\n", with the L indicating that this is a wide string rather than a simple 8-bit string. uefi_call_function() then rearranges these arguments into the appropriate calling convention and calls the UEFI function. The firmware then prints "Hello World!" on the screen. Success.

(Warning: uefi_call_function() does no type checking on its arguments, so if you pass in a struct instead of a pointer it'll build without complaint and then give you a bizarre error at runtime)

Finally, we clean up by simply returning EFI_SUCCESS. Nothing more for us to worry about. That's the simple case. What about a more complicated one? Let's do something with that EFI_HANDLE that got passed into efi_main().

#include <efi.h>
#include <efilib.h>
EFI_STATUS
efi_main (EFI_HANDLE image, EFI_SYSTEM_TABLE *systab)
 
        EFI_LOADED_IMAGE *loaded_image = NULL;
        EFI_GUID loaded_image_protocol = LOADED_IMAGE_PROTOCOL;
        EFI_STATUS status;
        InitializeLib(image, systab);
        status = uefi_call_wrapper(systab->BootServices->HandleProtocol,
                                3,
                                image, 
                                &loaded_image_protocol, 
                                (void **) &loaded_image);
        if (EFI_ERROR(status))  
                Print(L"handleprotocol: %r\n", status);
         
        Print(L"Image base        : %lx\n", loaded_image->ImageBase);
        Print(L"Image size        : %lx\n", loaded_image->ImageSize);
        Print(L"Image file        : %s\n", DevicePathToStr(loaded_image->FilePath));
        return EFI_SUCCESS;
 

UEFI handles can have multiple protocols attached to them. A handle may represent a piece of physical hardware, or (as in this case) it can represent a software object. In this case we're going to get a pointer to the loaded image protocol that's associated with the binary we're running. To do this we call the HandleProtocol() function from the Boot Services table. Boot services are documented in section 6 of the UEFI specification and are the interface to the majority of UEFI functionality. HandleProtocol takes an image and a protocol identifier, and hands back a pointer to the protocol. UEFI protocol identifiers are all GUIDs and defined in the specification. If you call HandleProtocol on a handle that doesn't implement the protocol you request, you'll simply get an error back in the status argument. No big deal.

The loaded image protocol is fairly different to the simple text output protocol in that it's almost entirely data rather than function pointers. In this case we're printing the address that our executable was loaded at and the size of our relocated executable. Finally, we print the path of the file. UEFI device paths are documented in section 9.2 of the UEFI specification. They're a binary representation of a file path, which may include the path of the device that the file is on. DevicePathToStr is a gnu-efi helper function that converts the objects it understands into a textual representation. It doesn't cover the full set of specified UEFI device types, so in some corner cases it may print "Unknown()".

That covers how to use protocols attached to the image handle. How about protocols that are attached to other handles? In that case we can do something like this:

#include <efi.h>
#include <efilib.h>
EFI_STATUS
efi_main (EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *systab)
 
        EFI_STATUS status;
        EFI_GRAPHICS_OUTPUT_PROTOCOL *gop;
        EFI_GUID gop_guid = EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID;
        InitializeLib(image_handle, systab);
        status = uefi_call_wrapper(systab->BootServices->LocateProtocol,
                                   3,
                                   &gop_guid,
                                   NULL,
                                   &gop);
        Print(L"Framebuffer base is at %lx\n", gop->Mode->FrameBufferBase);
        return EFI_SUCCESS;
 

This will find the first (where first is an arbitrary platform ordering) instance of a protocol and return a pointer to it. If there may be multiple instances of a protocol (as there often are with the graphics output protocol) you should use LocateHandleBuffer() instead. This returns an array of handles that implement the protocol you asked for - you can then use HandleProtocol() to give you the instance on the specific handle. There's various helpers in gnu-efi like LibLocateHandle() that make these boilerplate tasks easier. You can find the full list in /usr/include/efi/efilib.h.

That's a very brief outline of how to use basic UEFI functionality. The spec documents the full set of UEFI functions available to you, including things like direct block access, filesystems and networking. The process is much the same in all cases - you locate the handle that you want to interact with, open the protocol that's installed on it and then make calls using that protocol.Building these examplesBuilding these examples is made awkward due to UEFI wanting PE-COFF binaries and Linux toolchains building ELF binaries. You'll want a makefile something like this:

ARCH            = $(shell uname -m   sed s,i[3456789]86,ia32,)
LIB_PATH        = /usr/lib64
EFI_INCLUDE     = /usr/include/efi
EFI_INCLUDES    = -nostdinc -I$(EFI_INCLUDE) -I$(EFI_INCLUDE)/$(ARCH) -I$(EFI_INCLUDE)/protocol
EFI_PATH        = /usr/lib64/gnuefi
EFI_CRT_OBJS    = $(EFI_PATH)/crt0-efi-$(ARCH).o
EFI_LDS         = $(EFI_PATH)/elf_$(ARCH)_efi.lds
CFLAGS          = -fno-stack-protector -fpic -fshort-wchar -mno-red-zone $(EFI_INCLUDES)
ifeq ($(ARCH),x86_64)
        CFLAGS  += -DEFI_FUNCTION_WRAPPER
endif
LDFLAGS         = -nostdlib -znocombreloc -T $(EFI_LDS) -shared -Bsymbolic -L$(EFI_PATH) -L$(LIB_PATH) \
                  $(EFI_CRT_OBJS) -lefi -lgnuefi
TARGET  = test.efi
OBJS    = test.o
SOURCES = test.c
all: $(TARGET)
test.so: $(OBJS)
       $(LD) -o $@ $(LDFLAGS) $^ $(EFI_LIBS)
%.efi: %.so
        objcopy -j .text -j .sdata -j .data \
                -j .dynamic -j .dynsym  -j .rel \
                -j .rela -j .reloc -j .eh_frame \
                --target=efi-app-$(ARCH) $^ $@

There's rather a lot going on here, but the important stuff is the CFLAGS line and everything after that. First, we need to disable the stack protection code - there's nothing in the EFI runtime for it to call into, so we'll build a binary with unresolved symbols otherwise. Second, we need to build position independent code, since UEFI may relocate us anywhere. short-wchar is needed to indicate that strings like L"Hi!" should be 16 bits per character, rather than gcc's default of 32 bits per character. no-red-zone tells the compiler not to assume that there's 256 bytes of stack available to it. Without this, the firmware may modify stack that the binary was depending on.

The linker arguments are less interesting. We simply tell it not to link against the standard libraries, not to merge relocation sections, and to link in the gnu-efi runtime code. Nothing very exciting. What's more interesting is that we build a shared library. The reasoning here is that we want to perform all our linking, but we don't want to build an executable - the Linux executable runtime code would be completely pointless in a UEFI binary. Once we have our .so file, we use objcopy to pull out the various sections and rebuild them into a PE-COFF binary that UEFI will execute.

comments

Inspired by this clock, which made the rounds a few weeks back, and fueled by a bet to see who could make it first (hi Fabs!), ladies and gentleman, I m honored to introduce to you, the Silly Wal(l Cloc)k:

This is a prototype version, done without any fancy precision tools and whatnot, so you can easily see that it s not the most high quality product you ll ever get your hands on. If anyone wants to reproduce this at home, here s how it was done: First get a hold of some cheap wall clock like this one: Some things to think about when buying this beauty *cough*:

• Cheap clocks seem to usually be built in a disassemble-friendly way, but just to be sure, check you won t have to break anything to get to the clock face and the hands.
• If possible, get a clock with the numbers printed to the glass. This isn t that important, but saves you a bit of work.
• Depending on how the hands are attached to the clock mechanism, it may be a good idea to get a clock with black hands. This is important because of the little tack used in some models to cover the axis on which the hands are attached. Otherwise John Cleese will have to perform his silly walk with some randomly colored dot on his ass.

Disassembling this specific cheap model was pretty straightforward and didn t take more than a minute to get to juicy parts:

As a base image I used this and traced it to produce this SVG with the legs in different layers and a marker for the holes. For this prototype, it would probably have been ok to use the original photo directly, but having the vector version ready means it s easy to use a 3d printer and laser-cutter to make the next version of the hands and face, which will definitely look and behave a lot better. Next step is printing the three layers body and two legs. Since this specific clock is wider than an A4 sheet, I had to use some white cardboard as background (the clock s own box hurray for resourcefulness) and painstakingly cut the printed body s outline, to avoid the slight tone difference between cardboard and paper. If you use the provided SVG and a laser-cutter, you d hopefully avoid this sort of issue. I didn t have any black cardboard lying around, so I had to print the legs, cut the outline again and strengthen them with some thin cardboard from tea boxes (taken directly from the trash hurray for resourcefulness again). Again something that 3d printing will hopefully side-step. It s important to use light materials for the legs, since most clock mechanisms don t have enough force to move heavy hands, which would cause the clock to quickly loose precision. Depending on the width of the clock s axis, a paper hole puncher might be the ideal tool to make holes in the legs. Since the prototype was largely made with paper, there was no point in going for precision when making the holes, as there s no way to fit the legs tightly to the clock s axis. For this, the original hands were cut in length and used as a base on which to super-glue the legs, so the original holes could be used. After these few easy steps, it s only a matter of reassembling the clock and voil !

mbiebl has recently uploaded systemd 44-5 to Debian unstable. If you are already using systemd, you should definitely upgrade to this version, it fixes a lot of little annoyances. This is especially important if you are using libvirt >= 0.10.2-4 or if you experienced a hanging boot when your network cable is attached.

This entry is not really about computers, technology, or other work-related topics, but more about a hard-hack that I wanted to try for a while. How to make a kilt !!! After a bit of duck-ducking, I decided to follow this excellent tutorial. At first sight the entire process seems a bit long, but you will realize that after the first read, that everything boils down to 3 steps: measure, fold and pin, sew. For the measure part, I've the impression that the formula that is given in the instructable (waist/3*8+1) is a bit short for my comfort and taste. This is the size for the internal apron, the folded part that goes all around your left hip, back, and right hip, and the front apron. My suggestion would be to make the inner apron a bit longer then the front apron. This way the kilt will feel in my opinion more comfortable and it will envelop you body completely. For the fold and pin part, you just need a bit of patience and a ruler. Put the pins parallel to the folding as in the instructable and not perpendicular. This will help you later when sewing everything. The sewing ... If you know how to use a sewing machine, this is going to be a piece of cake. Otherwise, well, I spent more time troubleshooting the machine then sewing the kilt. I've broken few needles in the process and learned how to thread the machine with my eyes blindfolded. Not to mention that you have to learn how to disassemble this machines in a thousand parts to understand how the thread got stuck. It was fun. A lesson that I've learned is that a sewing machine works much better in the morning than late at night when you are tired and sleepy. Really ! Other then that, it was a fun experience. Maybe I'll make another one to commit this skill to mind. Maybe I'll run a kilt making workshop at the next debconf :)

Attachment	Size
PA250002.JPG	134.19 KB
PA250003.JPG	131.11 KB

To experiment with network stuff, I was using UML-based network labs. Many alternatives exist, like GNS3, Netkit, Marionnet or Cloonix. All of them are great viable solutions but I still prefer to stick to my minimal home-made solution with UML virtual machines. Here is why:

• I didn t want to use disk images. They take a lot of space and they have to be maintained. They also become cluttered, especially if you try to reuse them across several labs. They are also difficult to share.
• I want to be able to access my home directory. It contains the important configuration files related to the lab and I can put them in the right place thanks to symbolic links when the lab starts. It also makes exchanging files with the lab quite easy.
• I don t want to boot a complete system. This allows me to be cheap on memory and each virtual system should boot in a few seconds.

The use of UML had some drawbacks:

• It may be buggy. For example, it is currently not possible to use gdbserver inside UML without a patch. Sometimes, the kernel won t even compile.
• It is slow.

However, UML features HostFS, a filesystem providing access to any part of the host filesystem. This is the killer feature which allows me to not use any virtual disk image and to get access to my home directory right from the guest. I discovered recently that KVM provided 9P, a similar filesystem on top of VirtIO, the paravirtualized IO framework.

Setting up the lab The setup of the lab is done with a single self-contained shell file. The layout is similar to what I have done with UML. I will only highlight here the most interesting steps.

Booting KVM with a minimal kernel My initial goal was to experiment with Nicolas Dichtel s IPv6 ECMP patch. Therefore, I needed to configure a custom kernel. I have started from make defconfig, removed everything that was not necessary, added what I needed for my lab (mostly network stuff) and added the appropriate options for VirtIO drivers:

CONFIG_NET_9P_VIRTIO=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y

No modules. Grab the complete configuration if you want to have a look. From here, you can start your kernel with the following command ($LINUX is the appropriate bzImage):

kvm \
  -m 256m \
  -display none \
  -nodefconfig -no-user-config -nodefaults \
  \
  -chardev stdio,id=charserial0,signal=off \
  -device isa-serial,chardev=charserial0,id=serial0 \
  \
  -chardev socket,id=con0,path=$TMP/vm-$name-console.pipe,server,nowait \
  -mon chardev=con0,mode=readline,default \
  \
  -kernel $LINUX \
  -append "init=/bin/sh console=ttyS0"

Of course, since there is no disk to boot from, the kernel will panic when trying to mount the root filesystem. KVM is configured to not display video output (-display none). A serial port is defined and uses stdio as a backend¹. The kernel is configured to use this serial port as a console (console=ttyS0). A VirtIO console could have been used instead but it seems this is not possible to make it work early in the boot process. The KVM monitor is setup to listen on an Unix socket. It is possible to connect to it with socat UNIX:$TMP/vm-$name-console.pipe -.

Initial ramdisk UPDATED: I was initially unable to mount the host filesystem as the root filesystem for the guest directly by the kernel. In a comment, Josh Triplett told me to use /dev/root as the mount tag to solve this problem. I keep using an initrd in this post but the lab on Github has been updated to not use one. Here is how to build a small initial ramdisk:

# Setup initrd
setup_initrd()  
    info "Build initrd"
    DESTDIR=$TMP/initrd
    mkdir -p $DESTDIR
    # Setup busybox
    copy_exec $($WHICH busybox) /bin/busybox
    for applet in $($ DESTDIR /bin/busybox --list); do
        ln -s busybox $ DESTDIR /bin/$ applet 
    done
    # Setup init
    cp $PROGNAME $ DESTDIR /init
    cd "$ DESTDIR " && find .   \
       cpio --quiet -R 0:0 -o -H newc   \
       gzip > $TMP/initrd.gz
 

The copy_exec function is stolen from the initramfs-tools package in Debian. It will ensure that the appropriate libraries are also copied. Another solution would have been to use a static busybox. The setup script is copied as /init in the initial ramdisk. It will detect it has been invoked as such. If it was omitted, a shell would be spawned instead. Remove the cp call if you want to experiment manually. The flag -initrd allows KVM to use this initial ramdisk.

Root filesystem Let s mount our root filesystem using 9P. This is quite easy. First KVM needs to be configured to export the host filesystem to the guest:

kvm \
  $ PREVIOUS_ARGS  \
  -fsdev local,security_model=passthrough,id=fsdev-root,path=$ ROOT ,readonly \
  -device virtio-9p-pci,id=fs-root,fsdev=fsdev-root,mount_tag=rootshare

$ ROOT can either be / or any directory containing a complete filesystem. Mounting it from the guest is quite easy:

mkdir -p /target/ro
mount -t 9p rootshare /target/ro -o trans=virtio,version=9p2000.u

You should find a complete root filesystem inside /target/ro. I have used version=9p2000.u instead of version=9p2000.L because the later does not allow a program to mount() a host mount point². Now, you have a read-only root filesystem (because you don t want to mess with your existing root filesystem and moreover, you did not run this lab as root, did you?). Let s use an union filesystem. Debian comes with AUFS while Ubuntu and OpenWRT have migrated to overlayfs. I was previously using AUFS but got errors on some specific cases. It is still not clear which one will end up in the kernel. So, let s try overlayfs. I didn t find any patchset ready to be applied on top of my kernel tree. I was working with David Miller s net-next tree. Here is how I have applied the overlayfs patch on top of it:

$ git remote add torvalds git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
$ git fetch torvalds
$ git remote add overlayfs git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git
$ git fetch overlayfs
$ git merge-base overlayfs.v15 v3.6
4cbe5a555fa58a79b6ecbb6c531b8bab0650778d
$ git checkout -b net-next+overlayfs
$ git cherry-pick 4cbe5a555fa58a79b6ecbb6c531b8bab0650778d..overlayfs.v15

Don t forget to enable CONFIG_OVERLAYFS_FS in .config. Here is how I configured the whole root filesystem:

info "Setup overlayfs"
mkdir /target
mkdir /target/ro
mkdir /target/rw
mkdir /target/overlay
# Version 9p2000.u allows to access /dev, /sys and mount new
# partitions over them. This is not the case for 9p2000.L.
mount -t 9p        rootshare /target/ro      -o trans=virtio,version=9p2000.u
mount -t tmpfs     tmpfs     /target/rw      -o rw
mount -t overlayfs overlayfs /target/overlay -o lowerdir=/target/ro,upperdir=/target/rw
mount -n -t proc  proc /target/overlay/proc
mount -n -t sysfs sys  /target/overlay/sys
info "Mount home directory on /root"
mount -t 9p homeshare /target/overlay/root -o trans=virtio,version=9p2000.L,access=0,rw
info "Mount lab directory on /lab"
mkdir /target/overlay/lab
mount -t 9p labshare /target/overlay/lab -o trans=virtio,version=9p2000.L,access=0,rw
info "Chroot"
export STATE=1
cp "$PROGNAME" /target/overlay
exec chroot /target/overlay "$PROGNAME"

You have to export your $ HOME and the lab directory from host:

kvm \
  $ PREVIOUS_ARGS  \
  -fsdev local,security_model=passthrough,id=fsdev-root,path=$ ROOT ,readonly \
  -device virtio-9p-pci,id=fs-root,fsdev=fsdev-root,mount_tag=rootshare \
  -fsdev local,security_model=none,id=fsdev-home,path=$ HOME  \
  -device virtio-9p-pci,id=fs-home,fsdev=fsdev-home,mount_tag=homeshare \
  -fsdev local,security_model=none,id=fsdev-lab,path=$(dirname "$PROGNAME") \
  -device virtio-9p-pci,id=fs-lab,fsdev=fsdev-lab,mount_tag=labshare

Network You know what is missing from our network lab? Network setup. For each LAN that I will need, I spawn a VDE switch:

# Setup a VDE switch
setup_switch()  
    info "Setup switch $1"
    screen -t "sw-$1" \
        start-stop-daemon --make-pidfile --pidfile "$TMP/switch-$1.pid" \
        --start --startas $($WHICH vde_switch) -- \
        --sock "$TMP/switch-$1.sock"
    screen -X select 0
 

To attach an interface to the newly created LAN, I use:

mac=$(echo $name-$net   sha1sum   \
            awk ' print "52:54:" substr($1,0,2) ":" substr($1, 2, 2) ":" substr($1, 4, 2) ":" substr($1, 6, 2) ')
kvm \
  $ PREVIOUS_ARGS  \
  -net nic,model=virtio,macaddr=$mac,vlan=$net \
  -net vde,sock=$TMP/switch-$net.sock,vlan=$net

The use of a VDE switch allows me to run the lab as a non-root user. It is possible to give Internet access to each VM, either by using -net user flag or using slirpvde on a special switch. I prefer the latest solution since it will allow the VM to speak to each others.

Debugging This lab was mostly done to debug both the kernel and Quagga. Each of them can be debugged remotely.

Kernel debugging While the kernel features KGDB, its own debugger, compatible with GDB, it is easier to use the remote GDB server built inside KVM.

kvm \
  $ PREVIOUS_ARGS  \
  -gdb unix:$TMP/vm-$name-gdb.pipe,server,nowait

To connect to the remote GDB server from the host, first locate the vmlinux file at the root of the source tree and run GDB on it. The kernel has to be compiled with CONFIG_DEBUG_INFO=y to get the appropriate debugging symbols. Then, use socat with the Unix socket to attach to the remote debugger:

$ gdb vmlinux
GNU gdb (GDB) 7.4.1-debian
Reading symbols from /home/bernat/src/linux/vmlinux...done.
(gdb) target remote   socat UNIX:$TMP/vm-$name-gdb.pipe -
Remote debugging using   socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-gdb.pipe -
native_safe_halt () at /home/bernat/src/linux/arch/x86/include/asm/irqflags.h:50
50   
(gdb)

You can now set breakpoints and resume the execution of the kernel. It is easier to debug the kernel if optimizations are not enabled. However, it is not possible to disable them globally. You can however disable them for some files. For example, to debug net/ipv6/route.c, just add CFLAGS_route.o = -O0 to net/ipv6/Makefile, remove net/ipv6/route.o and type make.

Userland debugging To debug a program inside KVM, you can just use gdb as usual. Your $HOME directory is available and it should be therefore straightforward. However, if you want to perform some remote debugging, that s quite easy. Add a new serial port to KVM:

kvm \
  $ PREVIOUS_ARGS  \
  -chardev socket,id=charserial1,path=$TMP/vm-$name-serial.pipe,server,nowait \
  -device isa-serial,chardev=charserial1,id=serial1

Starts gdbserver in the guest:

$ libtool execute gdbserver /dev/ttyS1 zebra/zebra
Process /root/code/orange/quagga/build/zebra/.libs/lt-zebra created; pid = 800
Remote debugging using /dev/ttyS1

And from the host, you can attach to the remote process:

$ libtool execute gdb zebra/zebra
GNU gdb (GDB) 7.4.1-debian
Reading symbols from /home/bernat/code/orange/quagga/build/zebra/.libs/lt-zebra...done.
(gdb) target remote   socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-serial.pipe
Remote debugging using   socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-serial.pipe
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007ffff7dddaf0 in ?? () from /lib64/ld-linux-x86-64.so.2
(gdb)

Demo For a demo, have a look at the following video (it is also available as an Ogg Theora video).

<iframe frameborder="0" height="270" src="http://www.dailymotion.com/embed/video/xuglsg" width="480"></iframe>

---

1. stdio is configured such that signals are not enabled. KVM won t stop when receiving SIGINT. This is important for the usage we want to have.
2. Therefore, it is not possible to mound a fresh /proc on top of the existing one. I have searched a bit but didn t find why. Any comments on this is welcome.

1. It is important to rehearse your space carefully. Find a quantity of friends equal to the attendees you expect, lay out the intended room, and check that everybody has free and easy access to their chair.
2. Invest in a decent access point and some power strips with a decent cord length.
3. Relax. I cannot stress enough the importance of this step!

---

Tips for a successful BSP is a post from: jwiltshire.org.uk Flattr