Choices, options, selections if you rather. These are what set us all apart from our fellow man, perhaps it is only the appearance of free will and individuality but our world is full of choice.

In software it might be argued programming is nothing more than making thousands perhaps millions of choices which result in a list of instructions for a machine incapable of making decisions of its own.

On a more mundane level sometimes a programmer cannot make a choice suitable for all expected use cases and a user option is created. These options have become something of a cause for arguments within certain sections of the open source software community especially amongst the groups that influence the graphical user experience.

The discussion should be nuanced and varied (perhaps that is my age, or maybe I am more diplomatic than I thought?) but there seems to be little compromise on this discussion which (from an outsiders point of view) splits into two viewpoints:

• On one side of the argument, which I shall label reductionist, is that all options should be removed with the software just doing the correct thing
• On the other, whom I shall refer to as maximalist, is that users should be presented with options to customise everything.

The reductionist group is currently winning the argument in the popular graphical environments and seem to be removing functionality which requires user choice wherever they can find it. This results in the absurd "joke" that the UI will become a single button, which they are trying to remove.

Personally my view is that an option should be present only when an option cannot be satisfactorily selected by the computer and even then a default suitable for as many users as possible should be picked.

You may ask why I have raised this topic at all? well over the last few days I have been trying to fix the preferences selection for the GTK port of NetSurf. The NetSurf project follows my view on this subject pretty closely but being a browser its very difficult to do the right thing for everyone

NetSurf has numerous frontends for the core browser, I use the term frontend because in some places the toolkit and OS are conflated (windows, cocoa) and not in others (gtk). For each frontend NetSurf is a native application, this is an important distinction, the windows and widgets a user interfaces with are produced by that platforms toolkit.

Old NetSurf Preferences Dialog

This is a deliberate choice unlike other browsers which render their UI themselves as web content, this is a beguiling solution when the authors wish there to be a single browsing "experience" with a common look and feel. However NetSurf looks and feels like a native browser for each frontend, which is what the project decided it wanted to achieve.

Given this the gtk frontend (is it a Gnome application? not sure of the distinction TBH) has had no dedicated maintainer for some time it has suffered from a bad case of bitrot from both GTK library changes and general neglect.

I have slowly been improving this situation, the browser can now be compiled with GTK version 2 (2.12 onwards) or 3, the menus and other UI elements have been translated for the supported languages and now the turn has come for the options dialog.

Oh, right, its "Preferences" not options, fair enough a common semantic throughout all applications does give a degree of homogeneousness but the word choice does seem to indicate the users control has been reduced. At this point I gained an education in just how unfriendly GTK and its tools have become towards the casual programmer.

The dialog I wanted to construct was a pretty standard tabbed layout which reflected the current options and allowed the user to change them in an obvious way. Given that I have constructed and equivalent interface in idiomatic manner for cocoa and windows I thought this would be straightforward, I was very wrong.

The interface construction tool is called glade which used to be the name of the UI "resource" file and interfaces. However the tool is still called glade but the interface is now GtkBuilder which has a different (but similar) XML based file format. Then we discover that despite it being an extensible file format the UI files are specifically versioned against the GTK library. Also why on earth can these resources not be compiled into my program? OK make them over-ridable perhaps. Generally it is yet another file to distribute and update in step with the executable.

New NetSurf Preferences

So because I want to support back to version 2.12 of GTK I do not get to use any of the features from 2.16 in my UI unless I load a different UI builder file...oh and GTK3? requires a completely different version of glade and the UI files which are incompatible with gtk 2. Once this was worked round with having multiple UI files I then moved on to the next issue.

The GTK model uses function callbacks on events, these are known as signals. Perfectly reasonable but because the ui files are loaded at runtime and not compiled in there must be a way for the GTKBuilder to map the textural signal names to function pointers.

The way the GTK developers have chosen to do this is to search the global function table which means any signal handler function symbol has also to be global which means unnecessary overhead for the ELF loader increasing load times.

This lookup should have been confined to a single object or even placed in an alternative section to avoid these issues, this would not have seemed especially challenging to implement as all callback handlers have to use a preprocessor to define already (G_MODULE_EXPORT).

Another thing that makes developing GTK applications worse is the documentation, this seems to be a continuous complaint about may open source projects, which is taciturn or simply missing in many cases. GTK seems to suffer dreadfully from having multiple API versions all subtly different resulting in a lot of work on the developers part to simply find what they want.

A specific example of this is the signals and under what circumstance they occur I wanted to update all the widgets with the current configuration state whenever the options window is shown (using gtk_widget_show()) so I figured that would be the "show" signal...right? nope, never did find the right one to use, ended up with "realize" which occurs once when the dialog is created, not what I wanted, but is at least consistent and works.

NetSurf with Preferences Dialog

Overall my impression of developing just one small dialog (60 widgets total exclusing labels and containers) for a GTK program is that the toolkit and its tooling is missing the facilities to make developers life easier when doing the drudge work that a good proportion of graphical program development.

It is not the case that one cannot do things, just that everything has to be done manually instead of having the tools do the work, and because of the documentation that tedium is magnified.

I did eventually reach the stage that the thought of writing the boilerplate to add another check button to enable "do not track" had me thinking "do they really need this or can I avoid the work?" perhaps that is why they are all reductionist?

Yet another yearly summary of my running activities. It seems, indeed, that running has now slightly taken precedence over free software activities in my life and priorities for my free time. To my fellow free software friends: don't worry, I'm not on my way to stop investing my time in Debian. It's quite clear that I'm reducing my involvement, mostly because days only have 24 hours...not because I'm bored or tired by free software development. But, certainly, it became an important thing that running currently has a small priority over Debian nowadays, for me. So, what happened on that front in 2012 for running bubulle? I finally managed to run 2900 km during the year, which is over 400km more than 2011. This is mostly due to the increasing part of running in my daily commute : a typical work week can now be something like this:

• Monday morning, leave home with a big bag and clothes for up to 4 work days (including those I'm wearing) plus running clothes (and shoes).
• Monday evening: come back by running. Either 4km, then 30 minutes train ride, then 3km more....or "the long version": 10km, then 20 minutes train ride, then 3km more.
• Tuesday: go to work by run+train+run, shower, change clothes, work, come back by run+train+run
• Wednesday: ditto
• Thursday: let's have a break. I usually work in another facility where it's less convenient to go by running
• Friday morning: go to work by run+train+run
• Friday evening: come back home with a big bag of dirty clothes

With such an organization, I can end up with weeks where I run up to 50-60 kilometers in 5 days, with peaks that may include 26km in one *work* day. I also end up having room in trains between the runs, I wonder why..:-) And, of course, during week-ends, I spend some parts of my time running too..:-). Indeed, just like for many drugs, I feel sad during days where I haven't run at all and it's usually hard for me to spend more than 2 days without. Indeed, the longest time period without running, this year, has been 9 days, just after the Caen marathon, in June. Mostly because, at that time, I reached a moment where training (and believe me, very boring and hard training) consumed all my motivation. But these moments are indeed very rare and training is mostly *never* boring for me. Mostly because I like running in the nature, in forests, woods, fields, country. Never wearing an MP3 player or any kind of such device toplay music, but just enjoying the outside, whether it's raining, winding, sunny hot. Running (particularly my daily commute) is also the moment where I *think* about my work, my technical activities, my own life, whatever. I think I even sometimes translated some software, mentally, while running..:) Indeed, there have been 205 days in this year where I ran at least once. Last year was 181 so....drug addiction is increasing. So, 2900 kilometers. That's about the distance from my place to Moscow in Russia. 282 hours (11 days18 hours.....9d21h last year). Average speed: 10,3km/h (10,5km/h last year). Cumulated height difference: 33,400 meters (27700m last year). More distance, more time, a bit slower: this is an obvious consequence of more trail running (which includes more difficulties, such as running on volcanoes!). Most active month: November with 331km. Less active month: June with 141km. This year was also a year of records:

• personal best broken in September by....14 seconds....on half-marathon with 1h37'28" after failing a first time by 20 seconds in March
• personal best broken twice in the two marathons I ran. first in June, in Caen, after a very hard 8-week training and in difficult conditions (rain, wind, hard race profile) with 3h38'45". Then, I managed to run a stunning 3h33'35" in October in Val du Rueil (a small town in the Seine river valley). That one will be hard to break now.
• longest distance run, with the "80km" Paris Ecotrail run in March, in 11 hours
• great progress in my other "ultra" race (70km in November) which I ran in 8h22 instead of 9h15 last year

I ran 10 official races during the year : two "ultra" races (70km Le Puy-Firminy in November, by night and 80km Paris Ecotrail in March), two marathons (Caen and Val de Rueil), four half-marathons and two "short" (less than 30km) trail races. How about next^W this year? Well, my goals are currently being secured:

• more trail races than roads races
• Paris 80km Ecotrail again in March. Goal: beat last year's 11 hours
• Le Puy-Firminy 70km again in November. A very nice "small" race (small by number of participants....but the effort is not small!).
• Mont-Blanc "marathon" in late June. Indeed a 42km trail race in Chamonix, starting at 1000m, finishing at 1600m, with a peak at 2300m and 2500m positive vertical gain (1490m negative).
• Another "long" trail race (40-60km) somewhere in April/May
• A "classic" marathon in October/November. Probably somewhere in France. Maybe La Rochelle. That might be the only classic marathon I run this year...unless I decide for another challenge like running two marathons in two consecutive Sundays..:-)
• Do once again my own personal marathon which is just "come back from work by foot".....which nearly equals the distance of a marathon. Something I achieved for the first time in September, in about 5 hours
• Maybe try a 6 or 12 hour race (running for 6 or 12 hours...or more...on a small 1-2km loop). These ones are very friendly ones and the only races where lame runners like me can run along with the best ones, not only for the first 100 meters.

All this of course is assuming that no injury comes up (my ankles are sometimes yelling outbut I'm fortunate enough to not have articulation problems that many runners have, particularly in knees). We'll see on January 1st 2014...:-)

Review: City of Diamond, by Jane Emerson

Publisher:	DAW
Copyright:	March 1996
ISBN:	0-88677-704-6
Format:	Mass market
Pages:	624

The setting is deep into the star-faring future of humanity. Humans have settled many other worlds, thanks largely to technology from a race named the Curosa. There is a thriving network of settlement based around a sort of warp conduit network. But, standing apart from most human settlements, are three huge ships: the City of Opal, the City of Pearl, and the City of Diamond. Their inhabitants are devout members of the Redemptionist faith, a merging of Christian (primarily Catholic) belief and Curosa religion. Their ships are gifts from the now-departed Curosa, featuring drives unknown to the rest of humanity that allows them to ignore the normal transport network. They are meant to witness and spread their religion, but now, six centuries later, they primarily trade. Much of this is not apparent at the start. City of Diamond opens with Adrian Mercati, chosen heir of the current dying Protector, dodging people in what seems to be an oddly-organized city. Impudent, occasionally flippant, utterly charming, and in love with danger, he inherits the position of Protector shortly after we meet him. Emerson slowly reveals more of the nature of the ship while Adrian is shocking (and charming) the nobility and the church, and consorting with demons. Or, at least, one demon: Tal, an entirely human-appearing creature except for his eyes, who the Redemptionists claim is born without a soul. Slowly, it becomes clear that he's actually the offspring of an alien and a human, a mating of species whose children cannot socialize as human and behave in ways that seem like human sociopaths. Tal is no exception, but his interests align with Adrian's for the time. To those two viewpoint characters, Emerson adds four more: a low-rank administrator named Spider who works for Tal after Tal saved him from death by recycling, Adrian's politically-arranged bride-to-be from the rival (and even more religious) ship City of Opal, a guard from a French-descended slum on City of Opal, and a dangerously honorable and literal mercenary from an off-shoot culture of humanity that's now only a legend. This is a book that tends to sprawl. There's quite a lot going on, starting with the negotiations between City of Diamond and City of Opal to put to rest the remnants of a civil war and leading to the search for a legendary Curosa artifact on the planet they're both visiting. All of the characters have their own agendas, their own history, and their own goals, and are pursuing them in parallel and occasionally in opposition. Emerson takes the time to let all of this build. Despite that, there is very little in this book that drags or bores, and that's due primarily to the strength of the characters. Every one of the viewpoint characters is unique, bringing a different perspective to the story, and every one of them changes and grow over the course of the story. They also play off of each other in delightful ways. This is a book that makes full use of the fact that three characters make seven character interactions. For example, Tal by himself is logical but just slightly askew, Tal seen by Spider is uncomfortable and worrisome, Tal and Adrian is a dance of mutual amusement and respect, and Tal and Keylinn finding points of commonality is just delightful. I did wish that there was a bit more Keylinn and Tal and a bit less of Spider, but once Emerson built her world, introduced her characters, and got the story moving, there aren't many weak pieces. Part of the fun is figuring out the world background. There are almost no infodumps here; Emerson just starts telling stories of characters in the surroundings in which they find themselves, and the reader is left to piece things together from background description, the occasional explanation from one character to another, and parallels with long-standing SF tropes. It's not a puzzle book, but it is a book of where the reader is dropped as an alien into a fully-developed culture. In both that way and in the use of nobility, religion, and formal hierarchy against a starfaring backdrop, City of Diamond reminds me of the sort of classic space opera that led up to Star Wars but isn't seen as much these days. It's also funny. I was unsurprised to discover that Emerson is a pseudonym for Doris Egan, who (in addition to writing some other SF) is an occasional writer and producer for House. The humor isn't entirely obvious at the start, although Adrian frequently provokes a grin from the start of the book, but Tal's perceptive but obliviously blunt comments become more and more fun as the book progresses, and both Iolanthe (Adrian's bride) and Keylinn (the mercenary, sort of) are a delight. I want to see more of both of them, but sadly the book ends just as they (particularly Iolanthe) are hitting their strides. And that's the biggest problem with City of Diamond: it's a failed (or at least postponed) trilogy. As with Walter Jon Williams's Metropolitan and City on Fire, there were supposed to be more books, but apparently either the publisher lost interest or other projects took precedence. The story does come to something of a conclusion, but most of the major story arcs aren't resolved. Emerson is still bringing more guns onto the set late in the book. None of the villains truly get their just desserts, nor do most of the character arcs fully resolve, although friendships and more have formed satisfyingly by the end of the book. There is obviously far more to both these characters and this universe, including an entire ship that was never visited and an alien whose presence should prove startling. This is a great book: warm, funny, full of interesting characters, and with a nicely complex background universe. If it were complete in itself, or if the sequels had materialized, I would recommend it unreservedly. As is, you will have to weigh your dislike of unfinished stories against its stand-alone merits. But I enjoyed it a great deal. Rating: 7 out of 10

One things I never quite got around to during 2012 was to blog about the awesome relay we ran in early June. This was the race formerly known as MC200 (for Madiston, WI, to Chicago, IL, by way of Milwaukee, WI, for about 200 miles) and is now part of the Ragnar Relay series: Ragnar Chicago. We ran as a so-called ultra team of six runners, as opposed to a regular ream of twelve. The course is cut into 36 segments; on regular teams you get 3, we each had twice that. My first leg was a combined 17 miles in what turned out to be pretty blistering heat in mid-to-late afternoon. My fellow team members were awesome in getting me lots of water an ice, and I managed to hold onto a pace of just over 8 min/miles. One of the harder runs I've had. Next was a wonderful run pretty much exactly at midnight under starry skies---about seven or so miles followed by ten more miles the next morning. We ended up coming third (yay!) beating the next time by about six or seven seconds (!!) over a total time of 25 or 26 hours. It was hard. It was fun. It was exhilirating. It may also have broken me as I haven't really run much since. So good intentions for 2013: get back into the groove.

mksh made quite some waves (machine translation of the third article) recently. Let s state it s not just Amigas ara5 is a buildd running the Atari kernel, an emulated though. On the other hand, the bare-metal Ataris used to be the fastest buildds, so I expect we get them back online soonish. I m currently fighting with some buildd software bugfixes, but once they re in, we will make more of them. Oh, and porterboxen! Does anyone want to host a VM with a porterbox? Requirements: wheezy host system (can be emulated), 1 GiB RAM, one CPU core with about 6500 BogoMIPS or more (so the emulated system has decent speed; an AMD Phenom II X4 3.2 GHz does just fine). Oh, and mksh is ported to more and more platforms, like 386BSD 0.0 with GCC 1.39, and QNX 4 with Watcom and more bugfixes are also being worked on. And let s not forget features! jupp got refreshed: it s got a bracketed paste mode, which is even auto-enabled on xterm-xfree86 (though the xterm(1) in MirBSD s a tad too old to know it; will update that later, just imported sendmail(8) 8.14.6 and lynx(1) 2.8.8dev.15 into base, more to come) and will be enhanced later (should disable auto-indent, wordwrap, status line updates, and possibly more), lots of new functions and bindings, now uses mkstemp(3) to create backup files race-free, and more (read the NEWS file). In MirBSD, Benny and I just added a number of errnos, mostly for SUSv4 compliance and being able to compile more software from pkgsrc without needing to patch. This is being tested right now (although I should probably go out and watch fireworks in less than a half-hour), together with the new imports and the bunch of small fixes we accumulate (even though most development in MirBSD is currently in mksh(1) and similar doesn t mean that all is, or worse, we were dead, which we aren t). I ll publish a new snapshot some time in January. The Grml 2012.12 also contains a pretty up-to-date MirBSD, with a boot(8/i386)loader that now ignores GUID partition table entries when deciding what to use for the a slice. If you haven t already done so, read Benjamin Mako Hill s writings!

It was too much to hope for that all that testing would catch everything in a 4.00 release, but it's ironic that the main bug was a bug in one of the test cases! BlueT - Matthew Lien - found that one of the tests malfunctioned when Test::Warn was not installed. Since I was cutting a new release anyway, I also improved the README somewhat and added POD documentation to the example script. You can get the latest version from the Term::ANSIColor distribution page.

I finally published a short essay I wrote about a year ago: Freedom for Users, Not for Software. Anybody who has hung around the free software community for a while will be familiar with the confusion created by the ambiguity between "free as in price" versus "free as freedom." In the essay I argue that there is a less appreciated semantic ambiguity that arises when we begin to think that what matters is that software is free. Software doesn't need freedom, of course; Users of software need freedom. My essay looks at how the focus on free software, as opposed to on free users, has created challenges and divisions in the free software movement. The essay was recently published in Wealth of the Commons: A World Beyond Market and State, a book edited by David Bollier and Silke Helfrich and published by Levellers Press. The book includes essays by 73 authors that include some other folks from the free software and free culture communities along with a ton of people working on very different types of commons. My essay is short and has two parts: The first is basically a short introduction to free software movement. The second lays out what I see as major challenges for free software. I will point out that these are some of the areas that I am working most closely with the FSF who are having their annual fundraiser at the moment to support and build advocacy programs around.

January Started cycling again. By the end of the year I had done 2,300 miles. (Strava profile) February Private concert at the British Museum of Haydn baryton trios for the annual winding of the Thomas Tompion Mostyn clock. March Zadok Baroque premiere concert, East Sheen. (review) April Belsize Baroque concert, Bloomsbury. May Lute concert, Greenwich. June Left Playfire. July/August YCombinator, California. September Renaissance viol weekend, West Sussex. October Moved house. November Battersea Park 5K. Ran 200 miles this year. December Beethoven Triple concerto, Cambridge. (more)

• Facebook Like
• Google Plus One
• Linkedin Share Button
• Tweet Widget

This year, I've gradually realized that I no longer identify as a perl programmer. For a decade and a half, perl was the language I reached for to solve any problem that didn't have a good reason to be solved in some other language. Now I only reach for it in the occasional one-liner -- and even then I'm more likely to find myself in ghci and end up with a small haskell program. I still maintain plenty of perl code, but even when I do, I'm not thinking in perl, but traslating from some internal lambda calculus. There's quite a few new-ish perl features that I have not bothered to learn, and I can feel some of the trivia that perl encourages be kept in mind slipping gradually away. Although the evil gotchas remain fresh in my mind! More importantly, my brain's own evaluation of code has changed; it doesn't evaluate it imperatively (unless forced to by an appropriate monad), but sees the gesalt, sees the data flow, and operates lazily and sometimes, I think in parallel. The closest I can come to explaining the feeling is how you might feel when thinking about a shell pipeline, rather than a for loop. Revisiting some of my older haskell code, I could see the perl thinking that led to it. And rewriting it into pure, type-driven, code that took advantage of laziness for automatic memoization, I saw, conclusively that the way I think about code has changed. (See the difference for yourself: before after ) I hear of many people who enjoy learning lots of programming languages, one after the other. A new one every month, or year. I suspect this is a fairly shallow learning. I like to dive deep. It took me probably 6 years to fully explore every depth of perl. And I never saw a reason to do the same with python or ruby or their ilk; they're too similar to perl for it to seem worth the bother. Though they have less arcania in their learning curves and are probably better, there's not enough value to redo that process. I'm glad haskell came along as a language that is significantly different enough that it was worth learning. The deep dive for haskell goes deep indeed. I'm already 5 years in, and have more to learn now than I ever did before. I'm glad I didn't get stuck on perl. But I may be stuck on haskell now instead, for the foreseeable future. I'd sort of like to get really fluent in javascript, but only as a means to an end -- and haskell to javascript compilers are getting sufficiently good that I may avoid it. Other than that, I sense adga and coq beckoning with their promises of proof. Perhaps one of these years. Of course if Bradley Kuhn is right and perl is the new cobol, I know what I'll be doing come the unix rollover in 2038. ;)

Review: Wasted Heart, by Lynn Galli

Publisher:	Penikila
Copyright:	2010
ISBN:	1-935611-32-1
Format:	Kindle
Pages:	252

Austy is a lawyer working for the US District Attorney's office in Seattle. Six months ago, she moved there from Virginia, in large part because she fell in love with her best friend. Her married best friend. Now, she's trying (mostly unsuccessfully) to deal with the Seattle lesbian scene while becoming (successfully) the go-to lawyer for prosecuting cases with a substantial technology component. Two things are about to complicate that life: meeting and working with FBI special agent Elise Bridie, and working on a case that involves her best friend's company. Yes, that best friend. This is, as you might have guessed, a lesbian romance novel, so with that should come my "reading out of genre disclaimer": it's rather easy, when reading a genre one normally doesn't read, to find books fresh and interesting when they're unoriginal to regular genre readers, or to miss genre conventions or reactions because one has not read the previous work. You should therefore take my opinions with a grain of salt, particularly if you read more romance novels than I do (which wouldn't be hard). I grabbed this book for a few reasons: I felt like reading a warm-hearted story focused on human relationships, I wanted a reasonably fast read (since I'm obsessed with meaningless numeric milestones, like how many books I read in a year), and I wanted something with an upbeat ending. Romance generally delivers on those fronts. Lesbian romance has the added bonus of avoiding a whole bunch of gender essentialism and heterosexual romance tropes that tend to give me hives. Landing was excellent, so I thought I'd see if I could find more of that. Wasted Heart does indeed deliver, although it's not quite as good as Landing. There's a bit too much physical attraction and love based on appearance for my taste, but Galli leavens that with quite a lot of interesting psychology. Both Austy and Elise are private and cautious people, and for me that makes the romance feel more realistic. It also draws out the initial phase of getting to know the other person, which is the part of the romance that I often find the most interesting. Elise in particular is a great character maybe a little too ideal in a few places, but smart and observant in a way that's very fun to read about. Like a lot of romances, there is some annoying miscommunication and misunderstanding (predictably right around the height of the tension), but it's not quite as stupid as those problems usually are, and Galli doesn't drag it out until one starts wanting to hit the characters with blunt objects. As good or better is the community that surrounds the main characters. Galli has a full cast of believable and interesting characters, most notably including Austy's old gang from Virginia. The tensions between having a group of friends one cares about deeply, wanting to build a new life after making a major break and major decision but having difficulty sustaining that under the onslaught of those friends, and being easily tossed about by expressive and outgoing people when one is private and a bit quiet all play out here. And, refreshingly, without a lot of angst, and with quite a bit of intelligent introspection. Both Austy and her friends are smarter and more considerate than most of the characters I've seen in this sort of situation. Another good bit is the little pieces of legal drama. Galli doesn't neglect Austy's work life, and while this is certainly not a legal thriller or procedural, we do get some nice scenes of Austy doing her job. I love courtroom scenes, particularly ones (like these) that are more on the realistic side, so that was a nice bonus. Austy and Elise's jobs also lead to a somewhat predictable bit of drama, and I was a bit less enamored of that; the more I thought about it, the less well it hung together for me. But I did like that it stayed firmly on the realistic side, including a look at psychological aftereffects for real people rather than thriller protagonists. This is not the sort of book that I would tell someone to rush out and buy, but it met my criteria nicely and delivered just the sort of entertainment I was looking for. (And it's not particularly expensive on Kindle, at least in the US.) There was a bit too much focus on physical appearance and physical infatuation, and Elise might be just a little too perfect, but I enjoyed it. Note that I read the special edition, which includes a (much appreciated) epilogue that closes out the story. That seems to be the default edition for sale everywhere I look, but in case you see the older version, I recommend the special edition. Rating: 7 out of 10

hey, this week the list of RC bugs I touched is even shorter than in the last weeks:

• #692979 libimager-qrcode-perl: "libimager-qrcode-perl: expects older version of Imager than dependencies supply"
  manual close after successful binNUMs
• #696329 lemonldap-ng: "lemonldap-ng: CVE-2012-6426: SAML messages signatures are not verified"
  upload package prepared by xguimard to testing-proposed-update (pkg-perl)

A while ago I wrote a blog post debunking the idea that swap should be twice as large as RAM [1]. The issue of swap size has come up for discussion in a mailing list so it seems that it s worth another blog post. Swap Size In my previous post I suggested making swap space equal to RAM size for systems with less than 1G of RAM and half RAM size for systems with 2-4G of RAM with a maximum of 2G for any system. Generally it s better to have the kernel Oom handler kill a memory hungry process than to have the entire system go so slow that a hardware reset is needed. I wrote memlockd to lock the important system programs and the shared objects and data files they use into RAM to make it easier to recover from some bad paging situations [2], but even that won t always make it possible to login to a thrashing system in a reasonable amount of time. But it s not always good to reduce swap size. Sometimes if you don t have enough swap then performance sucks in situations where it would be OK if there was more swap. One factor in this regard is that pages mapped read-only from files are discarded when there is great memory pressure and no swap space to page-out the read-write pages. This can mean that you get thrashing among memory for executables and shared objects while there s lots of unused data pages in RAM that can t be paged out. One noteworthy corner case in this regard is Chromium which seems to take a lot of RAM if you have many tabs open for a long time. Another factor is that there is a reasonable amount of memory allocated which will almost never get used (EG all the X based workstations which have 6 gettys running on virtual consoles which will almost never be used). So a system which has a large amount of RAM relative to it s use (EG a single-user workstation with 8G of RAM) can still benefit from having a small amount of swap to allow larger caches. One workstation I run has 8G of RAM for a single user and typically has about 200M of the 1G swap space in use. It doesn t have enough swap to make much difference if really memory hungry programs are run, but having 4G of memory for cache instead of 3.8G might make a difference to system performance. So even systems which can run without using any swap can still be expected to give better performance if there is some swap space for programs that are very inactive. I have some systems with as much 4G of swap, but they are for corner cases where swap is allocated but there isn t a lot of paging going on. For example I have a workstation with 3G of RAM and 4G of swap for the benefit of Chromium. Hardware Developments Disk IO performance hasn t increased much when compared to increases in RAM size over the last ~15 years. A low-end 1988 era hard drive could sustain 512KB/s contiguous transfer rates and had random access times of about 28ms. A modern SATA disk will have contiguous transfer rates getting close to 200MB/s and random access times less than 4ms. So we are looking at about 400* the contiguous performance and about 10* the random access since 1988. In 1988 2MB was a lot of RAM for a PC, now no-one would buy a new PC with less than 4G so we are looking at something like 2000* the RAM size in the same period. When comparing with 1993 (when I first ran a Linux server 24*7) it was 4M of RAM, maybe 15ms random access, and maybe 1MB/s contiguous IO approximately double everything I had in 1988. But I ll use the numbers from 1988 because I m more certain of them even though I never ran an OS capable of proper memory management on the 1988 PC. In the most unrealistically ideal situation paging IO will be a factor of 5* more expensive now relatively than it was in 1988 (RAM size increase by a factor of 2000 divided by contiguous IO performance increase by a factor of 400). But in a more realistic situation (random IO) it will be 200* more expensive relatively than it was. In the early 90s a Linux system with swap use equal or greater than RAM could perform well (my recollection is that a system with 4M of RAM and 4M of active swap performed well and the same system with 8M of active swap was usable). But nowadays there s no chance of a system that has 4G of RAM and 4G of swap in active use being usable in any way unless you have some corner case of an application allocating RAM and not using it much. Note that 4G of RAM doesn t make a big system by today s standards, so it might be reasonable to compare 2M in 1988 to 8G or 16G today. But I think that assuming 4G of RAM for the purpose of comparison makes my point. Using Lots of Swap It is probably possible to have some background Chromium tabs using 4G of paged out RAM with good system performance. I ve had a system use 2G of swap for Chromium and perform well overall. There are surely many other ways that a user can launch processes that use a lot of memory and not use them actively. But this will probably require that the user know how the system works and alter their usage patterns. If you have a lot of swap in use and one application starts to run slowly you really don t want to flip to another application which would only make things worse. If you use tmpfs for /tmp then you need enough swap to store everything that is written to /tmp. There is no real upper limit to how much data that could be apart from the fact that applications and users generally don t want to write big data files that disappear on reboot. I generally only use a tmpfs for /tmp on servers. Programs that run on servers tend not to store a large amount of data in /tmp and that data is usually very temporary, unlike workstation users who store large video files in /tmp and leave them there until the next reboot. Are there any other common corner cases that allow using a lot of swap space without serious performance problems? My Swap Space Allocation When I setup Linux workstations for other people I generally use 1G of swap. For a system that often has multiple users (via the user-switch feature) I use 2G of swap if it only has 3G of RAM (I run many systems which are limited to 3G of RAM by their chipset). For systems that I personally use I generally allocate 4G of swap, this is to cater to excessive RAM use by Chromium and also because I test out random memory hungry programs and can t easily predict how much swap space will be needed a typical end-user desktop system is a lot more predictable than a workstation used for software development. For servers I generally allocate 512M of swap space. That s enough to page out things that aren t used and make space for cache memory. If that turns out to be inadequate then it s probably best to just let the watchdog daemon reboot the system.

• [1] http://etbe.coker.com.au/2007/09/28/swap-space/
• [2] http://doc.coker.com.au/projects/memlockd/

For some time, I have been concerned about the amount of data I store on Google. This post aims to show alternatives to the service. This is primarily for two reasons. The first is simple. I would like to avoid a single point of failure, and the risk that I could be suspended for a number of months with no recourse. The second if more political. I'm worried about the way that certain governments can subpoena information which they may not be able to collect directly. Now, don't get me wrong. Google is good at transparency. They also provide a very useful service that works incredibly well. In fact, the quality of the integration and services provided is the primary reason I have not changed previously. Calendaring is something that I naturally feel shouldn't be hard, but somehow is. Google Calendaring service is rather good, and far outstrips anything open source I've found so far. However, I've managed to migrate to Zoho which offers a very similar level of functionality. It's also very easy to extract your entire calendar from Google, which makes the transition easier. Reader is a service provided to subscribe and read RSS feeds. In the past, I've used Gregarius as a self hosted solution, but this seems to have disappeared recently. I briefly flirted with NetVibes, which works well once you turn off the frustrating 'widget' mode, but has completely failed to address the need to have native tablet apps. In the end, I settled for NewsBlur which also has the clever use of hotkeys for navigation which I found essential in Google's products. Picasa has stood me well for hosting photos for some time. And photo hosting is something I have quite a large amount of professional experience with :) Despite having previously been in competition with them, I can heartily recommend Flickr, even with it's dogmatic reliance to creating a Yahoo! email address for each account. Search still has some way to go to find a viable replacement. I've tried DuckDuckGo, but found it lacking. Much like the mistake that Apple seem to have made with the launch of Apple Maps, there seems to be a misunderstanding that search is actually HARD.
There are two camps when it comes to searching. You can be dumb, or you can be smart. By being dumb, the user can provide enough information to help narrow the results down to a single (or at least manageable) result set. This works fine, and is a traditional model. The opposite is to be smart. Simply get the results right, using a variety of metrics to work out what the user wanted in the first place. This is obviously more difficult, but infinitely more valuable, and is something that Google excels at. The problem lies when you fall in between these stalls. Unfortunately, it seems that there is no viable alternative to the big G at the moment.

Installing an operating system When you want to install a new operating system, the first thing to do is to get an installer medium, which usually involves two steps: downloading an image, and writing it to a medium. There are two major types of media usable for that purpose: optical discs and USB sticks. There used to be a third one, floppy disks, but it is no longer used, and it was conceptually very similar to USB sticks.Power-user If you are a power-user, you probably use USB stick, to be able to reuse them and because they are easier to carry. For you, writing an optical disc or an USB drive is almost the same:

$ wodim image.iso
$ dd if=image.img of=/dev/sdc

Regular user But what would you suggest to a normal user? USB stick? You should not: while it is technically much easier to write an USB stick (just copy blocks from the image to the peripheral file, without any need to use any specific function), desktop file manager make it much harder. Indeed, for a desktop user, burning an ISO image to an optical disk is quite simple: right-click on the image file and choose the burn option. But writing an image to an USB stick? There is no similar option, so that will involve calling a power-user friend, which will tell you to open a terminal and type dd if= . Moral

Power-users

Continue using USB sticks as installation media, but tell regular users to use optical discs until desktop file managers adapt.

Regular users

Use optical discs. If your computer does not have an optical reader, either invite a power user friend, or buy an USB optical reader.

Desktop developers

Please, fix that! Writing an image to an USB stick should not be harder than burning it to an optical disc! People are using more and more computers without an optical reader, and that situation is certainly not new, as it is only similar to that of the time when we were using floppy disks.

Courtesy of a recent Google+ post/Stack overflow answer, I stumbled upon ghc-gc-tune, a simple but nice tool which generates interesting graphs for Haskell programs. What is does is quite trivial: iterate over a range of arena and heap sizes, and run a specified program with those RTS options, then generate a graph comparing the performance (by default cpu time) across the combinations of the values. Note that for newer GHC versions, you'll need to link with -rtsopts to allow for the -A/-H custom sizes. The reason I mention it is that the graphs it generates can be quite interesting. For one of the Ganeti programs, hspace, run with the command line ./hspace --simu p,20,1t,96g,16,1 --disk-template drbd, it generates this graph: This picture, if I read it correctly, says that this program is actually well behaved (well, +RTS -s says "3 MB total memory in use" with default options), and that the optimum sizes actually relate to the (L1? L2?) cache size. But note that the maximum difference is only about ~1.6 . By changing the parameters to hspace and make it allocate more memory (./hspace --simu p,20,1t,128g,256,32 --disk-template plain --tiered=1g,128m,1, ~52MB reported by +RTS -s), the graph changes significantly: Now we have somewhat the opposite situation: very small arena sizes are detrimental (and by a big factor, 4.5 ), large arena/heap sizes are OK-ish, and the sweet spot is around 2-4MB arena size with heap sizes up to 4MB. Now maybe these particular examples were not very elightening (and they were definitely not well-conducted tests, etc.), but they should allow some intuition into how the program behaves. Plus, the tool can also generate other plots, for example peak memory usage.

Term::ANSIColor is a Perl core module that provides a few different interfaces to get at the ANSI color and text attribute escape sequences. It can be used to, for example, print out bold text or colorize output. This is a major new feature release, incorporating patches sent to me by Kurt Starsinic and Stephen Thirlwall. Kurt Starsinic contributed support for 256-color emulators. These hang special extended sequences off the otherwise-unused 38 and 48 sequences, providing an alternative naming of the base 16 colors, 216 colors arranged in an RGB namespace, and 24 shades of grey. Those colors are now available via all the normal Term::ANSIColor interfaces. He also contributed a program to print out test data for 256-color emulators that I enhanced to print out test data for the basic and 16-color sets as well, which allowed me to get rid of the static test files in the distribution. Stephen Thirlwall contributed support for creating aliases for the standard colors via an environment variable. I extended his patch to also provide a coloralias() function interface. This feature probably won't be that widely used, but it allows a user to set up custom color names for applications that take color names from user configuration (which might be handy for doing things like using the Solarized color scheme), or to define an alias like "alert" or "warning" in one place and then use it throughout the code. With the new 256-color support, it may be useful to set up more human-readable aliases for some of the RGB colors. Since I was working on this project anyway, I also redid the whole module in the new Perl coding style that I've been experimenting, based on Damian Conway's Perl Best Practices, and added a ton of new tests. There are a lot of great CPAN modules out there to do automated tests of various aspects of Perl code, and most of them don't require much effort to use. I'll probably write a separate post about that later. That sort of comprehensive review of course uncovered a bunch of microbugs and lack of clarity in the code, which is now hopefully much improved. I'm pleased to report that the test suite achieves 100% code coverage as reported by Devel::Cover (and in fact there's a maintainer-only test to ensure that it stays that way). You can get the latest version from the Term::ANSIColor distribution page.

Yesterday's initial upload of RcppClassicExamples was lacking a versioned Depends: to prevent builds on older versions of R. This has been added in a new upload 0.1.1. We also added a NEWS file (see below); no code changes were made.

Changes in version 0.1.1 (2012-12-30)

• Added versioned Depends: in DESCRIPTION to not build under older versions of Rcpp and RcppClassic

Changes in version 0.1.0 (2012-12-27)

• Initial release, split from package RcppExamples

Thanks to CRANberries, you can also look at a diff to the previous release 0.1.0. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

I have been looking at backup solution for the last few days and I am stuck, so I am asking for pointers or suggestions.

Since all backup solutions are appropriate to various needs, here are my requirements for my home network (two laptops backup on a very low powered server with a 3TiB HDD):

• Free software/open source cross-platform solution - must be able to backup both Linux and Windows clients
• network backup (to a Debian server, storage on HDD)
• very low CPU and memory needs on the server side (server is a de-underclocked Linksys NSLU2 running Debian armel)
• automatic backups with low maintenance cost and easy setup and recovery (setup once, forget about it)
• easily accessible filesystem based storage
• clients should be smart enough to detect when they aren't in the home network and not try to backup when away
• available in Debian

These are the basic requirements, and bonus-points requirements include:

• Windows clients don't need Cygwin
• optional encryption (for storage)
• default sanity checking for stored files (detection/correction of corrupt backed-up files)
• unduplication (if present, sanity checking is mandatory)
• logarithmic storage is a plus
• Web/nice interface for both client and server is a plus

According to the info I found, candidates for these criteria are Amanda and BackupPC. I like the rsnapshot idea of deduplication and incremental backups, but it's not available for Windows. BackupPC seems to be OK-ish, but it sounds too much like a bunch of Perl scripts and Windows support sounds like an after-thought and I dislike the idea of installing Cygwin just to provide some *nix tools.

I discarded Bacula, because the general impression I got from what I read is that it is hard to set up, has its own storage and has heavy needs for both clients and server. It sounds like overkill.

So what do other people recommend for my setup? Is Amanda OK? Did I get the wrong impression regarding BackupPC?

MicroPeak Serial Interface Flight Logging for MicroPeak MicroPeak was original designed as a simple peak-recording altimeter. It displays the maximum height of the last flight by blinking out numbers on the LED. Peak recording is fun and easy, but you need a log across apogee to check for unexpected bumps in baro data caused by ejection events. NAR also requires a flight log for altitude records. So, we wondered what could be done with the existing MicroPeak hardware to turn it into a flight logging altimeter. Logging the data The 8-bit ATtiny85 used in MicroPeak has 8kB of flash to store the executable code, but it also has 512B (yes, B as in bytes ) of eeprom storage for configuration data. Unlike the code flash, the little eeprom can be rewritten 100,000 times, so it should last for a lifetime of rocketry. The original MicroPeak firmware already used that to store the average ground pressure and minimum pressure (in Pascals) seen during flight; those are used to compute the maximum height that is shown on the LED. If we store just the two low-order bytes of the pressure data, we d have room left for 251 data points. That means capturing data at least every 32kPa, which is about 3km at sea level. 251 points isn t a whole lot of storage, but we really only need to capture the ascent and arc across apogee, which generally occurs within the first few seconds of flight. MicroPeak samples air pressure once every 96ms, if we record half of those samples, we ll have data every 192ms. 251 samples every 192ms captures 48 seconds of flight. A flight longer than that will just see the first 48 seconds. Of course, if apogee occurs after that limit, MicroPeak will still correctly record that value, it just won t have a continuous log. Downloading the data Having MicroPeak record data to the internal eeprom is pretty easy, but it s not a lot of use if you can t get the data into your computer. However, there aren t a whole lot of interfaces avaialble on MicroPeak. We ve only got:

• The 6-pin AVR programming header. This is how we load firmware onto MicroPeak during manufacturing. It s not locked (of course), and the hardware supports reading and writing of flash, ram and eeprom.
• The LED. We already use this to display the maximum height of the previous flight, can we blink it faster and then get the computer to read it out?

First implementation I changed the MicroPeak firmware to capture data to eeprom and made a test flight using my calibrated barometric chamber (a large syringe). I was able to read out the flight data using the AVR programming pins and got the flight logging code working that way. The plots I created looked great, but using an AVR programmer to read the data looked daunting for most people as it requires:

• An AVR programmer. Adafruit sells the surprisingly useful USBtinyISP programmer. There are Windows drivers available for this, and you can get the necessary avrdude binaries from the usbtiny page above. The programmer itself comes in kit form, so you have to solder it together. There are other programmers available for a bit more than come pre-assembled, but all of them require that you wander around the net finding the necessary drivers and programming software.
• A custom MicroPeak programming jig. We have these for sale in the Altus Metrum web store but, because they need special pogo pins and a pile of custom circuit boards, they re not cheap to make.

With the hardware running at least $120 retail, and requiring a pile of software installed from various places around the net, this approach didn t seem like a great way to let people easily capture flight data from their tiny altimeter. The Blinking LED The only other interface available is the MicroPeak LED. It s a nice LED, bright and orange and low power. But, it s still just a single LED. However, it seemed like it might be possible to have it blink out the data and create a device to watch the LED and connect that to a USB port. The simplest idea I had was to just blink out the data in asynchronous serial form; a start bit, 8 data bits and a stop bit. On the host side, I could use a regular FTDI FT230 USB to serial converter chip. Those even have a 3.3V regulator and can supply a bit of current to other components on the board, eliminating the need for an external power supply. To see the LED blink, I needed a photo-transistor that actually responds to the LED s wavelength. Most photo-transistors are designed to work with infrared light, which nicely makes the whole setup invisible. There are a few photo-transistors available which do respond in the visible range, and ROHM RPM-075PT actually has its peak sensitivity right in the same range as the LED. In between the photo-transistor and the FT230, I needed a detector circuit which would send a 1 when the light was present and a 0 when it wasn t. To me, that called for a simple comparator made from an op-amp. Set the voltage on the negative input to somewhere between light and dark and then drive the positive input from the photo-transistor; the output would swing from rail to rail. Bit-banging async The ATtiny85 has only a single serial port , which is used on MicroPeak to talk to the barometric sensor in SPI mode. So, sending data out the LED requires that it be bit-banged directly modulated with the CPU. I wanted the data transmission to go reasonably fast, so I picked a rate of 9600 baud as a target. That means sending one bit every 104 S. As the MicroPeak CPU is clocked at only 250kHz, that leaves only about 26 cycles per bit. I need all of the bits to go at exactly the same speed, so I pack the start bit, 8 data bits and stop bit into a single 16 bit value and then start sending. Of course, every pass around the loop would need to take exactly the same number of cycles, so I carefully avoided any conditional code. With that, 14 of the 26 cycles were required to just get the LED set to the right value. I padded the loop with 12 nops to make up the remaining time. At 26 cycles per bit, it s actually sending data at a bit over 9600 baud, but the FT230 doesn t seem to mind. A bit of output structure I was a bit worried about the serial converter seeing other light as random data, so I prefixed the data transmission with MP ; that made it easy to ignore anything before those two characters as probably noise. Next, I decided to checksum the whole transmission. A simple 16-bit CRC would catch most small errors; it s easy enough to re-try the operation if it fails after all. Finally, instead of sending the data in binary, I displayed each byte as two hex digits, and sent some newlines along to keep the line lengths short. This makes it easy to ship flight logs in email or whatever. Here s a sample of the final data format:

MP
dc880100fec000006800f56d8f63b059
73516447273fa93728301927d91b7712
730bbf0491fe88f7c5ee8ee896e3fadc
9dd9d3d502d1afcea2cbafc6b4c34ec1
bfbfcabf10c03dc05dc070c084c08fc0
9cc0abc0b9c0c1c0ccc0dcc020c152c4
71c9a6cf45d623db7de05ee758edd9f2
b4f9fd00aa074311631a9221c4291330
c035873b2943084bbb52695c0c67eb6b
d26ee5707472fb74a4781f7dee802b84
09860a87e786ad868a866e8659865186
4e8643863e863986368638862e862d86
2f862d86298628862a86268629862686
28862886258625862486
d925

Making the photo-transistor go fast enough The photo-transistor acts as one half of a voltage divider on the positive op-amp terminal, with a resistor making the other half. However, the photo-transistor acts a bit like a capacitor, so when I initially chose a fairly large value for the resistor, it actually took too long to switch between on and off the transistor would spend a bunch of time charging and discharging. I had to reduce the resistor to 1k for the circuit to work. Remaining hardware design I prototyped the circuit on a breadboard using a through-hole op-amp that my daughter designed into her ultrasonic guided robot and a prefabricated FTDI Friend board. I wanted to use the target photo-transistor, so I soldered a couple of short pieces of wire onto the SMT pads and stuck that into the breadboard. Once I had that working, I copied the schematic to gschem, designed a board and had three made at OSHPark for the phenomenal sum of $1.35. Aside from goofing up on the FT230 USB data pins (swapping D+ and D-), the board worked perfectly. The final hardware design includes an LED connected to the output of the comparator that makes it easier to know when things are lined up correctly, otherwise it will be essentially the same. Host software Our AltosUI code has taught us a lot about delivering code that runs on Linux, Mac OS X and Windows, so I m busy developing something based on the same underlying Java bits to support MicroPeak. Here s a sample of the graph results so far: Production plans I ve ordered a couple dozen raw boards from OSH Park, and once those are here, I ll build them and make them available for sale in a couple of weeks. The current plan is to charge $35 for the MicroPeak serial interface board, or sell it bundled with MicroPeak for $75.