I can't simply put up any longer with all the shit thrown on the face being unable to do punish such bad behavior. There are just so many other things that increase my anger (like, for example, "WTH is python2.6 still not in unstable?!") that I don't need another one.

For sure, I don't advocate or encourage anyone to join the MIA team, and I'm speaking with a lot of experience... There is a proposal, from zack, to restructure how MIA works (another bright example of collaboration...): when will it be implemented? nobody knows. will it help? I don't think so (much).

My work (MIA work in general) was almost invisible, like invisible were all the harsh and aggressive replies I got. The nice part is that those replies came from people with the worst situation. Something like:

• me: "this package has no uploads since 2005", reply: "this package has no upstream release since 1983" - 1. probably is to remove? 2. what about all the bugs filed against the *Debian* package? 3. what about the gazillion of lintian errors and warnings on that package? 4. MIA doesn't contact about only 1 package (so maybe there's something more?)...
• me: "one of your packages has 4 RC bugs", reply: "it's team maintainer!!" - yeah, so let's pretend that since it's team-maintained 'someone else' (that's never you) will work on it.
  

and several other non-sense replies doesn't leave anything but a huge frustration in you.

And when you got really pissed off, and want to see some sort of reaction on such bad and unmotivated attacks and ask the "People with Powers", what do you got? "sorry, we can't do anything to help you, it's your problem". Something should be done to protect people from fuckheads, make them behave, or else kick them out of the project as far away as we can. But what we got is nothing (and nothing seems to be changing on this side).

Well, now it's not my problem anymore, it's someone else: in perfect Debian-style.

During the month of October I spent quite a bit of time thinking about the past, present, and future of F/OSS. This was due in part to participation in a Berkman Center event on Free Culture, where we discussed the historical arc of Free Software to Free Culture, the relationships between them (and their differences), and also the content and meaning each. Over the years, what I have found so interesting about Free Software is how it left its enclave to inspire countless groups into rethinking the politics and ethics of production and access and yet, as I raised in this pod-cast interview (due to the prompting of my interviewer, Elizabeth Stark), Free Software and/or Free Culture is still pretty bounded and contained phenomenon especially when compared to something like the existing consciousness over the environmental movement, which many folks know about and understand even when and if they are not involved in doing anything for the movement. I always ask my first year students whether they know what Free Software or Free Culture is and 9 out 10 stare at me with those blank eyes that basically speak in silence: no. Now, there are a group of activists, many located in Europe, a number of them with deep roots in the social justice movement who are taking Free Culture down a different path, trying to expand its meaning and conjoin it to social justice issues, build a broad set of coalitions across the political spectrum so as to override the fragmentation that is so characteristic to contemporary political moment, and use FC as an opportunity to critique the market fundamentalism of the last few decades. If you are interested in these issues, take a look at their charter: they are looking for comments (critical and constructive) as well as endorsements (here is the long version). I myself have a few comments, for example, I think it is worth noting something like the limits of what FC can do, that even if in many ways it can be activated to do good in the world, it is also best to highlight in the same swoop that FC is not some political panacea and has limits. For example some groups in the world, notably some indigenous communities abide by a different logic of access and culture, whereby full access is not culturally or ethically desirable, as the work of Kim Christen has illuminated. I also wonder in what ways issues of labor might be addressed more forcefully, and though they briefly raise the question of environmental sustainability, it is worth expanding these more directly and deeplyas this article by Toby Miller and Richard Maxwell make clear. There is more to say but I will leave it here for now and just say it is really great to see Free Culture taken down another political path that is rooted in coalition building.

During the month of October I spent quite a bit of time thinking about the past, present, and future of F/OSS. This was due in part to participation in a Berkman Center event on Free Culture, where we discussed the historical arc of Free Software to Free Culture, the relationships between them (and their differences), and also the content and meaning each. Over the years, what I have found so interesting about Free Software is how it left its enclave to inspire countless groups into rethinking the politics and ethics of production and access and yet, as I raised in this pod-cast interview (due to the prompting of my interviewer, Elizabeth Stark), Free Software and/or Free Culture is still pretty bounded and contained phenomenon especially when compared to something like the existing consciousness over the environmental movement, which many folks know about and understand even when and if they are not involved in doing anything for the movement. I always ask my first year students whether they know what Free Software or Free Culture is and 9 out 10 stare at me with those blank eyes that basically speak in silence: no. Now, there are a group of activists, many located in Europe, a number of them with deep roots in the social justice movement who are taking Free Culture down a different path, trying to expand its meaning and conjoin it to social justice issues, build a broad set of coalitions across the political spectrum so as to override the fragmentation that is so characteristic to contemporary political moment, and use FC as an opportunity to critique the market fundamentalism of the last few decades. If you are interested in these issues, take a look at their charter: they are looking for comments (critical and constructive) as well as endorsements (here is the long version). I myself have a few comments, for example, I think it is worth noting something like the limits of what FC can do, that even if in many ways it can be activated to do good in the world, it is also best to highlight in the same swoop that FC is not some political panacea and has limits. For example some groups in the world, notably some indigenous communities abide by a different logic of access and culture, whereby full access is not culturally or ethically desirable, as the work of Kim Christen has illuminated. I also wonder in what ways issues of labor might be addressed more forcefully, and though they briefly raise the question of environmental sustainability, it is worth expanding these more directly and deeplyas this article by Toby Miller and Richard Maxwell make clear. There is more to say but I will leave it here for now and just say it is really great to see Free Culture taken down another political path that is rooted in coalition building.

I've really enjoyed reading some of Matthew Garrett's entries about legacy PC hardware features - specifically the cute hack involving A20 and the keyboard controller. Reading things like that really takes me back. I remember when the z80 was cutting edge, and I discovered you could switch to a whole new set of shadow registers via "exx" and "ex af,af'". I remember using undocumented opcodes, and even now I can assemble and disassemble simple z80 machine code. (Don't get me started on the speedlock protectors and their fiendish use of the R register; that'll stick in your mind if you cracked it. I did.) I remember being introduced to the PC, seeing subdirectories appear in MS-DOS 2.0, network redirectors appearing in DOS 3.x, and support for big hard drives appearing in MS-DOS 4.0. I remember the controvosy over the AARD code in the betas of Windows 3, and the focus that was given in the book "Undocumented DOS" which mostly focussed upon the "list of lists". (At that time I'd have been running GEM on an IBM XT with hercules (monochrome) graphics.) I remember learning about that a .COM file was a flat image, limited to 64k which loaded at offset 100h, to accomodate the PSP (program segement prefix) for compatbility with CP/M (something I've never seen, never used, and known nothing about. I just know you could use the file control blocks to get simple wildcard handling for your programs "for free") I wrote simple viral code which exploited this layout, appending new code to end of binary, replacing the first three bytes with a jump to the freshly added code. Later you could restore the original bytes back to their original contents and jump back to 100h (I even got the pun in the name of 40Hex magazine.) I recall when COM files started to go out of fashion and you had to deal with relocation and segment fixups. When MZ was an important figure. I even remember further back to when switching to protected mode was a hell of tripple-faults, switching back from protected mode to real mode was "impossible" and the delight to be had with the discovery and exploitation of "unreal mode". All these things I remember .. and yet .. they're meaningless to most now, merely old history. Adults these days might have grown up and reached age 20 having always had Windows 95 - never having seen, used, or enjoyed MS-DOS. How far have we come since then? In some ways not far at all. In other ways the rise of technology has been exponential. Back then when I was doing things I'd not have dreamed I could update a webpage from a mobile phone whilst trapped upon a stalled train. There are now more mobile phones than people in the UK. In some ways that's frightening - People miss the clear seperation between home & work for example - but in other ways .. liberation. I have no predictions for the future, but it amazing how far we've come just in my lifetime; and largely without people noticing. The industrial revolution? Did that happen with people mostly not noticing? Or was there more concious awareness? Food for thought. ObFilm: Terminator

When writing a shell script you need to take some care to ensure that it won t run amok. Extra care is needed for shell scripts that run as root, firstly because of the obvious potential for random destruction, and secondly because of the potential for interaction between accounts that can cause problems.

• One possible first step towards avoiding random destruction is to start your script with #/bin/sh -e instead of #/bin/sh , this means that the script will exit on an unexpected error, which is generally better than continuing merrily along to destroy vast swathes of data. Of course sometimes you will expect an error, in which case you can use /usr/local/bin/command-might-fail true to make it not abort on a command that might fail.
• #!/bin/sh -e
  cd /tmp/whatever
  rm -rf * #!/bin/sh
  cd /tmp/whatever exit 1
  rm -rf * Instead of using the -e switch to the shell you can put exit 1 after a command that really should succeed. For example neither of the above scripts is likely to destroy your system, while the following script is very likely to destroy your system: #!/bin/sh
  cd /tmp/whatever
  rm -rf *
• Also consider using absolute paths. rm -rf /tmp/whatever/* is as safe as the above option but also easier to read avoiding confusion tends to improve the reliability of the system. Relative paths are most useful for humans doing typing, when a program is running there is no real down-side to using long absolute paths.
• Shell scripts that cross account boundaries are a potential cause of problems, for example if a script does cd /home/user1 instead of cd ~user1 then if someone in the sysadmin team moves the user s home directory to /home2/user1 (which is not uncommon when disk space runs low) then things can happen that you don t expect and we really don t want unexpected things happening as root! Most shells don t support cd ~$1 , but that doesn t force you to use cd /home/$1 , instead you can use some shell code such as the following: #!/bin/sh
  HOME= grep ^$1 /etc/passwd head -1 cut -f6 -d:
  if [ "$HOME" = "" ]; then
  echo "no home for $1"
  exit 1
  fi
  cd ~ I expect that someone can suggest a better way of doing that. My point is not to try and show the best way of solving the problem, merely to show that hard coding assumptions about paths is not necessary. You don t need to solve a problem in the ideal way, any way that doesn t have a significant probability of making a server unavailable and denying many people the ability to do their jobs will do. Also consider using different tools, zsh supports commands such as cd ~$1 .
• When using a command such as find make sure that you appropriately limit the results, in the case of find that means using options such as -xdev, -type, and -maxdepth. If you mistakenly believe that permission mode 666 is appropriate for all files in a directory then it won t do THAT much harm. But if your find command goes wrong and starts applying such permissions to directories and crosses filesystem boundaries then your users are going to be very unhappy.
• Finally when multiple scripts use the same data consider using a configuration file. If you feel compelled to do something grossly ugly such as writing a dozen expect scripts which use the root password then at least make it an entry in a configuration file so that it can be changed in one place. It seems that every time I get a job working on some systems that other people have maintained there is at least one database, LDAP directory, or Unix root account for which the password can t be changed because no-one knows how many scripts have it hard-coded. It s usually the most important server, database, or directory too.

Please note that nothing in this post is theoretical, it s all from real observations of real systems that have been broken. Also note that this is not an attempt at making an exhaustive list of ways that people may write horrible scripts, merely enough to demonstrate the general problem and encourage people to think about ways to solve the general problems. But please submit your best examples of how scripts have broken systems as comments.

By now everybody has now heard about Go, Google's expressive, concurrent, garbage collecting language. One big, glaring thing stuck out at me when I was reading the documentation:

Do not communicate by sharing memory; instead, share memory by communicating.

One of the examples given is a semaphore using a channel, which I'll copy here for posterity. Here is a little illustration of that in operation. Serve creates goroutines via the go keyword; each of which tries to get a slot in the channel. In the example there are only 3 slots, so it acts like a semaphore of count 3. When done, each thread returns its slot to the channel, which allows anyone blocked to be woken and continued. This instantly reminded me of the very first thing you need to do if you ever want to pass Advanced Operating Systems -- write a semaphore server to provide synchronisation within your OS. In L4, threads communicate with each other via inter-process communication (IPC). IPC messages have a fixed format - you specify a target thread, bundle some data into the available slots in the IPC format and fire it off. By default you block waiting for a reply -- this all happens within a single call for efficiency. On the other side, you can write servers who are listening for remote IPC connections, where everything happens in reverse. Here's another illustration the of the trivial semaphore server concept Shehjar and I implemented. Look familiar? Instead of a blocking push of a number into a slot into a channel, you make a blocking IPC call to a remote server. My point here is that both take the approach of sharing memory via communication. When using IPC, you bundle up all your information into the available slots in the IPC message and send it. When using a channel, you bundle your information into an entry in the channel and call your goroutine. Receiving the IPC is the same as draining a channel - both result in you getting the information that was bundled into it by the caller.

IPC	Go
Start thread	Start goroutine
New thread blocks listening for IPC message	Goroutine blocks draining empty channel
Bundle information into IPC message	Bundle data into type of your channel
Send IPC to new thread	Push data into channel
Remote thread unbundles IPC	goroutine drains channel and gets data

Whenever you mention the word "microkernel", people go off the deep-end and one thing they seem to forget about is the inherent advantages of sharing state only via communication. As soon as you do that, you've broken open an amazing new tool for concurrency, which is now implicitly implied. By communicating via messages/channels rather than shared global state, it doesn't matter where you run! One of those threads in the example could be running on another computer in your cloud, marshalling up it's IPC messages/channel entries and sending them over TCP/IP -- nobody would care! At any rate, do not communicate by sharing memory; instead, share memory by communicating is certainly an idea whose time has come.

Under UNIX, it is possible to get into a position where your shell's working directory is pointing to an inode that no longer exists, but a "newer" directory with the same name exists. The easiest way of reproducing this scenario is as follows: However, as concrete example, imagine you are debugging something deep inside a generated code tree if you ran make clean and then make again from another shell, your debugging shell would be left pointing to an inode that does not exist (as it was removed by make clean), yet the make call would have made an identically named directory in its place. Shells behave rather unintuitively in this situation; ls will (correctly) return no results, but attempting to do anything useful results in curious error messages. I use the following zsh precmd hook to transparently fix this for me: The cost of the stat calls seem justified by the convenience and are in any case insignificant compared to the other things I am doing in my precmd. If you are using zmodload zsh/stat, then replace stat -c %i with stat +inode. (My previous experience with zsh tricks is that the functionality invariably exists already in some obscurely named builtin. Here's hoping.)

While examining the mail queue after a big mail server migration, I found the following reason for a bounce (hostnames replaced according to RFC2606):

550-5.1.1 - 
550-5.1.1 -
550-5.1.1 TO LEARN WHY YOUR EMAIL WAS REJECTED PLEASE GO HERE: 
550-5.1.1 - 
550-5.1.1 http://www.example.com/answers/dWtsb3R0b3NAdWtsb3R0ZXJ5LmNvLnVrPgA=AAA=/
550-5.1.1 - 
550-5.1.1 Cheap, Reliable Webhosting
550-5.1.1 http://www.example.com/a/hostgator/
550-5.1.1 -
550-5.1.1 Round-Trip Flights under $200 from Priceline!
550-5.1.1 http://www.example.com/a/pricelinertf/
550-5.1.1 -
550-5.1.1 Free Skype-to-Skype calls on your mobile
550-5.1.1 http://www.example.com/a/skype/
550-5.1.1 -
550 5.1.1 -

What s next? Advertisements in HTTP headers? Oh, I forgot, they already exist and are called referrer spam .

It s now been nearly a week since I started using the Droid, and I have some updates from my initial review. Browser First, I, like many reviewers, complained about the lack of multi-touch in the browser, thinking it inferior to the iPhone/iPod Touch as a result. I think I, and all those other reviewers, were mistaken. The great part about the Droid browser is that I can just double-tap on the content column, and it automatically zooms in to the width of that column (hiding left/right navigation or whatever else may be there). Moreover, the Droid browser also seems to render these columns narrower than the iPhone/Touch browser does. So some websites aren t viewable on the iPhone even in landscape mode without horizontal scrolling, but work great on the Droid even in portrait. Perhaps multi-touch pinching is missed in the map, but the utility of it seems far less important now that I ve played with it some. Vibration I complained that the shortcut on the lock screen puts the device into silent mode, with no corresponding shortcut to put it in vibrate mode. That s true, but there s another annoyance: if you enable a lock pattern the Droid version of a power-on password even the shortcut to silent mode is gone. Doh. Now you have to unlock the device before you can put it on silent or vibrate. Cool Stuff in Settings There s some nifty stuff in settings that some reviewers seem to have missed. You can see what services are running, and kill any of them. Even niftier, in the about the device screen, it can show you what s using battery power, and how much. It lists apps, cell modem, wifi, OS, etc. It would be nice if the iPod Touch, with its worse battery life, had this feature; my Android, at least, doesn t seem to really need it, as the battery life is fine. I also appreciate that just about everything on the Droid has more options than on the iPhone/Touch. I was in the Wifi menu on Terah s iPod Touch today, trying to figure out where the rest of the settings where. Then I realized they aren t anywhere; the iPod Touch doesn t have those as configurable options. Consistency and Duplication I get rather annoyed with UIs or devices put similar functionality multiple places. The Droid has a habit of doing this sometimes. Here s an example. Say you want to remove an app. You can, as I did, go to Settings, then Applications, and remove it from there. But later, I was wondering how you can rate applications. I noticed a Downloads screen available from the menu in the Market. That s where you rate it. You can remove apps from there too. Strangely, removing apps from the Downloads screen asks you why you re removing it, but the other screen doesn t. Wonder why that is? Camera I did check out the camera. I have very little experience using cell phone cameras. So I guess I ll say the Droid s camera is far better than the Blackberry Pearl s and far worse than my Digital Rebel. Compared to the Digital Rebel, or even an ancient 35mm camera, it s terrible. Maybe it s typical for a cell phone, and I guess it works in a pinch. I wouldn t rely on it for anything I want a print of.

I m widely know (by my peers, not globaly know, as you may guess) for being a well controlated person when it comes to spending my sacred and beloved money. Actually, I m so well controlated that I m sometimes seen as someone who hates to spend money. Sometimes I m almost convinced that people thinking that way about me are right, but then I get back to my conscious mind and realize that they think like this because people tend not to agree with what they don t practice and, generally speaking, society has been continuosly teaching them to become as consumists as possible. Obviously, I m not completely protected against consumism, as I m human and have as much desires as anyone else, but I think I m doing well on keeping myself from being taken to the path which lead us to bit the bullet and spend our money on something. What contributes to this control also demonstrates how we re used to learn better only from mistakes, as I already spent too much money on things which I initially thought I desperately needed, but which after I realized weren t actually all that much needed after all. I have been trying to avoid wasting money on an e-book reader for months. At the beggining it was a piece of cake, as there wasn t a real option (mind you that I live in Brazil, not in the USA or some other so called first world country). However, in the last few months, the scenario has been changing and it seems that these days there are actually some options available. I have been researching privately about options for months and have become well aware of all the advantages and disavantages of all the options available today. Sorry, I won t be pointing the right option to you, as your best option probably will be different from mine and recommending things isn t the point of this post. Technically speaking, the options available today seems to provide me with what I need. There are no doubts that almost any of them would be good (but not all of them would be the right ) choices and actually would represent a real improvement over the current situation. So, one might ask, why not go ahead and just buy the damn thing ? For most of the people out there, it would seem to be the right thing to do, as I would be doing conscious and well researched purchase after all. Well, I would be doing something good for myself for sure, but the government actually would need to show me some respect and prove me they have some respect for me as well. Amazon, for example, is shipping the Kindle for other countries and Brazil is one of them. I would surely just go ahead and buy it the day the shipment was announced and that was almost what I did. What prevented me from doing so was the insane/absurd takes government is applying on anything one wants to import from another country. Even after converting the price from dollars to the local currency (reais), the price is something like three to four times higher that the original price in the origin country. It s insanely prohibitive for the vast majority of citizens and one could wonder why this is so when this is a device used basically for reading. Mind you that the government has big tax reductions for importing books, for example, which are seen as culture related items and so get to win some advantages over other non-culture related items. To summarize it all, if you want to convince yourself how not to spend money on something, even when you really want to, just do the math regarding the amount of the taxes you would need to pay and I assure you there will be no regrets when latter you think why you did not bought it.

I've spent chunks of the last couple of days fighting a problem that's existed for about 25 years. The 8086 was a 16-bit processor with a 20-bit address space, limiting the maximum physical address that could be accessed to 1MB. However, quirks of the segmented memory system meant that addresses greater than 1MB could be constructed - these would wrap around to the bottom of memory. Because loading the segment registers was a time consuming operation, some programmers used this behaviour as a performance optimisation.

The 80286 introduced 24 bit address space. Unfortunately, this meant that the addresses that previously wrapped to the bottom of memory now pointed at real addresses - not ideal if you were expecting the old behaviour. IBM fixed this by tying the 21st address line (A20 - they're zero indexed) through an and gate, with the default behaviour being to keep it tied at 0 and thus maintaining the old wraparound behaviour. Applications that wanted to access the full address space needed to enable the A20 logic gate. IBM didn't want to add any extra hardware to their system if they could avoid it, so tied the other side of the and gate to a spare pin on the keyboard controller. By writing a couple of bytes to the keyboard controller, your PC-AT stopped pretending to be an XT and gave you access to all of the insanely expensive RAM it had stuffed in it. Hurray!

PCs have been emulating this behaviour since the AT was first cloned. Of course, this being the PC industry, many have got it wrong. There's a set of approaches for controlling the A20 gate that may work, varying in terms of performance and desirability. Most hardware will give the desired result (ie, I have no desire to run DOS executables from 1982, make my A20 work damnit) using any of the various methods of A20 enabling. Some hardware doesn't. The most common method used in bootloaders (where we still have access to system BIOS services) is to call int 15h with an ax of 0x2401, which asks the BIOS to enable A20 for us. This isn't implemented on all hardware, but we should get a failure back that lets us go and bang on the keyboard controller in an attempt to get it to pay attention[1].

Enter the Kohjinsha SC3.

I picked this up second hand in Japan. It's a ridiculously cute little tablet, only slightly larger than hardware that's comfortably in the MID range. It booted a Fedora liveCD perfectly, though having GMA500 graphics meant that what appeared wasn't terribly attractive. Installation proceeded happily enough, followed by a reboot and... nothing. Grub loaded the kernel and initrd, jumped to the kernel and everything hung.

So, for the past couple of days, I was stepping through the kernel setup code, trying to work out where and why it was hanging. I'd got it narrowed down to the region where the kernel tried to free the memory used by the initramfs, but the failure hopped around depending on my kernel build. Something was clearly very wrong. The strangest thing about this was that if I booted the liveCD boot menu and selected "Boot from local drive", everything worked perfectly. isolinux was clearly doing something that grub wasn't, but there's rather a lot of code to step through there.

Things became a lot easier once I found that the OpenSuse version of grub worked. Their grub has a rather smaller set of patches than ours, and only a few looked even plausibly relevant. It only took ten minutes or so to figure out that it was one that altered the A20 code. Things became much clearer then.

The main functional difference between the Suse A20 implementation and the upstream one[2] is that the Suse one explicitly tests whether the A20 enabling worked by putting values at two different addresses that would be the same if A20 is disabled. By comparing them, we know whether A20 is working properly or not. If not it can then fall back to other mechanisms. The Fedora code trusted the BIOS's claim that the int 15 call had worked. The Kohjinsha's BIOS lied, A20 remained disabled, grub copied the kernel and initramfs to chunks of address space that contained lies rather than RAM and everything fell over horribly.

Thankfully, not a difficult fix once the problem was identified. But seriously, people. How hard can it be not to screw this up?

(For an excrutiatingly detailed analysis of how hard it can be not to screw this up, see here)

[1] the Intel Macs don't implement the int 15 approach, but return a failure. They also don't have a legacy keyboard controller, so attempting to hit that resulted in grub falling over. The magic IO port approach works. Another example of how the Intel Macs aren't really PCs...

[2] grub2 implements the more paranoid check

Everyone who does any significant amount of sysadmin work will break a server. Most people who have any significant experience will have broken several. Anyone who has never broken one should be treated with suspicion by other members of the sysadmin team, they probably haven t learned the caution that most of us learn from stuffing something up badly. When you break a server there are some things that you can do to greatly mitigate the scale of the disaster. Firstly carefully watch what happens, for example don t type rm -rf * and then walk away, watch the command if it takes unusually long then press ^C and double-check that you are removing the right files. Fixing a half broken server is often easier than fixing one that has been broken properly and completely! If you don t know what to do then do nothing! Doing the wrong thing can make things worse. Seek advice from the most experienced sysadmin who is available. If backups are inadequate then leave the server in a broken state while seeking advice, particularly if you were working at the end of the day and it doesn t need to be up for a while. There is often time to seek advice by email before doing something. Do not reboot the system! Even if your backups are perfect there is probably some important data that has been recently modified and can be salvaged. Certain types of corruption (such as filesystem metadata corruption) will leave good data in memory where it can be recovered. Do not logout! If you do then you may not be able to login again and this may destroy your chances of fixing the system or recovering data. Do not terminate any programs that you have running. There have been more than a few instances where the first step towards recovering from disaster involved using an open editor session (such as vi or emacs). If the damage prevents an editor from being used (EG by removing the editor s program or a shared object it relies on) then having one running is very important. These procedures are particularly important if you are unable to visit the server. For example when using a hosted server at an ISP the more cost effective plans give you no option to ever gain physical access. So plugging the disks into another machine for recovery is not an option. Does anyone have any other suggestions as to what to do in such a catastrophy? PS This post is related to the fact that I had to recover the last couple of weeks of blog comments and posts from Google s cache Update: I got my data back, not I have to copy one day of blog stuff from one database to another.

Credit Writedowns has a populist interpretation of the latest Boom-Bust cycle [1]. It s an interesting analysis of the way the US economy is working. Bono writes for the NY Times about Rebranding America [2]. He praises Barack Obama suggesting a different reason to believe that the peace prize is deserved and describes what he believes to be the world s hope for the US. IsMyBlogWorking.com is a useful site that analyses your blog [3]. It gives some advice on how to improve some things as well as links to feed validation sites. Evgeny Morozov gave an interesting TED talk How the Net Aids Dictatorships [4]. I don t agree with his conclusion, he has some evidence to support his claims but I think that a large part of that is due to people not using the Internet well. I expect things to improve. The one claim that was particularly weak was when he mentioned radio stations in Rwanda as an example of technology being used for bad purposes the entire point about the first-world discussion about such things is the radio vs the Internet. Ray Anderson gave an inspiring talk about The Business Logic of Sustainability [5]. He transformed his carpet company, decreasing it s environmental impact by 82% and it s impact per volume of product by more than 90% while also significantly increasing it s profitability. He says that corporate managers who don t protect the environment should be regarded as criminals. Making his company more environmentally friendly reduced expenses (through efficiency), attracted more skillful employees, and attracted environmentally aware customers. Managers who don t follow Ray s example are not only doing the wrong thing for the environment, they are doing the wrong thing for their stockholders! Ray s company Flor takes carpet orders over the web [6]. They won t ship a catalogue outside the US, so presumably they only sell carpet to the US too. Marc Koska gave an interesting TED talk about a new syringe design that prevents re-use [7]. His main aim is to prevent the spread of AIDS in the developing world where even hospital staff knowingly reuse syringes. It will also do some good in developed countries that try to prohibit drug use. David Logan gave an interesting TED talk about tribal leadership [8]. His use of the word tribe seems rather different from most other uses, and I am a bit dubious about some of his points. But it is definitely a talk worth seeing and considering. Deirdre Walker is a recently retired Assistant Chief of Police who has worked for 24 years as a police officer, she describes in detail her analysis of the flaws in the TSA security checks at US airports [9]. Brian Krebs wrote an article for the Washington Post recommending that Linux Live CDs be used for Internet banking [10]. Windows trojans have been used to take over bank accounts that were accessed by security tokens, that could only be accessed by certain IP addresses, and that required two people to login. It seems that nothing less than a Linux system that is solely used for banking is adequate when a lot of money is at stake. The NY Times has an interesting review of the book Ayn Rand and the World She Made [11]. It seems that Ayn was even madder than I thought. Gary Murphy has written an interesting analysis of the latest stage in the collapse of the US Republican party [12]. The ABC (AU) Law Report has an interesting article about Evony s (of China and the US) attempting to sue Bruce Everiss (of the UK) in Australia [13]. The Guardian has an insightful article about the IEA making bogus claims about the remaining oil reserves [14]. It seems that the experts who work for the IEA estimate that oil is running out rapidly while the US is forcing them to claim otherwise. Dean Baker of the Center for Economic and Policy Research has written an interesting article about the economic effects of the war in Iraq [15]. Apparently it caused the loss of over 2,000,000 jobs considerably more than the job losses that could ever result from efforts to combat global warming.

• [1] http://www.creditwritedowns.com/2008/03/populist-interpretation-of-latest-boom.html
• [2] http://www.nytimes.com/2009/10/18/opinion/18bono.html
• [3] http://ismyblogworking.com/
• [4] http://www.ted.com/talks/evgeny_morozov_is_the_internet_what_orwell_feared.html
• [5] http://www.ted.com/talks/ray_anderson_on_the_business_logic_of_sustainability.html
• [6] http://www.flor.com/
• [7] http://www.ted.com/talks/marc_koska_the_devastating_toll_of_syringe_reuse.html
• [8] http://www.ted.com/talks/david_logan_on_tribal_leadership.html
• [9] http://www.hlswatch.com/2009/10/15/
• [10] http://voices.washingtonpost.com/securityfix/2009/10/avoid_windows_malware_bank_on.html
• [11] http://www.nytimes.com/2009/11/01/books/review/Kirsch-t.html
• [12] http://politicalmoderation.wordpress.com/2009/11/04/over-analysis-perhaps/
• [13] http://www.abc.net.au/rn/lawreport/stories/2009/2737300.htm
• [14] http://www.guardian.co.uk/environment/2009/nov/09/peak-oil-international-energy-agency
• [15] http://www.cepr.net/index.php/op-eds-&-columns/op-eds-&-columns/defense-spending-job-loss/

Earlier in the year I bought some stuff from UK clothes retailer USC. USC kindly included some gift vouchers in my delivery. This is a good example of how to sweeten a deal and make your customers happy. The vouchers did not include any expiry date on them. I banked them until the autumn, so I could maximize their potential by using them in conjunction with a heavy winter coat. I recently spotted a coat I liked a lot. It's a bit pricey at nearly 200, so I decided to cash in my vouchers. Unfortunately, the vouchers didn't work Frustrating! I emailed the company to ask them why, and it turns out that -- despite not including a printed expiry notice -- the vouchers had expired. I asked the company if they would be willing to issue me with a replacement voucher, and they stopped replying to my emails. This is a good example of how to really sour a customer's impression of you and make them look elsewhere. I did so, and found another retailer selling the coat 20 cheaper.

So yesterday we had a really nice meeting with few french Debian folks. Luk and Ralf were visiting us, so we went at Prosper's for some meat and wine (and Aberlour). We were a bit too much to be able to speak to everybody (so mainly there were two 6-packs of people), but it was fine anyway. Though, the main point of the evening was that some (most?) teams in Debian are really (*really*) understaffed, and it might become problematic in the near future (like now). For example, from what I understood:

• debian-x has one active contributor
• pkg-xfce has one active contributor
• evolution has one de factor contributor
• pkg-gnome is better since Josselin had some help, but it's still not perfect
• d-i has only one active contributor
• libc has only one active contributor

I don't know the state of other teams, but it doesn't really look fine for the incoming freeze. So if you want to give some help, even if you're not yet technical enough, feel free to offer it to the teams above. Usually, starting with just using the package is enough. If you use it, you want it well-maintained, and there _are_ stuff to do to which don't require large technical knowledge, and the (few) active contributor will be happy to have some help and point you to what you can do. If you want to give some time to Debian, before filing some ITPs (even if you'd really like to have that package you use everyday), check if you can't do something for those above teams (and others which might need help too, like release team or kernel). Oh, and don't be scared, working on Debian is fun. It's time consuming, but it's rewarding. Don't let it become not fun.

ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events. ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged. Then things get more interesting. The majority of GPEs are implemented in the ACPI tables via methods with names like _Lxx or _Exx. The xx is the number of the GPE in hex, while the leading _L or _E indicates whether the GPE is level- or edge-triggered. If an ACPI interrupt is fired and GPE 0x1D is flagged as being the source of the interrupt, the ACPI interpreter will then look for an _L1D or _E1D method. Upon finding one, it'll execute it. What this method does is entirely up to the firmware - on most HP laptops, GPE 0x1D is hooked up to the lid switch[1] and so executing it will send a notification to the OS that the lid switch has changed state. The OS will then evaluate the state of the lid switch (generally by making another ACPI query) and send the event up to userspace. How does the lid end up triggering GPE 0x1D? Things get pretty hardware specific at this point. Intel motherboard chipsets have a set of general purpose io (GPIO) lines that can, for the most part[2], be used by the system vendor for anything they want. For a lid switch, one of these lines is hooked to the switch and the BIOS configures the GPIO as an input. Pressing the switch will cause the GPIO line to become active. The GPIO lines are mapped to GPEs in a 1:1 manner, though with an offset of 16 - ie, GPIO 0xd will map to GPE 0x1d. If GPIO 0xd becomes active, GPE 0x1d will be flagged and an ACPI interrupt sent. The ACPI code will then do something to quash the interrupts, such as inverting the polarity of the GPIO[3], as well as send the notification to the OS. Why are the GPIOs offset by 16 relative to the GPEs? The lower 16 GPEs (again, talking about Intel hardware) have pre-defined purposes[4]. These range from things like "Critically low battery" to "PCIe hotplug event" down to "This device triggered a wakeup". And the latter is what I'm most interested in here. Various pieces of modern hardware can be placed into power saving states when not in use. The problem with this is that the user experience of having to turn on hardware before you can use it is not a good one, so in order to make this the default behaviour we need the hardware to tell us that something happened that requires us to wake the hardware up. There's something of a chicken and egg problem here, but thankfully most of the relevant modern hardware has out of band mechanisms to tell us about things going on. The PCI spec defines something called Power Management Events (PME), which are driven by an additional current that's supplied to the hardware even when it's otherwise turned off. On plug-in PCI Express cards, firing a PME generates an interrupt on the root bridge and a native driver can interpret that, but for legacy PCI devices and integrated chipset devices the notification has to come via ACPI. The example I've been working on is USB. It's a good choice for various reasons - firstly, there's already support for detecting when the USB controller is idle. Secondly, modern USB host controllers have support for generating PMEs on device insertion, removal or (and this is important) remote wakeup. In other words, as long as the USB bus is idle we can power down the entire USB controller. If the OS tries to access a USB device, we'll power it back up. If the user unplugs or plugs a device, we'll power it back up. If a previously idle device suddenly responds to some external input, we'll power it back up. And it's all nicely invisible to the user. How does this work? The controller retains a small amount of power even when nominally pwoered down. This is used to keep the detection circuitry alive. When it receives a wakeup event, it asserts the PME line. The chipset detects this and fires a GPE. The OS runs this GPE and receives a device notification on the ACPI representation of the USB controller, telling us to power it back up. We do so and process whatever woke us - if the bus then goes idle again, we can power down once more. The astonishing thing is that this all works. The only problem we have is that it relies on the machine vendor to have provided the ACPI methods that are associated with the GPEs. If they haven't, we can't enable this functionality - even though the hardware is capable of generating the GPEs, we have no method to execute to let us know which device has to be woken up. The GPE is never answered, we never acknowledge the PME and the hardware keeps on screaming for attention without getting any. And, more to the point, it never gets powered up and your mouse doesn't work. There's a pretty gross hack to deal with this. In general, we know what the GPE to device mappings are - they're pretty static across Intel chipsets, and while AMD ones can be programmed differently by the BIOS we can read that information back and set up a mapping ourselves. This trick also comes in handy when some vendors (like, say, Dell) manage to implement one of the GPE events wrongly. Everything looks like it should work, but the method never sends a notification because it's buggy. In that case we can unregister the existing method and implement our own instead. This code isn't upstream yet, but patches have been posted to the linux-acpi mailing list and with luck it'll be there in the 2.6.33 timeframe. My tests suggest about 0.2W saving per machine, which isn't going to save all that many polar bears but seems worth it anyway. [1] _L1D = lid. Sigh. [2] There's a few that are reserved for specific purposes [3] So where before it had to be high to be active, it now has to be low to be active - this means that it'll now trigger on the switch being opened rather than closed, so you'll get another event when you open the lid again. [4] You can find a list in the documentation for the appropriate ICH chip - the relevant section is "GPE0_STS" under the LPC interface chapter.

I was referred to Creating a World Without Poverty during a discussion on advogato some months ago, but it took rather a long time to obtain the book for various reasons and then I forgot about this review until a recent discussion on new Cooperatives-UK Chief Ed Mayo s site. Better late than never, here s my review of it. Let me know what you think of this book. About the Book Author Muhammad Yunus is founder of the Grameen Bank, Bangladesh, one of the microcredit pioneers, but that story has been told in another book Banker to the Poor. The prologue of this book starts with the creation of Grameen Danone a joint venture between the bank and the France-based Groupe Danone then part one describes problems in capitalism and introduces the idea of Social Businesses as a complement to what the book calls Profit-Maximising Businesses (PMBs), or what most people know as usual shareholder companies. Part two then covers the Grameen Group from its start in banking, through an impressive list of short company case studies some Social, some PMBs, some joint-ventures up to the first year or so of Grameen Danone. The final part then talks about creating a marketplace which is open to Social Businesses, harnessing Information Technology to spread these ideas (more on this in a later article on this site) and using those innovations to banish poverty from the world. The epilogue is the Nobel Prize Lecture Poverty Is a Threat to Peace which seemed to read almost as a simplified summary of the book s main ideas. Annoyance I got quite annoyed with this book because I feel there is a lurking absurdity in its main big idea:- On the one hand, the argument seems to be that profit-maximising businesses are problematic because of their single-minded focus on profit-maximisation. This is an argument that I largely agree with, as you may remember from my short review of Joel Bakan: The Corporation. On the other hand, it proposes to remedy this by creating Social Businesses with equally single-minded focus on social problems this seems like a recipe for disaster! If profit-maximisers threaten to wreck the environment, human rights or whatever else isn t in their goals, why won t Social Businesses using similar single-purpose company formation rules just as easily threaten to wreck whatever mentioned isn t in their goals? If this book is correct and entrepreneurs are multi-faceted human beings who will act to promote social aims (which they must if they are to invest in Social Businesses), then the only robust way forward is to promote businesses that do not try to simplify that multi-faceted nature, whether for profit-maximisation or for a subset of social aims. The best form of businesses that I know for trying to represent all interests of their owners are cooperatives. Dismissed after less than a page Cooperatives are examined in part one of the book and dismissed after less than a page (p35) because they are not inherently oriented towards helping the poor or producing any other specific social benefit (which is a misleading claim because concern for community is a basic principle of cooperatives which all should uphold), they can be owned equally by the wealthy as well as the needy (why is this a problem?) and they can sometimes be demutualised into PMBs. Ultimately, if that s the will of the people, that can t be completely prevented, but recent developments have found ways to make that much less attractive (so-called anti-carpetbagging rules). Doing what people want is not usually a problem in other spheres it s democracy. What s really infuriating is that the author proceeds to describe Social Business which are aimed at specific social benefits (rather than all), can only be owned by the needy but can access the resources they need by donations from the wealthy or by forming joint ventures with them (even very unequal ones) and can be desocialised into profit-maximisers (p178). The same problems which were used to reject cooperatives exist just as badly, if not worse! Conclusion: a bit disappointing So, ultimately, I found this book a bit disappointing. There are lots of good ideas, but when presented with a ready-made round wheel for the basic problem of tempering profit maximisation, this book reinvents an oval one. I suppose I shouldn t be surprised. The misleading criticism of cooperatives is sadly typical of much of the current social enterprise movement, which I mentioned as a drawback in the recent case study of our co-op by Cooperatives-UK. I don t know if social enterprise enthusiasts are informed by the thinking of people like Muhammad Yunus, or if the high-profile authors simply reflect the movement. Have you read this book? Did you spot this contradiction? Do you feel it has other problems that I m ignoring? Or is it a work of genius and I shouldn t be disappointed with it?

Version 0.6.7 of the Rcpp package---a set of C++ classes making it easier to glue C/C++ code to R---is now on CRAN and Debian. This is a fairly incremental release that contains just a few small changes that arose since the previous release in August. One change is that a new type RcppList was added which allows us to build R 'list' types in C++. In particular, this makes it possible to have data structures of different types and dimensions. We had not previously accomodated this as our data structures where more regularised (i.e. think matrices or vectors of various types). The need for this came out of the work with the recently started RProtoBuf package which interfaces the Google ProtoBuf library. The incredible Romain Fran ois has joined the project and added a number of nice tricks; this will need another blogpost sooner rather than later. In the meantime, check out the RProtoBuf pages at R-Forge. Another small change was the addition of a self-contained example of a function callback from C++ into R which wasn't all that well documented before; this was in response to a user request. Finally, I started RcppList as a first step in reorganising the code a little better and split the class header and body off into separate files. This required changes to the infrastrucure files Makevars, cleanup, ... and I promptly forgot the Makevars.win Windoze variant which will require a 0.6.8 release real soon.

I love bugs that are just confusing and/or break your mental model of how you think the software works. Here is a good example: Given the following HTML in Gecko, what happens when a user clicks the button?

GET /anchor-target HTTP/1.1
GET /anchor-target HTTP/1.1

2 GET requests? Okay, perhaps some sort of event bubbling problem; it's relatively common in GUI internals. But what about:

POST /form-action HTTP/1.1
GET /anchor-target HTTP/1.1
GET /anchor-target HTTP/1.1

Wait, this time it POSTs the form and makes 2 GET requests? Answers in a Bugzilla report, thanks.