ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events. ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged. Then things get more interesting. The majority of GPEs are implemented in the ACPI tables via methods with names like _Lxx or _Exx. The xx is the number of the GPE in hex, while the leading _L or _E indicates whether the GPE is level- or edge-triggered. If an ACPI interrupt is fired and GPE 0x1D is flagged as being the source of the interrupt, the ACPI interpreter will then look for an _L1D or _E1D method. Upon finding one, it'll execute it. What this method does is entirely up to the firmware - on most HP laptops, GPE 0x1D is hooked up to the lid switch[1] and so executing it will send a notification to the OS that the lid switch has changed state. The OS will then evaluate the state of the lid switch (generally by making another ACPI query) and send the event up to userspace. How does the lid end up triggering GPE 0x1D? Things get pretty hardware specific at this point. Intel motherboard chipsets have a set of general purpose io (GPIO) lines that can, for the most part[2], be used by the system vendor for anything they want. For a lid switch, one of these lines is hooked to the switch and the BIOS configures the GPIO as an input. Pressing the switch will cause the GPIO line to become active. The GPIO lines are mapped to GPEs in a 1:1 manner, though with an offset of 16 - ie, GPIO 0xd will map to GPE 0x1d. If GPIO 0xd becomes active, GPE 0x1d will be flagged and an ACPI interrupt sent. The ACPI code will then do something to quash the interrupts, such as inverting the polarity of the GPIO[3], as well as send the notification to the OS. Why are the GPIOs offset by 16 relative to the GPEs? The lower 16 GPEs (again, talking about Intel hardware) have pre-defined purposes[4]. These range from things like "Critically low battery" to "PCIe hotplug event" down to "This device triggered a wakeup". And the latter is what I'm most interested in here. Various pieces of modern hardware can be placed into power saving states when not in use. The problem with this is that the user experience of having to turn on hardware before you can use it is not a good one, so in order to make this the default behaviour we need the hardware to tell us that something happened that requires us to wake the hardware up. There's something of a chicken and egg problem here, but thankfully most of the relevant modern hardware has out of band mechanisms to tell us about things going on. The PCI spec defines something called Power Management Events (PME), which are driven by an additional current that's supplied to the hardware even when it's otherwise turned off. On plug-in PCI Express cards, firing a PME generates an interrupt on the root bridge and a native driver can interpret that, but for legacy PCI devices and integrated chipset devices the notification has to come via ACPI. The example I've been working on is USB. It's a good choice for various reasons - firstly, there's already support for detecting when the USB controller is idle. Secondly, modern USB host controllers have support for generating PMEs on device insertion, removal or (and this is important) remote wakeup. In other words, as long as the USB bus is idle we can power down the entire USB controller. If the OS tries to access a USB device, we'll power it back up. If the user unplugs or plugs a device, we'll power it back up. If a previously idle device suddenly responds to some external input, we'll power it back up. And it's all nicely invisible to the user. How does this work? The controller retains a small amount of power even when nominally pwoered down. This is used to keep the detection circuitry alive. When it receives a wakeup event, it asserts the PME line. The chipset detects this and fires a GPE. The OS runs this GPE and receives a device notification on the ACPI representation of the USB controller, telling us to power it back up. We do so and process whatever woke us - if the bus then goes idle again, we can power down once more. The astonishing thing is that this all works. The only problem we have is that it relies on the machine vendor to have provided the ACPI methods that are associated with the GPEs. If they haven't, we can't enable this functionality - even though the hardware is capable of generating the GPEs, we have no method to execute to let us know which device has to be woken up. The GPE is never answered, we never acknowledge the PME and the hardware keeps on screaming for attention without getting any. And, more to the point, it never gets powered up and your mouse doesn't work. There's a pretty gross hack to deal with this. In general, we know what the GPE to device mappings are - they're pretty static across Intel chipsets, and while AMD ones can be programmed differently by the BIOS we can read that information back and set up a mapping ourselves. This trick also comes in handy when some vendors (like, say, Dell) manage to implement one of the GPE events wrongly. Everything looks like it should work, but the method never sends a notification because it's buggy. In that case we can unregister the existing method and implement our own instead. This code isn't upstream yet, but patches have been posted to the linux-acpi mailing list and with luck it'll be there in the 2.6.33 timeframe. My tests suggest about 0.2W saving per machine, which isn't going to save all that many polar bears but seems worth it anyway. [1] _L1D = lid. Sigh. [2] There's a few that are reserved for specific purposes [3] So where before it had to be high to be active, it now has to be low to be active - this means that it'll now trigger on the switch being opened rather than closed, so you'll get another event when you open the lid again. [4] You can find a list in the documentation for the appropriate ICH chip - the relevant section is "GPE0_STS" under the LPC interface chapter.

Jersey City needs a candy stimulus.

I ve just been setting up new virtual servers at Linode [1] and Slicehost [2]. I have previously written a review of both those services [3], based on that review (and some other discussions) one of my clients now has a policy of setting up pairs of virtual servers for various projects, one server at Linode and one at Slicehost. Now both virtual hosting providers work very well and I m generally happy with both of them. But Linode seems to be a better offering. Linode has graphs of various types of usage, I can look at graphs of disk IO, CPU use, and network IO for the last 24 hours, 30 days, or for previous months. The three graphs have the same scale of the X axis so I can correlate them. The stats on Slicehost just allow you to get the current raw numbers, which doesn t help if I want to know what happened last night when performance sucked. When I build a Linode instance I can have multiple filesystems configured (Slicehost can t do any of this). I can use less disk space than is available to reserve space for other filesystems. Separating filesystems makes it easier to track IO performance and also allows some bounds to be set on the amount of disk space used for various tasks. Nowadays the use of multiple partitions is not as popular as it once was, but it s still a real benefit. Of course one of the benefits of this is that I can have two partitions on Linode that are suitable for running as the root filesystem. If an upgrade fails then it would be an option to boot with the other filesystem (I haven t actually done this but it s good to have the option). I believe that this feature of Linode could do with some improvements. Firstly when creating or resizing filesystem it should be possible to specify the number of Inodes when using Ext3. The fsck time for a large Ext3 filesystem that has the default number of Inodes is quite unreasonable. It would also be good if other filesystems such as XFS were supported, for some use cases XFS can significantly outperform Ext3 and choice is always good. When BTRFS becomes stable I expect that every hosting provider will be compelled to support it (any provider that wants my continued business will do so). Now Linode and Slicehost both allow sharing bandwidth allowances between virtual servers. So if you run one server that uses little bandwidth you can run a second server that needs a lot of bandwidth and reduce the potential for excess bandwidth use problems. The next logical extension to this is to allow sharing disk allocation between servers on the same physical system. So for example I might want to run a web server for the purpose of sending out large files, 360M of RAM as provided by the Linode 360 offering would be plenty. But the 128G of storage and 1600GB per month of bandwidth usage that is provided with the Linode 2880 plan would be really useful. At the same time a system that does computationally expensive tasks (such as a build and test server) might require a large amount of RAM such as 2880MB while requiring little disk space or bandwidth. Currently Linode allows sharing bandwidth arbitrarily between the various servers but not disk space or RAM. I don t think that this would be a really difficult feature to implement. Finally Linode has a Pending Jobs Queue that shows the last few requests to the management system and their status. It s not really necessary, but it is handy to see what has been done and it gives the sysadmin a feeling of control over the process. These management features provide enough value to me that if I was going to use a single virtual hosting provider then I would choose Linode. For certain reliability requirements it simply wouldn t be a responsible decision to trust any single hosting company. In that case I m happy to recommend both Linode and Slicehost as providers.

• [1] http://www.linode.com/
• [2] http://www.slicehost.com/
• [3] http://etbe.coker.com.au/2008/11/27/slicehost-vs-linode/

A quick bug fix released of RPostgreSQL was uploaded to CRAN earlier today; it updates last week's 0.1-5 release. An editing error left one of the helper functions broken. I also updated a few of the test scripts to also allow for optional environment variable for the port at which the PostgreSQL server is listening. This is handy when you have parallel installations of different PostgreSQL versions which Debian makes really easy. As always, all the details (including the full ChangeLog) are at the local RPostgreSQL page, the RPostgreSQL package page at CRAN and of course the Google Code repository for RPostgreSQL.

So Debian-NYC, mainly the DebConf10 local team, organized a workshop for people interested on Debian packaging. The nice people of drop.io offered their handy location in Brooklyn. So this is the first attempt to bring new comers to Debian aiming DebConf10 next summer. Micah and dkg are the ones leading the workshop so far. Pictures!

If this product should fail in your lifetime, we will replace it at no charge. If the proeducts is damaged by aggressive music listeners sliding a rail, sliding down the emergency ramp of your aircraft, slammed in your locker, slammed in your car door, run over by a car, running into a wall, getting run out of town, mountain biking, road biking, sky diving, beating your boyfriend unmercifully, getting beat down by the man, blown up in an accidental experimentation with flammable substances or damaged in any other every day experience, it means you are living your life the way we want our product used! In these, or any other damaging events, we will replace the product for a 50% discount from retail.

Skullcandy rock! I m loving my new headphones

Consider the following scenario:

You work on a project for a customer that is handled in a Subversion Repository.
The work you get comes in form of some tickets. Tickets may affect different parts
of the project, but they could also affect the same parts of the project. Testing of your work
is done by a different person. For some reasons its wanted that the fix for each ticket
is committed in only one commit in the subversion repository and only after it has been
tested. Commits for two tickets changing the same files should not be mixed.

Now: How do you avoid a mess when working with several patches that possibly affect the same files?

The answer is: git with git-svn. Currently my workflow looks the following:

1. Create a branch for each ticket
2. Make my changes for the ticket in this branch
3. Create a patch from the changes in this branch and supply the tester with it
4. Wait for feedback and eventually repeat steps 2-4
5. When testing is finished, run git rebase -i master in the branch, now squash each commit into one commit, build a proper commit message from the template git provides.
6. Switch to master branch and merge the changes from the branch.
7. Rebase master against the latest svn (git svn rebase) and dcommit

That workflow works good for me. It gives me the possibility to do micro-commits as much as I want and still only commit a well-defined, well-tested commit into the projects SVN.
Just some minor drawbacks I haven't yet solved (you know, git /can/ do everything, but that is actually a problem in itself):

• Its a bit annoying that I need to use git rebase -i and change n-1 lines for a number of n commits, so that they are squashed. It would be handy to say: Squash all commits, that happened in this branch, into one.
• Creating the diff for the tester requires me to do git log, search for the last commit before my first commit in that branch and use it with git diff <rev> to create a patch. I had a quick look at git-rev-parse and felt that is overwhelming complex to do find out how to do it better. To many possibilites. git is complex.

For now I cannot tell how good merging works, as soon as conflicts arise. But I guess git can do well, although there is the possibility that it could get complex again.

Nevertheless, its probably not a credit to git, but instead to DVCSes in common, but anyway. I like it.

I m back from Taiwan since Thursday already. It was a nice trip, the whole week and I do like Taiwan. We had a transit stop in Bangkok, pretty annoying useless stop (I m sure the smokers disagree, seeing how fast they got into the smokers lounge :) ). One nice detail at Bangkok airport are the ramps down to the Gates. If you use a wheelchair, you might end up having some fun in case you ever lose control of it. Better don t. The ramp down directly ends in a set of stairs, with nothing between which would stop you. Makes too great a jump down, hehe. We arrived in Taipei Thursday (the 24th) evening and so got rid of our luggage and went out for Dinner. Which was the only time in Taiwan I wasn t happy about the food, only one of the dishes was ok, the rest not for me. But the rest of the group did like em, so maybe it was just me after a long flight. On Friday we had nothing planned, except for Dinner, so took a very relaxed day. Somewhere around noon we went out for food, after that a bit sightseeing around the hostel . There are two attributes that are always true for Taipei:

1. It s a busy place, lots of people. No matter where.
2. It has more scooters than you ever saw before. (True, there are two times more scooters than cars).

Oh, and of course there are the same rules of the road than in Mexico: Something might be written on paper, but better don t count on that. Red traffic lights are followed (mostly), and then its whoever is stronger. But if everyone is following this, it does seem to work out. :) Friday evening we had Dinner in a place run by Taiwan aborigines where we met the other people attending the mini-DebConf. Oh, and as it was the third time I had to use them, I was slowly growing accustomed to chopsticks, a thing I was never able to get done before. (Ok, fine, if its the only way to get food and all the others are stealing your food if you can t handle them, now thats a good reason to learn to eat with chopsticks fast :) ). Saturday and Sunday we had the mini-DebConf as part of the ICOS. I took no pictures there, but a number of other people have, you can find them on gallery.debconf.org. It was a pretty nice event where I held one talk about the Debian archive and lead one discussion round about SPI and non-profit organizations behind OpenSource Projects. The slides for the archive talk are available (beware of the kittens!), the SPI stuff only had a small introduction, no slides here. Oh, before I forget it: Taiwan does feel very hot. And we found a nice notice in our hostel Monday til Wednesday was planned for just play the Tourist game and see stuff . There was a plan made to visit the Taipei 101. It turned out that Canonical has an office within that building, as well as Google. So people planned to visit their offices too. Just - what the hell would I want to visit a Canonical office? Or a Google one? Besides that it was much too early in the day and promised to be boring, so h01ger and myself skipped it. Luck did mean it good with us, we managed to get Sabrina as a local guide for the day and so went to the Taipei 101 later. The 101 is a pretty nice building, though I would have loved to be able to get up to the real top. And not just in the upper part. Anyways, we had been lucky and had a pretty nice view from it. Getting up from the 4th floor to the 89th is also an experience one has to take: The elevators go up with 1010m/min, which makes it a 37second trip only. Down is much slower, only 600m/min, pretty boring (though you do feel it more than enough in your stomach. I wouldn t want to do lift-boy job there). After the 101 we went and visited the Lungshan temple and in the evening a night market. As it was Monday the market was empty. (Where empty translates to enough people around that you never ever feel alone. Nor think its dead). Good that, I don t think I want to experience it during weekends when its busy We had a pretty good time there, with all the various little shops. Fun how many old-style game places there are. Like, playing basketball, throwing things at other things to win stuff, all those non-computer-using games. In a country where lots of the high tech comes from this is not exactly what you would expect. On Tuesday our trip continued as good as it started on Monday, as we had the pleasure to keep Sabrina as guide for Tuesday and Wednesday. So our Tuesday started out with a visit to the National Palace Museum. Which has a large enough collection that you can spend multiple days exploring it. We didn t have the time nor memory for all of that, so left it after half a day. Lunch we took in a Japanese restaurant and here it was the first and only time during my Taiwan visit that I used a fork. Well, none of the dishes would have worked otherwise. The afternoon we spent in the Yangming Park, so we actually saw a bit of nature near Taipei. Tuesday did end early for us, as we planned to attend the TOSSUG meeting, where I met some more people from debian.org.tw and had an interesting chat about non-profit organizations in Taiwan, their way of operation and what to think of when trying to create a new one. I ve also got a nice debian.org.tw shirt, but I won t ever be able to wear it. They didn t expect someone that large when producing them, oh well. (Hrr, people, if you make Debian shirts of any kind, always do some XXL too. There might be fat, large guys like me visiting. :) ) Anyway, they have another thing, one which I like very much. They created a map of usergroups of other open source projects in Taiwan. It s a foldable map style, titled FLOSS Community Manual , and a lot of different Projects/Usergroups/ are listed there. While its all written in Chinese and I can t read it, it does list them, then describes what they are/do (I think), gives contact information and further links. The backside of it also seem to list events happening there. You can find more information (so you can read it :) ) at their website. Does seem a very nice idea to take. Sometime during the evening I also managed to pick up a Chinese name. Or better, Andrew did so for me. I don t even know how to type it, so copy&paste it has to be. :) There: . And that is:

• Definition in English: long, far, remote, distant; liesurely; Mandarin Pronunciation: YOU1
• Definition in English: axle; personal name of mencius; Mandarin Pronunciation: KE1

Wednesday was our last day in Taiwan, unfortunately. We went out of Taipei, visting the Golden Mountain City. Another nice place, with a market and a history, but as our Taxi was leaving at 1630 we didn t have much time for sightseeing. You could spend a lot there. Especially not as we still needed to buy postcards, something people left home always request from one. Tssk. I really enjoyed the time I had there. Not only was the organization of the conference we attended good (especially the mini-DebConf is what I saw. The event around it, while named International, was so much chinese spoken, that I couldn t make much of it), but the details around the whole stay had been nicely taken care off. And then people in general did appear very friendly. Now, it might have to do a bit with me not being exactly their height, but I dont think that is all of it. Even without such a conference as an opportunity I think I will visit the country again, but that time I will have much more time available. Like two or three weeks. And I certainly will see more than Taipei City then. Like the whole country. :) (And hope I am lucky again with local guides :) ). So thank you SLAT, Andrew, Arne, (Sabrina), Rita, (Candy) and all the people who worked for the event which names I forgot. Oh, if you didn t find it yet with all the links, a set of my pictures is available from here.

Last week's 0.1.0 release of RInside, and the first to have been published on CRAN, still had some issues with builds and use on OS X. Thanks to testing and fixes by Jan de Leeuw, Jeff Horner and particularly Simon Urbanked, things are said to be better now with the new release 0.1.1 which went onto CRAN yesterday. So no new features, but fixes to the main Makefile as well as the Makefile for the examples directory, some minor fixes and editing for the examples. I also added a file THANKS to show some appreciation for the various patches and fixes I have been receiving -- they are appreciated! However, today I committed a new example to SVN archive at R-Forge. It is based on this thread on r-devel. Abhijit Bera tries to do this in C, but to me his questions provide rather clear motivation for showing how much simpler things can be via C++ and the Rcpp classes along with RInside. Using a small example, the task was to pass a weight vector to a portfolio solver from the Rmetrics package fPortfolio and to then access the computed solution. The original poster struggled with access from C to the S4 classes used by fPortfolio and could not set the weights. But when using RInside, we simply pass a C++ vector of weights down to R, solve the problem and pass a solution vector back using the handy evaluation of R expressions:

// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Another simple example inspired by an r-devel mail by Abhijit Bera
//
// Copyright (C) 2009 Dirk Eddelbuettel and GPL'ed
#include "RInside.h"                    // for the embedded R via RInside
#include "Rcpp.h"                       // for the R / Cpp interface used for transfer
#include <iomanip>
int main(int argc, char *argv[])  
    try  
        RInside R(argc, argv);          // create an embedded R instance
        SEXP ans;
        std::string txt = "suppressMessages(library(fPortfolio))";
        if (R.parseEvalQ(txt))          // load library, no return value
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
        txt = "lppData <- 100 * LPP2005.RET[, 1:6]; "
	  "ewSpec <- portfolioSpec(); "
	  "nAssets <- ncol(lppData); ";
        if (R.parseEval(txt, ans))      // prepare problem
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	const double dvec[6] =   0.1, 0.1, 0.1, 0.1, 0.3, 0.3  ; // choose any weights you want
	const std::vector<double> w(dvec, &dvec[6]);
	R.assign( w, "weightsvec");	// assign STL vector to R's 'weightsvec' variable
	txt = "setWeights(ewSpec) <- weightsvec";
        if (R.parseEvalQ(txt))		// evaluate assignment
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	txt = "ewPortfolio <- feasiblePortfolio(data = lppData, spec = ewSpec, constraints = \"LongOnly\"); "
	  "print(ewPortfolio); "
	  "vec <- getCovRiskBudgets(ewPortfolio@portfolio)";
        if (R.parseEval(txt, ans))      // assign covRiskBudget weights to ans
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	RcppVector<double> V(ans);      // convert SEXP variable to an RcppMatrix
	R.parseEval("names(vec)", ans);	// assign columns names to ans
	RcppStringVector names(ans);
	for (int i=0; i<names.size(); i++)  
	  std::cout << std::setw(16) << names(i) << "\t"
		    << std::setw(11) << V(i) << "\n";
         
      catch(std::exception& ex)  
        std::cerr << "Exception caught: " << ex.what() << std::endl;
      catch(...)  
        std::cerr << "Unknown exception caught" << std::endl;
     
    exit(0);
 

I ve been back from KDE (4.2!) to Gnome and with the mighty powers of Compiz my desktop is the envy of my Windows coworkers and still geeky functional. Well, somehow functional. For a while I wondered why I can t properly use VNC to remote-control my desktop from home. The connection works and I see the desktop but nothing seems to happen when I click or type. Now I figured out the desktop in the VNC client window is just not refreshed while my mouse and keyboard actions really do what they were supposed to on the VNC server. This is my sad tale of how I learned that a regression may mean driving the motorcycle to work in the middle of the night after cursing VNC. After a little searching through the lazyweb I found out that it s a bug reported on Ubuntu already. Apparently Compiz doesn t use the XDAMAGE extension properly. Somehow when using the graphics adapter s magic 3D powers X11 loses control of what sections are changed I m not an X11 guru. At least the client doesn t know when to redraw areas of the screen that have changed. The suggested workarounds were pretty ridiculous. Start another x11vnc session without compiz and connect to that. Very funny. I want to control my running desktop not create a new one. So I switched off compiz at work and voila VNC works again. Some eye-candy and shortcuts are gone but remote controlling the desktop is far more important while im on call. But I quickly missed Compiz. The missing eye-candy wasn t that bad. But metacity s window placement algorithm sucks as badly as back in 2002. As if I can hear it whisper: I have no idea where to put that stupid window. You tell me. I researched a little further and somehow the bug is known to Gnome, too. And other users suffering from the problem are pointed to the VirtualGL page of Wikipedia. Apparently the problem was fixed for several people until a regression was reported where things failed again. Seems like the vino developers removed the piece of code that detected the use of 3D window managers as a 2008 christmas present. Their reason: This should work fine nowadays. Hell, no, it doesn t! Somehow I downgraded vino to version 2.24.1-3 and ran the vino server manually (/usr/lib/vino/vino-server sm-disable) to see if it prints any error message. Indeed it does:

Composite detected, disabling XDamage extension.
This is a workaround while XDamage extension does not work correctly with 3d desktops.
Hopefully it will work at next vino release.

This time it worked well, too . I upgraded to the current version 2.28.0-1 (as in unstable by today) and suddenly my VNC connection had the same problems again. Aha! Let s try the Lenny version 2.22.2-1. Works, too. This has taken me hours to debug. Finally I know I can report a bug against the vino package. I wonder how long it would have taken a mortal to figure that out. [Sorry to my sponsorees who's package sponsoring requests were delayed again due to this mess.]

The NYT has an interesting article about research into treating insomnia over the internet [1]. I wonder how many other psychological issues can be effectively treated over the net. From next year all Cadbury Dairy Milk Chocolate sold in Australia will be made from fair-trade cocoa [2]. Cadbury Dairy Milk Chocolate is the most popular type of chocolate sold in Australia so this is a significant market shift. For a long time Cadbury has sold fair trade chocolate under the name Green and Black. Of course we now have to wait for Cadbury to use fair trade cocoa in all their other chocolate varieties. Mike Rowe gave an interesting TED talk about the value of manual labour [3]. He suggested that there should be a PR campaign for skilled manual labour jobs and noted that his observation (through his work on his Dirty Jobs TV series) was that the people who do some of the less popular jobs appear to be happier. The GapMinder.org web site has some interesting analysis of statistical information on countries and regions [4]. It is based on the work of Hans Rosling who is well known for his high quality TED talks [5]. Unfortunately the web site requires Flash, I will probably try it out with Gnash some time. Miru Kim gave a TED talk about her work photographing herself nude in abandoned buildings and industrial spaces [6]. Among other things she photographed herself lying naked on a pile of bones in the crypt underneath Paris which is fairly dangerous. I ve visited the crypt, it s an interesting experience but I was very careful to touch nothing you never know which of the bones came from victims of smallpox and other nasty diseases. Strangely they have an ongoing problem of visitors stealing bones, when I visited there were several bones at the exit that had been confiscated from visitors some of which had mummified flesh attached Steinar H. Gunderson wrote a good description of the basics of how the TCP protocol works [7]. He also links to a web page he wrote that will measure your potential TCP throughput and give you information on the link. This is really handy if you are behind some sort of firewall and want to know what is being done to your TCP stream when it s in transit. Apparently Christian couples tend to use a shared email account to reduce the risk of cheating [8]. It s hardly a surprise that Christians have a much higher divorce rate than atheists and agnostics [9]. The NY Times has an interesting article about iPhones overloading the AT&T network [10]. Recently I ve been having some problems sending MMS with my Three phone, some relatives who use Three have been having connection problems in certain areas with marginal signal quality, and the download speed of my Three data connection is significantly reduced (used to be ~70KB/s, now I m lucky to get 20KB/s). I suspect that the new smart phones that are being sold are largely to blame. But the up-side is that when they engineer their network to work properly with the smart phones then my Internet use (ssh and basic web browsing) will work really well. Michael Tieman wrote an interesting blog post about software patents which compares them to Land Mines [11]. Of course this analogy falls down badly while the US is still leading the world in manufacturing land mines. Rebecca Saxe gave an interesting TED talk about how brains make moral judgeents [12]. In her research she did some tests with using magnetic pulses to decrease the function of the region of the brain that allows people to judge the others and she was able to significantly affect the results of judgement tests. Brendan Scott analyses the netbook wars and concludes that it has been a significant loss for Microsoft [13]. ArsTechnica has an analysis of the real word-processing requirements [14], they suggest that in most cases MS-Word (and other word-processor) documents could be replaced with HTML or Wiki pages for a better end result.

• [1] http://www.nytimes.com/2009/08/11/health/11slee.html
• [2] http://www.news.com.au/story/0,27574,25985054-29277,00.html
• [3] http://www.ted.com/talks/lang/eng/mike_rowe_celebrates_dirty_jobs.html
• [4] http://www.gapminder.org/
• [5] http://www.ted.com/speakers/hans_rosling.html
• [6] http://www.ted.com/talks/lang/eng/miru_kim_s_underground_art.html
• [7] http://blog.sesse.net/blog/tech/2009-08-30-11-33_trying_to_understand_tcp_performance.html
• [8] http://www.smh.com.au/technology/technology-news/christian-couples-share-email-accounts-to-stay-faithful-20090907-fdaw.html
• [9] http://www.religioustolerance.org/chr_dira.htm
• [10] http://www.nytimes.com/2009/09/03/technology/companies/03att.html
• [11] http://opensource.org/node/465
• [12] http://www.ted.com/talks/rebecca_saxe_how_brains_make_moral_judgments.html
• [13] http://brendanscott.wordpress.com/2009/08/10/netbooks-microsoft-a-turning-point/
• [14] http://arstechnica.com/microsoft/news/2009/08/microsoft-word-1983 2009-rest-in-peace.ars

Although Django is well packaged for Debian, I've recently come to the conculsion that the packages are really not what I want. The problem is that my server runs Debian stable, while my development laptop runs unstable, and Django revisions definitely fall into the "unstable" category. There really is no way to use a system Django 1.1 on one side, and a system Django 1.0 on the other. After a bit of work, I think I've got something together that works, and I post it here in the hope it is useful for someone else. This info has been gleaned from similar references such as this and this. This is aimed at running a server using Debian stable (5.0) for production and an unstable environment for development. You actually need both to get this running. This is based on a project called "project" that lives in /var/www

1. First step is to install python-virtualenv on both.
2. Create a virtualenv on both, using the --no-site-packages to make it a stand-alone environment. This is like a chroot for python.

   $ virtualenv --no-site-packages project
   New python executable in project/bin/python
   Installing setuptools............done.
   

3. The unstable environment has a file you'll need to copy into the stable environment - bin/activate_this.py. The stable version of python-virtualenv isn't recent enough to include this file, but you need it to essentially switch the system python into the chrooted environment. This will come in handy later when setting up the webserver.
4. There are probably better ways to keep the two environments in sync, but I simply take a manual approach of doing everything twice, once in each. So from now on, do the following in both environments.
5. Activate the environment

   /var/www$ cd project
   /var/www/project$ . bin/activate
   (project) /var/www/project$
   

6. Use easy_install to install pip

   (project) /var/www/project$ easy_install pip
   Searching for pip
   Reading http://pypi.python.org/simple/pip/
   Reading http://pip.openplans.org
   Best match: pip 0.4
   Downloading http://pypi.python.org/packages/source/p/pip/pip-0.4.tar.gz#md5=b45714d04f8fd38fe8e3d4c7600b91a2
   Processing pip-0.4.tar.gz
   Running pip-0.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Wu9O-U/pip-0.4/egg-dist-tmp-xjSdxq
   warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
   no previously-included directories found matching 'docs/_build/_sources'
   zip_safe flag not set; analyzing archive contents...
   pip: module references __file__
   Adding pip 0.4 to easy-install.pth file
   Installing pip script to /var/www/project/bin
   Installed /var/www/project/lib/python2.5/site-packages/pip-0.4-py2.5.egg
   Processing dependencies for pip
   Finished processing dependencies for pip
   

7. Install setuptools, also using easy_install (for some reason, pip can't install it). There is a trick here, you need to specify at least version 0.6c9 or there will be issues with the SVN version on Debian stable when you try to get Django in the next step.

   (project) /var/www/project$ easy_install setuptools==0.6c9
   Searching for setuptools==0.6c9
   Reading http://pypi.python.org/simple/setuptools/
   Best match: setuptools 0.6c9
   Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg#md5=fe67c3e5a17b12c0e7c541b7ea43a8e6
   Processing setuptools-0.6c9-py2.5.egg
   Moving setuptools-0.6c9-py2.5.egg to /var/www/project/lib/python2.5/site-packages
   Removing setuptools 0.6c8 from easy-install.pth file
   Adding setuptools 0.6c9 to easy-install.pth file
   Installing easy_install script to /var/www/project/bin
   Installing easy_install-2.5 script to /var/www/project/bin
   Installed /var/www/project/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg
   Processing dependencies for setuptools==0.6c9
   Finished processing dependencies for setuptools==0.6c9
   

8. Create a requirements.txt with the path to the Django SVN for pip to install, then and then install it.

   (project) /var/www/project$ cat requirements.txt
   -e svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django
   (project) /var/www/project$ pip install -r requirements.txt
   Obtaining Django from svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django (from -r requirements.txt (line 1))
     Checking out http://code.djangoproject.com/svn/django/tags/releases/1.0.3/ to ./src/django
   (project) /var/www/project$ pip install -r requirements.txt
   Obtaining Django from svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django (from -r requirements.txt (line 1))
     Checking out http://code.djangoproject.com/svn/django/tags/releases/1.0.3/ to ./src/django
   ... so on ...
   

9. Almost there! You can keep installing more Python requirements with pip if you need, but we've got enough here to start.
10. Create a file in /var/www/project called project-python.py. This will be the Python interpreter the webserver uses, and basically exec's itself into the virtalenv. The file should contain the following:

    activate_this = "/var/www/project/bin/activate_this.py"
    execfile(activate_this, dict(__file__=activate_this))
    from django.core.handlers.modpython import handler
    

11. Now it's time to start the Django project. I like to create a new directory called project, which will be the parent directory kept in the SCM with all the code, media, templates, database (if using SQLite) etc. In this way to keep the two environments up-to-date I simply svn ci on one side, and svn co on the other.

    (project) /var/www/project$ mkdir project
    (project) /var/www/project/project$ mkdir db django media www
    (project) /var/www/project/project$ cd django/
    (project) /var/www/project/project/django$ django-admin startproject myproject
    

12. Last step now is to wire-up Apache to serve it all up. The magic is making sure you specify the correct PythonHandler that you made before to use the virtualenv, and include the right paths so you can find it and all the required Django settings.

    DocumentRoot /var/www/project
    <Location "/">
        SetHandler python-program
        PythonHandler project-python
        PythonPath "['/var/www/project/','/var/www/project/project/django/'] + sys.path"
        SetEnv DJANGO_SETTINGS_MODULE myproject.settings
        PythonDebug On
    </Location>
    Alias /media /var/www/project/project/media
    <Location "/media">
        SetHandler none
    </Location>
    <Directory "/var/www/project/project/media">
        AllowOverride none
        Order allow,deny
        Allow from all
        Options FollowSymLinks Indexes
    </Directory>
    

With all this, you should be up and running in a basic but stable environment. It's easy enough to update packages for security fixes, etc via pip after activating your virtualenv.

Yeah, OK, I suck and I should have posted about DebConf 9 while I was there or shortly afterwards. Well, I've been busy with various things (or, to be more specific, one person \o/). I'm slowly catching up on stuff now, so I thought I'd write something and link to some photos! I drove down from the UK with Neil, Chris and Chris and we had a good trip. We left Cambridge really early on Wednesday the 22nd, stopped at Le Mans for lunch, then headed to San Sebastian overnight. The French autoroutes were mostly empty and we made very good time, swapping drivers every two hours. The venue was superb, with just about all the facilities we needed on site. And there was a handy outdoor bar in the park just up the street. Yay! As always, it was excellent to meet up with Debian friends old and new from all over the world. The DebConf team did a great job of making the conference happen and I had a wonderful time, as did everybody else I spoke to. There was an official day trip involved, but as we had 2 cars from the UK a group of us rebelled and made our own trip out to do touristy things. We visited the Roman ruins in Merida (photos here), then as we were so close to Portugal we decided to head over the border and found the lovely old town of Elvas. After the conference, we headed slowly back up to the north coast of Spain, stopping over in Madrid and then Bilbao before catching the overnight ferry back to England. Along the way on that trip we saw a few sights such as the historic city of Pamplona and the "Valley of the Fallen", the monument to people killed in the Spanish Civil War. I got back to Cambridge late on Monday the 3rd, tired but happy. We'd done 2,400 or so miles in the car and several hundred more on the ferry, and we'd all had a great journey. We saw some silly stuff along the way too, and I'll post more about those soon.

I have just got a LG U990 Viewty mobile phone [1]. It s a 3G phone and came free on the $29 monthly cap plan from Three (minimum monthly spend is $29 - but this is free if you use $29 per month). My previous plan was the $29 cap but had a minimum spend of $20 per month, as I never happened to use less than $29 per month I am not paying more. For a modern mobile phone the actual phone functionality is a sideline. If a device was strictly designed to be a phone then I think it would be very similar to the Nokia phones that were available 5+ years ago - the Nokia I had in 1999 performed every phone function that I desired of it. Like most modern phones the LG U990 Viewty suffers in it s phone functionality from the desire to make it do non-phone tasks and from the desire to cripple it to meet the desires of the carriers (not the desires of the users). For example the home screen will always have at least two buttons for paid Three services and I have no configuration option to remove them. Replacing them with speed dial options for a couple of numbers that I regularly call would be handy. As the main screen is a touch-screen there is no excuse for this, they should allow the software to be reconfigured with more useful options. Eventually the Android will kill most of the other phones and this problem of phones being designed to suit the telephone companies instead of the users will be solved. One of the most annoying mis-features of the phone is that it doesn t properly handle address book entries with multiple phone numbers. One of my friends turns his mobile off when he is at home, so I regularly call his mobile and then immediately call his home number if the mobile is unavailable. With my previous two mobile phones I could press the dial button to bring up the list of previous calls and then use the arrow buttons to select from the other numbers that are attached to the same address-book entry, so with three button presses I would be dialing his other number. With the Viewty I have to go back to the address book. The compelling feature of the Viewty is the camera and display. It has a 5MP camera which makes it the second highest resolution camera-phone offered by Three - the LG Renoir (KC910) has 8MP but needs a $99 plan for it to be free. It also has a 240*320 resolution touch-screen display (in a quick search in January when I bought my phone the best resolution display I could find on a phone is 240*400 in the LG Renoir). While the camera is documented as being 5mp there are no specs available about the resolution of the CCD. I want to use the native resolution of the CCD for pictures (I think interpolation is a waste of space). The CCD might actually be 5mp, a picture of the 1400*1050 resolution screen on my Thinkpad allows me to read all the text even when small fonts are in use, so the CCD resolution must be significantly greater than 1400*1050. This is a really important feature as the Viewty will work well for making screen-shots for bug reports about crashed computers (several of my clients have expressed interest in getting one after seeing such a demonstration). One annoying problem is that the camera software takes a while to load, my trusty Sony digital camera starts a lot faster and the LG U890 phone I used for the past two years is also a lot faster and more convenient. This won t work well for photographing unexpected events. When I am traveling by public transport I will photograph the relevant pages of my street directory as I can zoom in to photos of the maps and read the street names, it saves some weight when traveling. The 2G micro-SD card (which incidentally cost $10 from OfficeWorks) will allow me to store a lot of maps. One interesting feature is the video recording capabilities. It can do 640*480 resolution at 30fps (which is pretty good) and 320*240 resolution at 120fps (they claim that you can film a balloon popping). In my quick tests the standard 640*480*30fps mode works well, but the 120fps mode requires much brighter light than most of my test environments, so I have not yet got it working properly. The phone has a reasonable voice recording function, it can record considerably more than 34 hours of audio and the quality is reasonably good if you use an external microphone. It is however quite poor if you use the built-in microphone for a dictaphone function, it seems that quality is poor at any distance. I had wanted to record my LCA mini-conf talks with my phone but unfortunately forgot to bring the adapter for the external microphone. It s a pity that the phone doesn t have a standard microphone socket as I have misplaced my Viewty microphone, when designing the phone they should assume that misplacing attachments is a common occurrence and design it to use common parts. I recently spoke to a journalist who uses his mobile phone to record interviews. He said that his phone supported phone calls, voice recording, and taking pictures - all the essential tasks for his work. It seems that the Viewty would be better than most phones for journalistic work apart from the issue of low quality voice recording when you have misplaced your external microphone. The text editor is unfriendly in the keyboard mode (I have not tried hand-writing recognition), one thing I don t like is the fact that the letters jump when you press them. This does allow changing a letter by moving the stylus before releasing the press (some people consider this a great feature). There are no cursor control keys (which is a serious omission), and the keyboard doesn t resemble a real keyboard. My iPaQ is far better for writing (I once wrote a full-length magazine article on an iPaQ). The stylus is quite strange and interesting. In the picture of the front of the phone the stylus is compacted with it s lid on. In the picture of the back of the phone the stylus is extended with the lid off. The end of the stylus clips in to the lid so that removing the lid drags the central part out of the body. It s an interesting design and with the string on the lid allows the stylus to be attached to the phone when it s not being used. But I have never used it. Even with an iPaQ (which had a proper stylus that attached firmly inside the body of the device) I often used a fingernail on the touch screen. I have not felt the need to ever use a stylus with my Viewty. One final noteworthy thing is the support for Google services. It seems to have client support for YouTube, Google Maps, GMail, and Blogger. This seems to be a major win for Google, the Viewty is one of the most popular phones at the moment and I expect that lots of people who buy them will now have an incentive to use the Google services. Between these sorts of deals and the Android I think that it will be necessary to have some sort of anti-trust action against Google. Google are generally doing good things for the users. I have been quite satisfied to use Google search, Google advertising on my blog, and Gmail. Also I have been moderately happy with Blogger (it was good when I started blogging) and Google Maps is useful on occasion. So generally I am happy with Google, but monopolies are bad for the users so I think that if things continue on their current trend then Google may have to be split into several little Googlets some time in the next few years.

• [1] http://en.wikipedia.org/wiki/Viewty

Today is towel day! So if you leave the house make sure to take a towel with you, to show your tribute to Douglas Adams and also because a towel is a very useful thing to carry around. I did and indeed it was very handy to have one, to protect against the sun and to remove sweat from my neck and face :-)

• Marco d Itri about adding support for yet-unknown-to-kernel-drivers devices. More specifically: how to make udev tell ftdi-sio that it should accept a certain VendorID/ProductID combination in addition to previously supported ones.
• Thomer M. Gil about ICMPTX, a combination made from proxy and server that tunnels IP through ICMP. Nice thing if you need a connection from some NATed and heavily firewalled location I guess. (Thanks go to Runa Sandvik for finding this.)
• Non-technical in a way, but still a nice link: Image of the Shuttle transiting the sun, thanks to Pablo Lorenzzoni for this link.
• Another post by Marco d Itri that explains how to install Debian on a SheevaPlug. Not that I need this info now, but it might come in handy some day.

That s it for now, will update the post if I find more interesting links in the next few days.