I recently bought myself a Canon EOS 500D also known as EOS Rebel T1i, and EOS Kiss Digital X3. I also got myself a Canon EF 50mm f/1.4 USM lens. One of the new things about the 500D, compared to my previous 400D, is that it has live view. So I wanted to see if that's useful or not. I'm not that happy with it. If you want to let it auto focus normally you would half press the release button but instead you need to press the AE lock (*) button, half pressing the release button doesn't have any effect. The live view auto focus is also very slow, takes about 5 seconds on average. It's also has a habit of not focusing it all, or going in the wrong direction to find the focus. I sometimes suddenly feel the focus ring vibrate on the 50 f/1.4 USM while it normally never moves. There is also some weird clicking noise on my zoom lens, it appears to keep trying to send it beyond the smallest focus range while the object is meters away. There is also a "quick" focus mode, so that if you press the AE lock button the mirror goes down, temporary turning live view off, doing the focus the normal way, and then the mirror goes up again. In that mode the auto focus takes about 2 seconds instead. The only thing live view seems useful for is that you can zoom in to see if something is in focus. I'm also seeing some weird behavior when using the flash. When using the 50mm f/1.4 it really seems to prefer using f/2.8, sometimes f/1.4 or f/4.0, and you can't select anything else in the auto, CA or P mode. Sometimes the depth of field is just too small, so I tried to set it to Av mode and manually select the one I want. What I didn't expect in that case is that it's calculating the shutter time like it won't be using the flash, and you suddenly get shutter times of 5 seconds. Other modes have similar problems, only setting everything manual seems to work properly.

I'm a notoriously late adopter of new technologies. Sure, if I think that something is potentially a good idea, I'll go for it, but for the most part, I assume that things are stupid. I refused to use the WWW for many years, thinking it was idiotic. Now I use the WWW. I find microblogging to be inane and narcissistic at best, but I have derived mild amusement from Shaq on Twitter. Microsoft Windows is something I thought would be ridiculously unsuccessful, because it didn't do anything I would find useful. The list goes on and on, and includes something known in the popular vernacular as configuration management . I've been hearing about BladeLogic and Opsware for quite some time, and idly wondering who the incompetent fools are that need something like this. Back in my day we could manage 100 servers with relative ease; no centralized authentication (because it was a running argument about whether NIS sucked more than NIS+ or vice-versa), no cfengine, no prebuilt server images or any of that jazz. If, for some reason, we needed to make the same change to all servers at once, or some subset thereof, we would just script it. So when one of my acquaintances started raving about Puppet, I assumed it was equally pointless. We'll call him Downs. Downs exhibits a pattern of behavior where he will rave about something for a few weeks, sometimes without having even tried it first, and then eventually become disillusioned with it and start ranting with the same fervor with which he had insisted that whatever fad of the week was the greatest thing ever. It came as no surprise to me when he fell out of love with Puppet and started raving about how much better Chef is. In case you're wondering, he still has not tried Chef to this day. Finally someone convinced me of the value of Puppet. I do not think it is the Way and the Light, but I do see a few situations wherein it makes sense to sacrifice efficiency, flexibility, sanity, and system resources to gain a way of building a near-identical machine from scratch. Now, beyond the flaws inherent to a solution like this, Puppet does have some annoying flaws which I find unnecessary, and which Downs ranted about back when I had absolutely no idea what he was talking about. I disregarded his alleged preference for Chef, but I have heard intelligent people extoll the virtues of Chef, and to a lesser extent Bcfg2. So I took a very brief look at each of those, and was pretty much horrified by what I found. If I need something Puppet-like in the very near future, I will probably be choosing Puppet. I have no intention of running Puppet on machines I manage all by myself though. For that eventuality I am NIHing something with what I consider a better design. The current code is published, but it does not do very much, and requires linking with libdpkg.a which does not exist in any package (but Guillem's working on that, I think). I have no idea whether or not I'll have the motivation to finish it all by myself, but it is Free Software. This post is intentionally weak on details, or deets as the kids call them.

RInside, a set of convenience classes to facilitate embedding of R inside of C++ applications, was updated to release 0.2.0 over the weekend. Sources as well as binary packages are at CRAN; sources are also at my RInside page The biggest news is that we now support builds on Windows -- if and only if you use the R toolchain -- more on that below. Since I had first released RInside people had emailed about support on Windows. However, Richard Holbrey deserves special mention as he actually sat down and tried to build it. Over a few email exchanges we made decent progress, but stalled. More recently, I sat down and had another go and lo and behold, it works. This required a few #ifdef statements here and there, as well as a function setenv() which Windows does not have -- Richard kindly helped here and borrowed most of the code from R itself. Now, to make this plain: you need gcc (as this is how R itself is built). This gcc compiler is the native compiler on Linux and OS X. But on Windows this requires installing the Rtools compiled by Duncan Murdoch as detailed in the The Windows Toolset appendix to the R Installation manual. Do not, I repeat, do not report an error if you fail to build this with another compiler as this is not supported. That said, if you have the proper tools, things are peachy. Install the package, change into the examples directory installed with it and say make (or on Windows: make -f Makefile.win). That's it -- now run the examples (which on Windows will most likely require setting the environment variable R_HOME). Now you can run simple examples like the one from the previous blog post or any of the other ones. Happy embedding!

The WebKit hackfest is now over, and I think it was a very productive week. Thank you very much to all who attended, to Igalia for organizing the hackfest, and hosting us so well, to Collabora for having sponsored the event, and allowed me to spend the week working on it, and to the GNOME foundation for having payed all of my costs!
Xan blogged about day 0, and also a summary of all that was done, so I ll focus on the stuff he forgot to mention ;D. The hackfest, for me, started on day -1 with me not allowing Xan to go sleep before he had reviewed a couple patches of mine to fix DOM context menu handling. It always bothered me that Epiphany failed to open right-click menus in some pages, or let pages handle the right click. Well, this is fixed now, and Zimbra users can now have their right click menus, and WoW players can remove talent points from their calculators =P. It turns out that many of the attendees don t like pages messing with their context menus, though, and they had some good points to back up their positions (like pages making it hard to save images, for instance), so I implemented a way to force openning the custom menu: Ctrl-rightclick. We wanted to use a GtkInfoBar to display questions regarding the form saving - our initial implementation always saved all credentials, but that didn t sound good enough. Xan and I thought it would be very complicated to make this work, because there were assumptions in the code regarding which widget contains which, but it turned out to be quite trivial - making EphyEmbed a descendant of GtkVBox instead of GtkScrolledWindow, fixing a small number of assumptions, and that was it. The passwords are saved in the GNOME Keyring. It s interesting to point out that GNOME Keyring seems to be unhappy with the number of passwords a browser stores - Xan s daemon was hanging, crashing, and spawning a large number of threads. My daemon decided to take up some 300MB of RAM at one point. It s somewhat funny to see how much a browser pushes the limits of our platform. We are hoping this will improve with the new keyring APIs, and the rewrite that is ongoing. It s nice to see my browser form passwords in seahorse, though, and be able to manage them like any other. One more thing worth of notice, although this post is already a bit too big: one of the main concerns people had during the Hackfest was on making build time smaller. Touching a single file in WebCore causes a debug build of 10 minutes on my laptop. Evan Martin and Benjamin Otte made a push at removing unnecessary includes from WebCore, and WebKitGTK+ files, which brough the build time down a bit. They end up inspiring Aroben, from Apple, to go even further into this, and remove many includes from files all over WebKit. Evan was also able to bring linking time down by making it possible to link libwebkit without having to build all the intermediate libraries, which brought build time down to 1 minute, when touching a single file in WebCore. Behdad and I also started looking into breaking WebCore up into lots of shared libraries for Debug builds, since we don t care too much about speed penalties in those. None of these experiments got committed yet, but I am hoping we will be having a better time hacking on WebKitGTK+ in the near future. It was awesome meeting everyone, by the way! See you around =).

I usally find blog rants useless, but sometimes something is just so annoying one is sufficiently inspired. Today I went with my parents to buy them a Tivo at Harvey Norman, Norwest, Castle Hill, NSW, Australia. I am a big Tivo fan; the interface is good and it "just works". I don't mind paying for (or in this recommending paying for) good products. After selecting the Tivo model, I asked for a HDMI cable. The salesman made a series of questions about what sort of HD TV it was being plugged into; I quickly sensed this as a probe to see what sort of suckers we were, and requested just a "normal" cable. At this point, he insisted on a $130 (you guessed it) Monster cable, and had the audacity to say that we didn't need one of the really expensive cables because our TV wasn't good enough! I openly expressed my concern, but the annoying high-pressure sales pitch had just begun. The amount of, frankly, crap that he spewed about 4-bit this, 10-bit that, legislating of labels, DA signal levels, mythical customers who regretted buying the cheap cables and who knows what else was to the point of being comical if it weren't so insistent and said with such seeming authority. There is only one thing that matters - if the cable has passed the functional requirements for being certified to have the distinctive HDMI logo plastered on it. From the HDMI FAQ:

Q. What testing is required? Prior to mass producing or distributing any Licensed Product or component that claims compliance with the HDMI Specification (or allowing someone else to do such activities), each Adopter must test a representative sample for HDMI compliance. First, the Adopter must self test as specified in the then-current HDMI Compliance Test Specification. The HDMI Compliance Test Specification provides a suite of testing procedures, and establishes certain minimum requirements specifying how each HDMI Adopter should test Licensed Products for conformance to the HDMI Specification.

Now, I can understand that if you buy any old HDMI cable off Ebay for $1, it may be a knock-off that uses the HDMI logo illegally. But there is no way that the certified $50 Philips cable (still very over-priced, but at least not insane, and discounted to $35) performs any differently to some overpriced Monster model certified to exactly the same standard. The thing that annoyed me most was his analogy to buying a tyre. He stated that "if you walked up to a tyre salesman and I said don't want the Pirelli's, just put the cheap-o tires on my Ferrari" I'd be insane, and thus by extension of that logic I was insane for not buying a Monster cable for my great new Tivo. This analogy is completely flawed and really just dishonest. A Ferrari is much more powerful and goes much faster than a standard car. It is plausible it needs a better engineered tyre to perform adequately given the additional stresses it undergoes. A Tivo doesn't put out any more or any less bits than any other HDMI certified equipment, no matter what you do. If the cable is certified as getting all the bits to the other end under whatever environmental conditions specified by the HDMI people, then it's going to work for the 99% of people with normal requirements. Nobody wants to make a significant investment in a piece of audio-visual equipment and feel they are getting something that isn't optimal. Harvey Norman's use of this understandable consumer sentiment to sell ridiculously over-priced cables that do nothing is extremely disappointing. I'm sure the commissions on these things encourage this behaviour, so it is useless expecting the retailer or individual sales assistant to change their policy to reccommend reasonably priced cables. However, it is really Tivo and other manufactuers who get the raw end of this deal; a $130 cable is over 20% of the price the actual Tivo! That is surely affecting people's purchasing decisions. If Tivo and others included a certified HDMI cable with their device, as they do with component cables, and had "Certified HDMI 1.3 cable included" plastered on the box, it would be a harder sell to explain why the manufactuer would bother shipping a certified cable that is supposedly insufficient, and consumers would hopefully avoid the very distasteful high-pressure theatrics I was subjected to today.

I was trying to dump a large Logical Volume (LV) over ethernet from one machine to another. I found some behavior which surprised me.fun constraints

• I have only a fairly minimal debian installation on each machine (which fortunately includes netcat-traditional)
• The two machines are connected directly by a single (gigabit) ethernet cable, with no other network connection. So no pulling in extra packages.
• I have serial console access to both machines, but no physical access.
• The LV being transfered is 973GB in size according to lvs (fairly large, that is), and contains a LUKS volume, which itself contains a basically-full filesystem -- transferring just the "used" bytes is not going to save space/time.
• I want to be able to check on how the transfer is doing while it's happening.
• I want the LV to show up as an LV on the target system, and don't have tons of extra room on the target to play around with (so no dumping it to the filesystem as a disk image first).

(how do i get myself into these messes?)This entry has been truncated read the full entry.Tags: lvm, netcat

Last week I was given a development board for a system on chip (SoC) we were looking at using. All i had to do was ensure I could get a reasonable, supportable, bootloader and Linux kernel running on it.

Now I am a fairly competent engineer, I have worked with the ARM processors and SoC for many years. It should not therefor take me four days of almost continuous frustration to get a bootloader and kernel onto a system, but it has.

Firstly the system arrives and we look it over, the hardware design is not exactly elegant but seems like it will get the job done. That is at least a step up from several previous offerings which have been just plain broken. So I power it up and it boots into Fedora core 8, possibly not my first OS choice for a diskless system, but it does work.

Perhaps its worth mentioning that my experience with doing any Open Source development has caused me to have a severe aversion to software which is not upstream, especially kernels. The effort required to run your own patch series and keep it up to date is nothing short of Herculean.

Let us be honest, how many developers do you know who are talented enough to generate good code and patient enough to separately maintain it over a long period of time? I shall not belabor this point as others have repeatedly made it for me in much more eloquent ways. Lets us just say from my point of view. Upstream - good. Long term forks - bad.

With this in mind, the solutions Simtec develop are always with a view to our customers being able to simply download and use the standard upstream software wherever possible. Here is where the story stops being smooth and becomes a rant.

The system was running 2.6.22, one that was patched all which ways. "maybe they just shipped an old kernel" I thought. I opened the CD they provided and...oh, OK its just zips and tarballs of the binaries and source they used (actually that in itself is a minor miracle with a lot of vendors right now - see the almost continuous stream of copyright infringement litigation from the FSF) but sill stuck back in July 2007.

So i did teh normal thing, entered the URL printed on the outside of the box and went looking for the updates. No dice, just the same old stuff. So I contacted the company and recived the news that no there were no updates, that was what was published for the device, thats what I could have.

My initial reaction was of tired resignation...this would not be my first SoC port (probably around the fifth or sixth!) So I responded with cd linux-2.6;git checkout master;git pull and went to see what I could see. and lo and behold, there in the kernel, was full support for not only the SoC but for the other version of the very board I had.

Fortunately after mentioning this on IRC my giraffe loving friend. pointed me at the correct community git tree to find the tiny patch required to support my board.

So the vendor was not only shipping old code but actively dissuading customers from even looking for new stuff, which it turns out was mainline anyway! To add to this somewhere along the lines the company name on the box and contact information has changed and...well confusion reigned.

Right Plain sailing from here on? not quite...that would have been too easy. So you can build a mainline kernel...but you cannot boot it. you guessed it the bootloader they ship on the board is ancient, specifically a u-boot fork from December 2005. Having finally figured out how this was going to go, I went to the uboot repo, pulled the latest and built it for my target. And then I needed to upgrade the u-boot and joy of joys bricked the system.

Fortunately the system has a integrated FDTI serial/JTAG port so re-writing the flash should be as simple as running openOCD configured correctly. Of course this did not work...in the end I resorted to running their binary built versions from the CD (tarfile inside a zip inside a tarfile...whatever) and wrote their original image back to the board...which worked.

Long story short, next day I discover that i need a magic different uboot target (make u-boot.kwb for those that care) write this with the binary openocd build and...success
From then on its all been a bit anticlimactic and straightforward and I now have a system capable of starting a kernel over the network and a root fs on network, disc or flash.

My main complaint from all of this is what the hell are vendors doing shipping crap like this? Why do they all insist on taking a dreadful code drop from the SoC manufacturer when they first brought the part up and shipping that prototype junk until the part reaches the end of its life?!?

All the usual excuses about support etc. will come out, but seriously, you recive *no* support from vendors unless you pay for it and even then they will simply tell you whats on the CD is "it". Btw the diff for u-boot to support this board is larger than the original sources and the 2.6.22 is attacked with a 65MB diff which cannot be good.

The Open Source option is continuously innovating and improving, producing ever better software...but the embedded world seems to be forever stuck shipping crap. Is this just me? Does anyone have any solutions? Or am I stuck with this until I change profession?

This is a question that Project Honey Pot's report answers.

They have released a report of the analysis of not only spam messages, but also of the harvesters behind spam.
Available with a simple overview the complete report provides some rather interesting details.

Do spammers take holidays?
The volume of spam drops approximately 21% on Christmas Day and 32% on New Year's Day. Saturday is the quietest day of the week.

And for those sending mails to public mailing lists, the BTS, etc:

How much harm does a harvester visiting my site do?
You can expect to receive 869 spam email messages every time a harvester picks up your email address from a website online.

The interesting bits are really worth the five-ten minutes it takes to read the complete report, enjoy it (and yay, it is CC-BY).

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

• [1] http://www.ted.com/talks/lang/eng/dan_gilbert_researches_happiness.html
• [2] http://zgp.org/~dmarti/freedom/property-rights/
• [3] http://www.ted.com/talks/rachel_pike_the_science_behind_a_climate_headline.html
• [4] http://www.good.is/post/a-happy-writer-is-a-lousy-writer/
• [5] http://www.cerias.purdue.edu/site/blog/post/firefox_vulnerabilities_souvenirs_of_windows_95/
• [6] http://www.guardian.co.uk/technology/2009/nov/26/digital-economy-file-sharing-mandelson
• [7] http://www.antipope.org/charlie/blog-static/2009/11/the_myth_of_the_starship.html
• [8] http://blogs.wsj.com/developments/2009/10/30/its-ok-to-walk-away-a-law-professor-argues/
• [9] http://mako.cc/copyrighteous/20091201-00
• [10] http://www.practicalethicsnews.com/practicalethics/2009/12/diluted-evidence-is-there-anything-special-with-homeopathy.html
• [11] http://www.publishersweekly.com/article/CA6709919.html
• [12] http://www.ted.com/talks/thulasiraj_ravilla_how_low_cost_eye_care_can_be_world_class.html
• [13] http://www.ted.com/talks/scott_kim_takes_apart_the_art_of_puzzles.html
• [14] http://www.ted.com/talks/gordon_brown_on_global_ethic_vs_national_interest.html
• [15] http://blogs.broughturner.com/2009/10/is-att-wireless-data-congestion-selfinflicted.html
• [16] http://code.google.com/speed/public-dns/

Signal boost. While returning from a trip to the US (in other words, going to Canada) on December 8th, Canadian SF author Peter Watts was attacked, beaten, pepper-sprayed, and imprisoned without access to legal counsel by the US Border Patrol. See also his post on the subject. The government's case has obviously not yet been presented. However, given the past behavior of the US Department of Homeland Security, I find Watts's description of the event entirely credible. This is an outrage. This is the kind of thing that happens in third-world dictatorships. See also Making Light and Boing Boing. John Scalzi's post has information on how to donate money to Watts to get through this legal fight. I will be giving him money. I also plan on calling my US Representative and Senators when I'm no longer at work, as this sort of treatment of a Canadian citizen warrants a Congressional investigation and heads rolling, not only at the agent level but at the Homeland Security management level. Even apart from basic questions of human rights, Homeland Security is going to systematically destroy the tourist industry in the United States if this sort of behavior is not stomped on immediately and severely. There are border patrol guards who are probably still on duty today who are common criminals and belong in prison for assault. ETA: The government's side of the story. This is pretty much what I expected, and with the data available so far, I feel confirmed in my initial reaction. This level of force is completely disproportional and flatly unacceptable short of Watts pulling a weapon on the agents, which they are not claiming that he did and which therefore he clearly didn't do. I'm highly skeptical of this supposed "choking" of a police officer after they pepper-sprayed him. This reads like the standard cover-your-ass charges filed by the police after they beat someone up. Once they've provoked the struggle by grabbing you and have done something as painful and disorienting as pepper-spraying you, chances are you're going to physically react in ways that they can conveniently misinterpret and use as the basis for charges. People elsewhere have pointed out that contacting Congressional representatives is probably better done Monday morning since nothing will happen during the weekend.

Over Thanksgiving this year, we took a train trip to visit Terah s family in Indiana. Here s the story of the trip. The Train to Indiana Our last train trip (to Chicago) was in May, before Oliver was born. Jacob always has enjoyed the train, but this time wow. We told Jacob about the trip the day before we were to leave. Several times in the day, he d get a sly smile, and ask, What are we going to do? Terah or I would spell out the day s activities, and when we d get close to the part where I d wake him up at night and we d go to the train station, he d have a big smile. The big hour (2AM) arrived and Jacob woke up while Terah was putting his shoes on. I was there a moment later. Normally Jacob is terribly grumpy after waking up, even more so if somebody else woke him up. That day, I saw a very groggy-looking boy. I asked him, Jacob, are you ready to go to the train station? He answered, YEAH! while rubbing his eyes. Dad, let s go! We had Amtrak s family bedroom for the first time this trip, and it worked out great. Jacob actually slept, despite his extreme excitement. He d been talking about eating in the diner car for quite some time, and was really excited when breakfast time rolled around. We sat down, and he pressed his nose up against the window right away. He counted the freight trains the he saw, pointed out train tracks and crossing guard lights , noticed barns and trucks, and gave us frequent updates on how fast we were going. His voice was high-pitched from excitement, and sometimes we d have a running commentary we could barely interrupt, and other times he d sit there silently just soaking it all in. But he didn t eat. He didn t want to turn his head away from the window for a second to take a bite. I kept reminding him to take a bite, and finally, fearing he d be really hungry as soon as we went back to our room, fed him a few bites for the first time in ages. I don t believe he realized that happened. He just opened his mouth by reflex and once I had to tell him to close his mouth around the fork that was in it. It was much the same story for lunch, and for dinner on the train though less so for dinner since it was dark outside. At dinner, a man sitting across the aisle from us said, If you weren t getting your meal free [since we're in the sleeper], I d buy it for you. Your boys are amazingly well-behaved! I thanked him politely Jacob and Oliver both were doing well but didn t really think it was that unusual. Oliver and Jacob were both big hits on the train. They got smiles from so many people on board and in the stations. Several of the dining car staff seemed to linger at our table longer than elsewhere, looking at them (and especially Oliver). Oliver slept well on the train: And here are a couple of typical Jacob photos: In Indiana I ve got to start this out with one of the family gatherings at Terah s aunt s house. They have a large kids area in their basement, and a swingset with a slide outside. Jacob loved all of this, and spent hours playing with me. Sometimes if I d go upstairs with the adults, I d hear Jacob s voice from the basement a few minutes later: DAAAAAD!!!! Shall you come back downstairs? DAAAD! And, of course, it s hard to resist a 3-year-old that wants to play with me, so I inevitably would. We played air hockey Jacob shrieked with delight whenever anybody scored a goal in any way. At one point, he stood up, looked around, and said, Hmmm. Where are all the trains? Apparently he expected every home to be stocked with toy trains, and this one wasn t. A few seconds later, he was all excited. He ran over to a toy semi, and said, I will use the semi train! Outside the short slide had a ladder going up to it. I held Jacob s hands while he climbed up the first few times, but I knew he could do it himself. I started helping him less and less, and eventually refused to help him at all. He was rather frustrated with that for a moment, but a couple times later, climbed up and said, Dad, I got up here all by myself! A few times later, it looked like he was having trouble, so I tried to help, and got shooed away with, No, dad! I can do it. The best moment of the weekend came during the drive back to Terah s mom s house where we were staying. Terah was driving, and I heard Jacob say, Shall you come closer? Not sure how to do that from the front seat of the car, I reached my hand back there and he held it. He held one or the other of my hands for 20 minutes until he fell asleep in the car. It s funny what feeling needed by a 3-year-old makes a person feel, but that was surely a happy day. Jacob got to see all of his Indiana grandparents on this trip, and loved playing with them. He especially liked using a perfect red train (his words!) he drug home from the nursery at church on Sunday, and a child-sized tent in the basement. I m not sure how many times I heard Gampa Mike! Shall we go to the basement? that weekend. One point he wanted me to come down to see it and play with him, so I did. Then he decided we all needed to be down there, and told me I ll go pick up mom. I ll be right back. Stay downstairs, dad. Fearing that Terah wouldn t know what to do when Jacob attempted to pick her up , I tried to discreetly follow at a distance. That was no match for a 3-year-old. He looked back at me from the top of the stairs, pointed an accusing finger at me, lowered his voice, and said, Dad. Stay. Terah and I spent a day in South Bend and stayed at the Oliver Inn while Jacob and Oliver got some time to themselves with their grandparents. We had dinner at the beautiful Tippecanoe Place, just next door. The Trip Home We had a few hours in downtown Chicago on our way home. We went and ate at a small Indian restaurant. Jacob kept alternating between three things: This food is very spicy! I like spicy food. I need another drink! A little while later, Terah decided we should have some dessert. So she and Oliver stayed in the waiting area while Jacob and I went asporing (exploring) to find something. We walked a block or two to the nearest Dunkin Donuts, then to a Starbucks, and then back. Jacob carried the sack of donuts, guarding them so carefully that I was afraid he might smash them. (They were cream-filled so that wouldn t have been good.) He had a lot of fun exploring Chicago Union Station, and also enjoyed walking in Chicago (though not quite as much; it was all a little too loud for him.) In the train station, he loved the escalators. Up and down we went, whenever we wouldn t get in people s way. I had to eventually drag him away from them as I was getting a little self-conscious at all the people standing around watching us repeatedly go up and down the escalators. Jacob has found a 3-year-old way to express his feelings: he ll pretend to be a kitty, and say the kitty is sad or something like it. All day he had been saying the kitty is hungry. He hadn t had a big breakfast, but he had plenty of opportunity to eat at lunch, and a snack. We were a bit confused, but I think we finally figured it out on the train back to Kansas. We didn t have time to eat breakfast in the diner on the train from South Bend to Chicago, so Jacob missed out on the dining car then, and I think he was disappointed or confused about that. Once we had dinner in the dining car, he was happier. Since we ve been back home, he s been talking about our next train trip. He says which suitcases we ll take, that we ll eat in the diner, that it will be a night train we take, and generally is still very excited about it. It was a fun trip.

Lately I have been thinking about methods to provide a stripped down, secured environment for running untrusted code on GNU/Linux. With this post I would like to present you with the first results of my research.

ujail - brief introduction

I have chosen ujail as the name for the technique I am proposing. ujail stands for micro jail in userspace and, in itself, describes the concept briefly. The main idea is to have a userspace process monitor system calls of one of its childs and emulate some calls, if needed. This is done using ptrace and namely both PTRACE_SYSEMU and PTRACE_SYSCALL.
The ujail process should not be able to monitor syscalls, like strace does, but also intercept and emulate them.

This sounds a lot like user mode linux (uml), but the method is different. Whilst uml comes with a complete kernel, emulates all system calls and this way provides a virtualized system, ujail is intended to only emulate some systemcalls, without emulating the kernel.

Revisiting PTRACE_SYSCALL & PTRACE_SYSEMU

To better explain how the ujail technique works I would like to have a quick look at PTRACE_SYSCALL and PTRACE_SYSEMU again.

PTRACE_SYSCALL allows a userspace process to be notified whenever a traced process enters or leaves a system call. This means that two notifications are normally sent: one before system call entry and one afterwards. Even though one is able to change the parameters of system calls this method does not allow system calls to be fully emulated (think virtual filesystem here).

PTRACE_SYSEMU on the other hand provides one notification on syscall entry and expects the receiver of the notification to emulate the syscall. This method alone sounds great, but this also means that memory allocation needs to be emulated too, which is quite complex in userspace.

A hybrid of PTRACE_SYSCALL & PTRACE_SYSEMU

Now on to the concept behind ujail. The method I am describing works by calling PTRACE_SYSEMU for a specific process and this way taking over emulation of all system calls. However, some system calls are complex to emulate in userspace, and so a hybrid of both PTRACE_SYSEMU and PTRACE_SYSCALL is needed. In short this works by checking whether the syscall needs to be emulated when the PTRACE_SYSEMU event is received.
Now one way is emulating the syscall, filling the processes' registers and resuming execution of the process. This is simple and straight-forward.

The second way is forwarding the system call to the kernel. The problem here is that calling the syscall in the monitoring process will make the new resources available to that very process, and not the process to be jailed. This is where the hybrid method kicks in.

The proof of concept code creates a backup of the next instruction to be executed along with a copy of the instruction pointer at this point and patches it with the opcodes for "int $0x80", causing the syscall to be made again. After that it resumes execution with PTRACE_SYSCALL and waits again. The first event to be received now is the program leaving the emulated system call, which can be ignored. Resuming yet again will give use two PTRACE_SYSCALL events, one for syscall entry and one for syscall exit.

The first event is not really interesting, but at the second event the opcode backup is restored and the eip set from the saved value. Now the kernel has handled the syscall and the result is ready for the child process. A final call of PTRACE_SYSEMU resumes execution of the child and waits for the next syscall.

Proof of concept

The proof of concept code can be downloaded from its bazaar branch at launchpad.net. It is intended to be used on i386 systems only and works with simple programs, but is known not to work with anything using fork, vfork and most likely will not work for binaries using threading.

Finally, I would like to thank Pradeep Padala for his "Playing with ptrace" articles [0][1], which were fun to read and worked as a great introduction of ptrace for me.

Now there is only one thing left to say: if you are interested in this method, see loopholes or problems or want to contribute, please go ahead and contact me:

debian at sp dot or dot at

Recently I've been spidering the internet, merrily downloading content for the past few days. The intention behind the spidering is to record, in a database, the following pieces of information for each image it stumbles across:

• The page that contained the link to this image. (i.e. the image parent)
• The image URL.
• The MD5sum of the image itself.
• The dimensions of the image.

I was motivated by seeing an image upon a website and thinking "Hang on I've seen that before - but where?". Thus far I've got details of about 30,000 images and I can now find duplicates or answer the question "Does this image appear on the internet and if so where?". Obviously this is going to be foiled trivially via rotations, cropping, or even resizing. But I'm going to let the spider run for the next few days at least to see what interesting things the data can be used for. In other news I'm a little behind schedule but I'm going to be moving from Xen to KVM over the next week or ten days. My current plan is to setup the new host on Monday, move myself there that same day. Once that's been demonstrated to work I can move the other users over one by one, probably one a day. That will allow a little bit of freedom for people to choose their downtime window, and will ensure that its not an all-or-nothing thing. The new management system is pretty good, but I have the advantage here in that I've worked upon about four systems for driving KVM hosting. The system allows people to enable/disable VNC access, use the serial console, and either use one of a number of pre-cooked kernels or upload their own. (Hmmm security you say?) ObFilm: Chasing Amy

To count means that you/it/whatever counts matters. If one counts the number of females in many tech/media conference, the number of women is dreadfully low, giving off the meaning and feeling they don t always count, even if they are very well received. There is a new project spearheaded by the efforts of Annina R st that will help us count women at conferences. The project is cleverly called Be Counted and it allows you to input information about gender representation in conferences. Here is a little more about the project and I urge you to check it out and contribute:

The project aims to collect a stream of user-contributed data on gender diversity in technology environments in the form of Gender Ratio Reports (GRRs). The longterm aim of the project is to not just collect but also provide tools for retrieving and visualizing the data in order to encourage others to collectively analyze the patterns behind the numbers.

To count means that you/it/whatever counts matters. If one counts the number of females in many tech/media conference, the number of women is dreadfully low, giving off the meaning and feeling they don t always count, even if they are very well received. There is a new project spearheaded by the efforts of Annina R st that will help us count women at conferences. The project is cleverly called Be Counted and it allows you to input information about gender representation in conferences. Here is a little more about the project and I urge you to check it out and contribute:

The project aims to collect a stream of user-contributed data on gender diversity in technology environments in the form of Gender Ratio Reports (GRRs). The longterm aim of the project is to not just collect but also provide tools for retrieving and visualizing the data in order to encourage others to collectively analyze the patterns behind the numbers.

As an aside to yesterday s Another Network Begins note, I should mention that one of the challenges of running any inter-business project is paying for it. It surprises some people that many great cooperative projects are running on very small budgets, whereas it looks like some Social Enterprise groups are awash with cash. How can this be, when it still looks like cooperatives are stronger financially than social enterprises? A written answer in Parliament on Monday suggests one possible reason: Co-operatives UK and The Plunkett Foundation together get 164k of Cabinet Office funding, while the Social Enterprise Coalition alone gets 488k and there are other SocEnt groups on the pay list. Once again, I m surprised by the way the Labour and Co-operative government is behaving, like I was on the way the real help initiatives exclude cooperative partnerships, while being open to both private sector employers and social enterprises. I m all in favour of intervening in markets, but this doesn t feel like it s making things fairer. Three strikes, funding social enterprise in preference to cooperatives would a Conservative or Liberal Democrat government really be worse?

It is our pleasure to announce that Giuseppe Iuculano is now a full member of the Security Team.

Also, Raphael Geissert has been added as an assistant to the Security Team.

Moritz M hlenhoff, on behalf of the Security Team

Reality needs fantasy to render it desirable, just as fantasy needs reality to make it believable. Stephen Duncombe

This fall I have been awash in a few obsessions including book piracy and spam. I recently got to talk about one of these obsessions when I was interviewed about book piracy by Nora Young for her weekly CBC radio and podcast show Spark. I mostly gave a lay of the land panorama with a nod toward some of the conditions, technological and social, that can help us grasp the contemporary explosion of book piracy and also raised some thoughts about what might change the future landscape. What I don t raise is whether a politics built around an explicit embrace of piracy is regressive, progressive, or something else but these ethical questions were posed in the comments left for the full interview. Some of the comments pointed to the pitfalls and shortcoming that can follow the terminology of piracy many of which I share. But what keeps me interested in the politics of piracy is how it can speak the language of spectacle, which can be a powerful tactic and technique for broadcasting a political message. Here I just paraphrasing and cribbing the work of Stephen Duncombe, who has argued, I think quite persuasively, that we cannot rely solely on reasoned debate for building political programs. Duncombe does not argue that we must toss out rationality and truth seeking (these are absolutely necessary) but notes how on their own or if not clothed in some other cloak, they may not be enough to convey and compel, especially in this day of total media saturation. Or to put a but more poetically by him Reality needs fantasy to render it desirable, just as fantasy needs reality to make it believable. Much (though not all) of contemporary digital piracy follows the logic of spectacle. It builds and conveys a fantastical drama of right and wrong, of new possibilities, of freedom from the noose of the law; it signals and speaks to the thrill and fun in twisting, even breaking, existing structures and constraints; and provides a window into another way of acting/behaving. In many cases what it provides is a commons (and I will be exploring it in depth in my class next semester on the commons) and many folks, I imagine, turn to piracy simply for the free stuff, and a number of them come out of the other side transformed into copy fighters willing to engage in a politics beyond sharing stuff and waving the pirate flag. For those of us who believe in greater access and different ways of imagining structures and strategies of re-compensation, piracy on its own is not certainly enough and I understand fully and even to some degree, share the skepticism many feel toward such language. But I am not quite ready to declare a politics of piracy as always politically bankrupt or necessarily backward. I guess what I embrace is a diverse political ecology. For some, the drama of spectacle and thrill of transgression are what turns their political mojo on; for others it is the cool and reasoned debate common to policy and reform; for others, they want to focus on building alternatives as we see with Free Software or radical tech collectives. For some, it is both the reasoned salt and the transgressive pepper that spices their political world. And I would rather have more spice than less, especially in an era where the blandness of political apathy is that which is our most dangerous enemy. Related Links: Here is a wonderful animation by the NZ Book Council that captures what I love about books and renders its materiality wonderfully alive. On the Media has a episode on book publishing and Cory Doctorow has penned some thoughts about the future of book selling. If you want to keep abreast on the politics of liberating books, check out Free our Books. If you are more interested in the technical side of things, check out the book liberator project.

Reality needs fantasy to render it desirable, just as fantasy needs reality to make it believable. Stephen Duncombe

This fall I have been awash in a few obsessions including book piracy and spam. I recently got to talk about one of these obsessions when I was interviewed about book piracy by Nora Young for her weekly CBC radio and podcast show Spark. I mostly gave a lay of the land panorama with a nod toward some of the conditions, technological and social, that can help us grasp the contemporary explosion of book piracy and also raised some thoughts about what might change the future landscape. What I don t raise is whether a politics built around an explicit embrace of piracy is regressive, progressive, or something else but these ethical questions were posed in the comments left for the full interview. Some of the comments pointed to the pitfalls and shortcoming that can follow the terminology of piracy many of which I share. But what keeps me interested in the politics of piracy is how it can speak the language of spectacle, which can be a powerful tactic and technique for broadcasting a political message. Here I just paraphrasing and cribbing the work of Stephen Duncombe, who has argued, I think quite persuasively, that we cannot rely solely on reasoned debate for building political programs. Duncombe does not argue that we must toss out rationality and truth seeking (these are absolutely necessary) but notes how on their own or if not clothed in some other cloak, they may not be enough to convey and compel, especially in this day of total media saturation. Or to put a but more poetically by him Reality needs fantasy to render it desirable, just as fantasy needs reality to make it believable. Much (though not all) of contemporary digital piracy follows the logic of spectacle. It builds and conveys a fantastical drama of right and wrong, of new possibilities, of freedom from the noose of the law; it signals and speaks to the thrill and fun in twisting, even breaking, existing structures and constraints; and provides a window into another way of acting/behaving. In many cases what it provides is a commons (and I will be exploring it in depth in my class next semester on the commons) and many folks, I imagine, turn to piracy simply for the free stuff, and a number of them come out of the other side transformed into copy fighters willing to engage in a politics beyond sharing stuff and waving the pirate flag. For those of us who believe in greater access and different ways of imagining structures and strategies of re-compensation, piracy on its own is not certainly enough and I understand fully and even to some degree, share the skepticism many feel toward such language. But I am not quite ready to declare a politics of piracy as always politically bankrupt or necessarily backward. I guess what I embrace is a diverse political ecology. For some, the drama of spectacle and thrill of transgression are what turns their political mojo on; for others it is the cool and reasoned debate common to policy and reform; for others, they want to focus on building alternatives as we see with Free Software or radical tech collectives. For some, it is both the reasoned salt and the transgressive pepper that spices their political world. And I would rather have more spice than less, especially in an era where the blandness of political apathy is that which is our most dangerous enemy. Related Links: Here is a wonderful animation by the NZ Book Council that captures what I love about books and renders its materiality wonderfully alive. On the Media has a episode on book publishing and Cory Doctorow has penned some thoughts about the future of book selling. If you want to keep abreast on the politics of liberating books, check out Free our Books. If you are more interested in the technical side of things, check out the book liberator project.

I've been working with Charlie DeTar and the Center for Future Civic Media on a project called Between the Bars which is a blogging platform for prisoners. The current platform is essentially a snail-mail to web gateway: prisoners send letters which semi-automatically scanned and posted to the site; comments are printed and mailed back. As we plan the launch of the project, we are trying to talk to as many stakeholders as possible -- this includes ex-prisoners, families and friends of people who are or have been in prison, non-profits working with prisoners, victims of people in prison, people who work in prisons or in corrections, probation officers, or almost anyone else with a perspective or set of experience that might help us understand the difficult space our project is trying to negotiate and who might be able to help influence the design. At the moment, we're not talking to current prisoners due to rules regulating research involving prisoners, but almost anyone else would be someone we'd love to connect with. We're interested in hearing from people about their experience with prisons or prisoners, about staying connected to families and friends while behind bars, and, ultimately, about how we might design, deploy, and support technology to help folks out. If you or someone you know are stakeholders and would be willing to talk about your experience and opinions with prisons, communication, and technology, please drop us a line at betweenthebars@mit.edu.