[So much for at-least-one-post-a-week : I'm now back online, courtesy of a new router]. In the this week is another announcement from the CoRoT group: CoRoT-7 is the first star (outside our own) known to host two Super-Earths. Two planets of 1.6 and (max.) 21 times Earths' mass, and the smaller one at least is rocky (the outer planet has not been seen in transit, so we can't tell yet). Its not the first Earth-sized planets to be discovered: as my supervisor points out, <http:>PSR 1257+12 holds that record, as well as the one for the first exoplanet to be found. As usual, Exoplanet.eu gives the details: CoRoT-7b has a flurry of papers to its credit already. CoRoT-7b transits its star and is the first Super-Earth with a clear radius measurement. Together with a mass calculated from a radial-velocity measurement by the HARPS spectrometer, they calculate a density of 5.5 g/cm3, and so its shown to be a rocky world. CsoRoT is a hot A-class star, and CoRoT-7b orbits only 2.5 million km from it. Because of this, it is unlikely that it has retained an atmosphere (it'll be way too hot), and is expected to be tidally locked: with one face permanently pointing at the star, one in permanent darkness, the two sides will be both too hot and too cold for life. (Is there a Goldilocks, twilight zone? unlikely: the tidal forces will be too high, and with no atmosphere to moderate matters, there can be little hope for life). Still, while it may not support life, it could help answer important questions, such as how many terrestrial planets are there out there. Most of the planets discovered to date have been gas giants, which are easier to discover. But are there lots of rocky planets hiding between the giants ? One way of answering this might involve the disks from which the planets evolved. One trend we've seen is that (gas giant) planets are more likely to be found around stars with high metallicity (astronomers are strange in calling all elements heavier than helium a metal). But our Sun is strange in being comparatively depleted in 'refractory' elements such as iron or silicon. Is this important for the story of terrestrial planets? Melenendez et al. have a paper on arxiv.org on this topic. They find that most stars similar to our sun found not to have giant planets have elemental abundances similar to our Sun: perhaps they have undiscovered rocky planets instead ? Tags astronomy, exoplanets, planet-of-the-week

It s that time of the year again. Everything is changing, and maybe for the better. The days are getting shorter. When I left for work on my bicycle yesterday morning, it was still dark outside, and a little nippy. There s nothing quite like riding a bicycle down a deserted country road at night, a cool breeze at your back, and having the sun come up as you ride. And with the cooler weather, we can open our windows at night instead of running the air conditioner. It s also nice to have a pleasant cool breeze flowing through the house, and hear the frogs, crickets, owls, coyotes, and other wildlife at night. Out here, we certainly don t hear sounds of traffic, or loud car radios, though on a really clear night we might hear the rumble and whistle of a train from a few miles away. This is also sunflower season in Kansas. The wild sunflowers, when their smaller-than-most-people-think flowers, grow everywhere. Some ditches turn into a sea of person-height yellow. Sunflowers are on the sides of bridges, around people s mailboxes just about anywhere that isn t mowed or farmed. Then you also pass the sunflower fields, with their larger flowers, even more sea-like. The beans are getting tall in the fields this time of year, and it won t be long before the milo starts to turn its deep, dark reddish brown. But, you know, we re Kansans. We can t really let ourselves enjoy it all that much. Just today, I heard a conversation apparently the Farmer s Almanac is predicting a brutally frigid winter this year. Gotta keep our sense of pessimism about weather alive now

Hi Rich! Re hour+long unit tests I agree that you need a comprehensive test suite, and that it should test all the dark and hidden corners of your code base. But time is not free! A long test suite inhibits:

cycle time - the fastest you can release a hot fix to a customer

developer productivity - you can't forget about a patch till its passed the regression test suite

community involvement - if it takes an hour to run the test suite, an opportunistic developer that wanted to tweak something in your code will have walked away long ago Note that these points are orthogonal to whether developers edit-test cycle runs some or all tests, or whether you use a CI tool, or a test-commit tool, or some other workflow. All that said though, I'm extremely interested in *why* any given test suite takes hours: does it need to? What is it doing? Can you decrease the time by 90% and coverage by 2%? I got another response back, which talks about keeping the working set of tests @ about 5 minutes long and splitting the rest off (via declared metadata on each test) into 'run after commit or during CI'. This has merits for reducing the burden on a developer in their test-commit cycle, but as I claim above, I believe there is still an overhead from those other tests that are pending execution at some later time. From a LEAN perspective, the cycle time is very important. Another important thing is handoffs. Each time we hand over something (e.g. a code change that I *think* works because it passed my local tests), there is a cost. Handing over to a machine to do CI is just as expensive as handing to a colleague. Add that contributors sending in patches from the internet may not hang around to find out that their patch *fails* in your CI build, and you can see why I think CI tools are an adjunct to keeping a clean trunk, rather than a key tool. The key tool is to not commit regressions :) Oh, and I certainly accept that test suites should be comprehensive... I just don't accept that more time == more coverage, or that there isn't a trade off between comprehensive and timeliness.

Finally, temperatures are back to sanity, and the air is no longer a mass of dust. For us Mexicans (well, for us central-Mexicans), spring is the hot, dry season. In spring, we usually reach up to 30 C... Not too much for many people, but hellish for me. Summer is much more palatable, having decently sunny days, cloudy evenings and a nice shower as it gets dark. Even if it is a bit harder for a cyclist to take the streets, I think this is my favorite season. This year, we had several (and early!) false starts. I have come to hate April/May's heath waves (that means I am usually grumpy for my birthday). But finally, we have had almost a week of cloudy skies, with almost daily rains. That makes me happy. But I am even happier when I come to work to my office, look out the window, and am not greeted anymore with a 5Km visibility range, but with 20Km. Mind you, this photo was taken past 5PM, when the sky turns nice and gray (just as I've proclaimed my life to be). According to the Weather Channel's information on Mexico City, we currently have 16.1Km Which makes sense, as I can no longer see the Chiquihuite mountain marking the Northern part of the city, but I can perfectly see some buildings near Polanco. The view earlier in the day was... Beautiful, partly perhaps because of the contrast just with last week's. Oh, and yes - Once again, thank you very much my dear University. Instead of the closed cubicles many people spend their hours in, this relaxing view was taken from my office's window. I love it!

We got back from our brief visit to Canada last night. We had a great time, Vancouver and Victoria were both very nice. YVR (Vancouver Airport) This airport has to win the award for nicest airport I've seen. Clearly they pulled out all the stops in preparation for the Winter Olympics next year. The airport is very bright, airy and open. Before you even hit the immigration line, you pass through a couple of water features, some indigenous art, all of that jazz. Immigration is done in the open in a huge hall, instead of in the dark bowels of the airport, which was a pleasant change from the usual US airport immigration experience. (That said, Houston airport had a pretty nice immigration experience). Oh, and it was blanketed with free WiFi, which was a nice touch. Ferry to Victoria The brief geography lesson I got was that Victoria is on Vancouver Island, and Vancouver (the city) is not on Vancouver (the island). Go figure. Getting to Victoria from the airport via BC Ferries was a very seamless experience. You board a bus from the side of the airport terminal, and it drives to Tsawwassen, drives onto the ferry, and then you get off the bus and go sit on one of the passenger levels of the ferry. BC Ferries BC Ferries deserves special mention. The ferry itself is huge - 7 decks (three of them for vehicles). It's a veritable shopping mall on water. It's got some stores, buffets, exclusive $10 entry lounges, video arcades, cellular payphones, the works. The trip to Victoria The ferry ride took about 1.5 hours, and was very scenic. We passed by a few smaller islands, and I had an instant flashback to the TV show The Beachcombers that I used to watch as a kid, particularly the opening sequence. Victoria Victoria is a beautiful city. It's the capital of the province of British Columbia, and I likened it to Canberra in a lot of ways. It had more modern conveniences than a town with a population of its size would otherwise normally have. It was a government and university town. Nice houses, very green in general, and being an island, there was water everywhere. It seemed to have lots of bays, and harbours and rivers. It was very nice. Flight back to Vancouver We took a Harbour Air seaplane back to Vancouver on Sunday afternoon. That was fun. I've never been on a seaplane before. It took about 35 minutes. Vancouver First order of business in Vancouver was the purpose of the whole trip: renewing our visas. That went fine, I'll write a separate post about the process. Afterwards, we bought tickets for the Big Bus, and toured the city for the remainder of Monday and Tuesday. Drive to Whistler For Wednesday and Thursday, we hired a car, and went a bit further afield. On Wednesday, we drove up to Whistler to check it out. The drive up took about 5 hours, because we stopped at every scenic spot we came by. Waterfalls abounded. There was a lot of work being done on the roads between Vancouver and Whistler, upgrading them for the Olympics. Whistler itself was pretty spectacular. Some crazy looking ski runs. The village looked like Squaw Village on steroids. The drive back only took about 2 hours. Cleveland Dam, Lynn Canyon suspension bridge For the last day, we drove to Cleveland Dam, where there was a salmon hatchery, and checked out the dam, hatchery and surrounding area. It was lovely and green. After that, we drove to Lynn Canyon to check out the suspension bridge there (unlike the Capilano Suspension Bridge, which costs $26 a pop, the Lynn Canyon one is free). That was all we had time for really. The weather wasn't fantastic during our stay, but it didn't really prevent us from doing anything either. I'd have liked to have checked out Stanley Park if we had more time, and Sarah wanted to go to Grouse Mountain. We had a near-100% success rate at being picked as Australians, unlike when we're in the US, where more often than not, people ask us if we're British. Apparently this is mainly attributed to the fact that Australians run all of the ski lifts at Whistler. We certainly heard a lot of Australian accents while we were getting around. I'd also never seen such a saturation of Starbucks before. In Vancouver, there were literally Starbucks across the road from Starbucks, and around the corner from another one. Whistler village had two. The Canadian accent is cute. They really do tack "eh?" on the end of everything, although it's phonetically more like "ay?". "House" and "about" are also pronounced distinctively differently. Overall, Vancouver wasn't what I'd call bursting at the seams with tourist attractions, but seemed like a nice city. It was fairly flat. The beggars were well dressed. Everyone was very friendly. A lot of the taxis were Priuses. It seemed pretty clean. I could handle living there. Not sure how bad it gets in winter.

Now that I am (thankfully) done teaching until September, I have time to devour two small mountains of readings that I need to finish before I return to my manuscript, which I will be working on, I hope uber-productively, all summer long. One pile of readings deals with coding, open source, and the commons, such as Scott Rosenberg s Dreaming in Code and David Bollier s Viral Spiral. Another pile of reading edges toward the theoretical side of things, having to do with craft, pleasure, and humor, since it is pleasure in its many many many guises from from the calm feeling of self-satisfaction that underlies pride in one s craft, to the more sublime feeling of ecstatic bliss that powers many creative sprints. If one entertains pleasure, one must also entertain its darker side, for all of this feel good stuff is nonetheless often springs forth from a deep sea of passionate frustration. This seems to be the driving theme of Dreaming in Code and it is also what animates Ellen Ullman s fictional account of pure frustration, The Bug. I am quite fond of native expressions of geek frustration and recently was provided with an exquisite example a rant against the Adobe PSD format. The author of Xee, A light-weight, fast and convenient image viewer for Mac OS X explained his utter contempt for the Adobe PSD in the following way:

At this point, I d like to take a moment to speak to you about the Adobe PSD format. PSD is not a good format. PSD is not even a bad format. Calling it such would be an insult to other bad formats, such as PCX or JPEG. No, PSD is an abysmal format. Having worked on this code for several weeks now, my hate for PSD has grown to a raging fire that burns with the fierce passion of a million suns. If there are two different ways of doing something, PSD will do both, in different places. It will then make up three more ways no sane human would think of, and do those too. PSD makes inconsistency an art form. Why, for instance, did it suddenly decide that *these* particular chunks should be aligned to four bytes, and that this alignement should *not* be included in the size? Other chunks in other places are either unaligned, or aligned with the alignment included in the size. Here, though, it is not included. Were it within my power, I would gather every single copy of those specs, and launch them on a spaceship directly into the sun.

Even if this account represents unadulterated irritation, he leaves us, the reader, with nothing of the irritation, only pleasure. This aftermath of frustration is delivered through the vehicle of humor, which within the hacker context, is the cultural container that best captures the spirit of hacker pleasure or so I will be arguing. Like many humorous rants from the world of hacking (and please send me any others you know of), this text dances with liveliness; it exudes its own rhythm; it glistens to use Ronald Barthes apt phrasing from his short book The Pleasure of Text, which I just finished as part of my theoretical escape into the pleasure-dome. Although there are parts of his book which are to be frank, *really* *not* *pleasurable*, partly due to obscure references to High French Theory, which elide even an academic pair of eyes, the book generally pleases. And one of the most pleasing chunks is his definition of a stereotype:

The stereotype is the word repeated without any magic, any enthusiasm, as though it were natural, as though by some miracle this recurring words were adequate on each occasion for different reasons, as though to imitate could no longer be sensed as an imitation: an unconstrained word that claims consistency and is unaware of its own insistence

In contrast to the stereotype, a string of words that enchants does so by slipping off the page to hit you squarely in the heart or the gut. Unfortunately, while academic writing steers clear of stereotypes, often trying to present the detailed singularity of a phenomena (even when conditioned by social forces), it does not exactly glisten, though there are a handful of exceptions. I think we need more texts that glisten, even if only during sections or parts of our books and articles (much like the rant helped enliven the more staid technical document). In recent years, in large part due to the influence of free software, there has been an explosion, a move toward going open access. All of this is laudable and I fully embrace it (and have gotten into some small battles over it). But without an aesthetic politics that values pleasure in reading and writing we are doomed to obscurity anyway. A move toward making our knowledge public also required a move toward thinking about the literary aesthetics of pleasure.

Now that I am (thankfully) done teaching until September, I have time to devour two small mountains of readings that I need to finish before I return to my manuscript, which I will be working on, I hope uber-productively, all summer long. One pile of readings deals with coding, open source, and the commons, such as Scott Rosenberg s Dreaming in Code and David Bollier s Viral Spiral. Another pile of reading edges toward the theoretical side of things, having to do with craft, pleasure, and humor, since it is pleasure in its many many many guises from from the calm feeling of self-satisfaction that underlies pride in one s craft, to the more sublime feeling of ecstatic bliss that powers many creative sprints. If one entertains pleasure, one must also entertain its darker side, for all of this feel good stuff is nonetheless often springs forth from a deep sea of passionate frustration. This seems to be the driving theme of Dreaming in Code and it is also what animates Ellen Ullman s fictional account of pure frustration, The Bug. I am quite fond of native expressions of geek frustration and recently was provided with an exquisite example a rant against the Adobe PSD format. The author of Xee, A light-weight, fast and convenient image viewer for Mac OS X explained his utter contempt for the Adobe PSD in the following way:

At this point, I d like to take a moment to speak to you about the Adobe PSD format. PSD is not a good format. PSD is not even a bad format. Calling it such would be an insult to other bad formats, such as PCX or JPEG. No, PSD is an abysmal format. Having worked on this code for several weeks now, my hate for PSD has grown to a raging fire that burns with the fierce passion of a million suns. If there are two different ways of doing something, PSD will do both, in different places. It will then make up three more ways no sane human would think of, and do those too. PSD makes inconsistency an art form. Why, for instance, did it suddenly decide that *these* particular chunks should be aligned to four bytes, and that this alignement should *not* be included in the size? Other chunks in other places are either unaligned, or aligned with the alignment included in the size. Here, though, it is not included. Were it within my power, I would gather every single copy of those specs, and launch them on a spaceship directly into the sun.

Even if this account represents unadulterated irritation, he leaves us, the reader, with nothing of the irritation, only pleasure. This aftermath of frustration is delivered through the vehicle of humor, which within the hacker context, is the cultural container that best captures the spirit of hacker pleasure or so I will be arguing. Like many humorous rants from the world of hacking (and please send me any others you know of), this text dances with liveliness; it exudes its own rhythm; it glistens to use Ronald Barthes apt phrasing from his short book The Pleasure of Text, which I just finished as part of my theoretical escape into the pleasure-dome. Although there are parts of his book which are to be frank, *really* *not* *pleasurable*, partly due to obscure references to High French Theory, which elide even an academic pair of eyes, the book generally pleases. And one of the most pleasing chunks is his definition of a stereotype:

The stereotype is the word repeated without any magic, any enthusiasm, as though it were natural, as though by some miracle this recurring words were adequate on each occasion for different reasons, as though to imitate could no longer be sensed as an imitation: an unconstrained word that claims consistency and is unaware of its own insistence

In contrast to the stereotype, a string of words that enchants does so by slipping off the page to hit you squarely in the heart or the gut. Unfortunately, while academic writing steers clear of stereotypes, often trying to present the detailed singularity of a phenomena (even when conditioned by social forces), it does not exactly glisten, though there are a handful of exceptions. I think we need more texts that glisten, even if only during sections or parts of our books and articles (much like the rant helped enliven the more staid technical document). In recent years, in large part due to the influence of free software, there has been an explosion, a move toward going open access. All of this is laudable and I fully embrace it (and have gotten into some small battles over it). But without an aesthetic politics that values pleasure in reading and writing we are doomed to obscurity anyway. A move toward making our knowledge public also required a move toward thinking about the literary aesthetics of pleasure.

There s little that scares Kansans more than a good tornado. And, truth be told, there s little that excites us more than a good tornado, either. After all, we know it s going to be good for a story. Here on the plains, it is easy to remember how powerful nature really is. Weather sweeps in from all directions with little to hinder it, and tonight was a night few will forget. The 300-member Kansas Mennonite Men s Chorus, of which I m a member, had a concert tonight at 7, and we had a practice starting at 3. We sang in the beautiful old Memorial Hall on the campus of Bethel College. I need to describe the building a bit for you. It was built in 1938, in the day before auditorium designers decided you need to remove the outdoors. Three sides are full of windows, and it s just a few feet from the front doors into the auditorium itself. There is no air conditioning, so in a very humid day like today, every possible door and window is open. To stand on the stage and look out over the audience, you see light coming in from every direction, extremely bright light when there s lightning. It s been storming all afternoon. Our rehearsal featured thunderclaps, driving rain that was very audible inside, flickering lights. It was, in all honesty, a bit exciting. At about 6:35, word came that a tornado warning had been issued for the county. A few minutes later, about 20 minutes before start time, sirens were activated and the decision was made to evacuate to safety in the building s basement and an adjoining building. I, and a number of college students singing with the choir, wound up as a sort of impromptu ushers, directing people to the best place for safety and helping to get people into space efficiently. It was rather stuffy in those basement hallways, but it was also quite loud due to all the excited visiting going on. The KMMC concerts aren t usually like this. A few minutes later, word came that the tornado warning was canceled. I went down the steps, yelled out the news, and got a big cheer. People poured back upstairs. We got started a few minutes late, but nobody quite cared. A group of brass players was going to provide the prelude, but their first trumpet player had to leave at the last minute because his tool shed had blown over. They covered for him well in the concert though. When I got home later, Terah saw me and said, I thought they d cancel the concert because of the tornado. I just had to laugh, and pointed out, This is Kansas. After a tornado evacuation, it just got everybody more excited for the program! (Of course, had there been a strike, it would have been a different story.) And it was a great program. There was excitement in the air. And it was incredibly hot and stuffy on the stage with 300 men. I had sweat running down my face. The rain and the thunder continued, sometimes improving the program. About halfway through, we sang Prayer of the Children, written by Kurt Bestor, someone that was terribly saddened by the destruction during the civil war in the Balkans. The lyrics go, in part:

Can you hear the prayer of the children
on bended knee, in the shadow of an unknown room? Empty eyes with no more tears to cry,
turning heavenward to the light. Cryin Jesus, help me
to see the morning light of one more day. . . Can you hear the voice of the children
softly pleading for silence in their shattered world?. . . Angry guns preach a gospel full of hate
blood of the innocent on their hands. For when darkness clears, I know you re near,
brining peace again. Can you hear
the prayer of the children?

And when we get to the last phrase, there s a long rest after Can you hear . We sang it the phrase, and with perfect timing as we paused for the rest, was a loud clap of thunder. And after the last word of the song, total silence. Nobody moved for a few seconds, and then applause rang out. A couple of years ago, I was in the audience at a concert in the same building. Ominous weather lurked that day too. Before the concert started, the director said something like this, before giving information about evacuation procedures, should they be needed: We are plains people. We are privileged to see the awesome power and beauty of the weather and nature up close every day. If we should be a little too privileged today, here s what we ll do

DarkSEO has some code to attack php3bb captchas. (Note: I didn't even look at the code, it could be a virus or anything).I do not find that very surprising that this has happened, most of the captchas around are very naive, and I've seen multiple scientific articles detailing how to attack various captchas. Many use colors and thin lines to make them look hard, but after applying a naive energy function and doing some blurring to remove the thin lines, they break down.ReCaptcha is quite interesting, because it doesn't bother with some useless colorification that doesn't change contrast. But I wonder if it can't be overrun by spammers and how long it will scale. Still I figure it is what I would pick right now, because they can upgrade it if it actually is attacked by solvers.It doesn't help much for the proxy attack on Captchas though (offer users to view some pr0n in exchange for solving a Captcha that you actually were given to solve by another site) - at least not when combined with some XSS and/or bot net. (The 'obvious' proxy approach can be IP-filtered.)

I have found myself, at least on a few occasions, talking to a person whose relationship to the Internets is much thinner than my own, trying to explain the nature of certain Internet memes. I find this a pretty tough task to pull off as I tend to make memes sound really infantile (and perhaps that is just what they are). To explain this world to the uninitiated, I either: 1) very quickly give up and move onto some other less obscure 2) go into semi-professorial lecture mode and give a mini-low down on memes, some of the early examples, a small theory of self-referential irony, 4chan, rickrolls, encyclopedia dramatic etc etc and then show a bunch of memes as really you can t do it justice without the audio visul component. Now, I can just point people to the Meme Factory to three youngish guys from Brooklyn who put together a **fantastic show**** a few weeks ago on, yes, Internet Memes. It was a dazzling ride into the (sometimes very obnoxious) territory that is memeology, addressing and displaying both the happy-cheery-cute side of memes as well as its darker side. The video they have on their site documents their presentation and the word is out that they will soon craft something more like a stand-alone video-essay about Internet memes. As we wait for this next step in their project, this video is an amusing watch (though the interesting stuff starts about 3 minutes in).

I have found myself, at least on a few occasions, talking to a person whose relationship to the Internets is much thinner than my own, trying to explain the nature of certain Internet memes. I find this a pretty tough task to pull off as I tend to make memes sound really infantile (and perhaps that is just what they are). To explain this world to the uninitiated, I either: 1) very quickly give up and move onto some other less obscure 2) go into semi-professorial lecture mode and give a mini-low down on memes, some of the early examples, a small theory of self-referential irony, 4chan, rickrolls, encyclopedia dramatic etc etc and then show a bunch of memes as really you can t do it justice without the audio visul component. Now, I can just point people to the Meme Factory to three youngish guys from Brooklyn who put together a **fantastic show**** a few weeks ago on, yes, Internet Memes. It was a dazzling ride into the (sometimes very obnoxious) territory that is memeology, addressing and displaying both the happy-cheery-cute side of memes as well as its darker side. The video they have on their site documents their presentation and the word is out that they will soon craft something more like a stand-alone video-essay about Internet memes. As we wait for this next step in their project, this video is an amusing watch (though the interesting stuff starts about 3 minutes in).

I got interesting comments on my previous post about SSH through jump hosts, that made me think a bit. The first one of them suggested to use a syntax that unfortunately doesn t work on anything other than zsh, but had the nice idea to add a login for each jump server. Sadly, this only works when there is only one hop, for reasons similar to those we will explore below. The second one suggested / would not be so good a separator, because it would fail in svn+ssh urls and such, which is a good point that I didn t think of only by lack of such use case. The simplicity of the resulting configuration was nice enough. This decided me to take a shot at implementing something that would allow adding both login and port number for each jump server. The first implementation detail to set was which characters to use for login, port number and hop separators. I chose respectively % (@ is unfortunately impossible to use because ssh would take it, leaving out any hop before the character), : (pretty standard) and + (not allowed in DNS, and not too illogical as such a separator, I thought). The result can look like dark magic when you are not savvy with sed and regular expressions:

Host *+*
ProxyCommand ssh $(echo %h sed 's/+[^+]*$//;/+/! s/%%/@/;s/:/ -p / ;s/\([^+%%]*\)%%\([^+]*\)$/\2 -l \1/ ) PATH=.:\$PATH nc -w1 $(echo %h sed s/^.*+//;/:/!s/$/ %p/;s/:/ / )

The syntax you can use to connect through jump hosts is the following:

ssh login1%host1:port1+login2%host2:port2+host3:port3 -l login3

We ll see further down why we can t put a login for the final destination host. Or if you want to skip the boring implementation details, you can also skip to the end, where a slightly more compact version lies: while writing this post, I got some optimization ideas. Let me try to split the sed syntax to make it a little more understandable:

s/+[^+]*$//; # Remove all characters starting from the last + in the string (i.e. keep the n - 1 first hops)
/+/! # If the string doesn t contain a + after the previous command (i.e. there is only one hop remaining), do the following, otherwise, skip until the closing curly brace
s/%%/@/; # Replace % with @ (% is doubled because of the ProxyCommand)
s/:/ -p / # Replace : with -p . Combined with the previous command, we rewrite login%host:port as login@host -p port
;
s/\([^+%%]*\)%%\([^+]*\)$/\2 -l \1/ # Rewrite hop1+hop2+login%lasthop as hop1+hop2+lasthop -l login

The second sed goes like this:

s/^.*+//; # Remove everything up to the last occurrence of + in the string (i.e. only keep the last hop)
/:/! # If there is no : in the string, do the following, otherwise, skip the next statement
s/$/ %p/; # Add %p at the end of the line
s/:/ / # Replace : with . These last three instructions prepare a host port combination for use with nc.

Let s see what the ProxyCommand looks like for some examples, skipping PATH setting and nc option for better readability:

host1+host2

ssh host1 nc host2 %p

login1%host1+host2:port2

ssh login1@host1 nc host2 port2

login1%host1:port1+host2:port2

ssh login1@host1 -p port1 nc host2 port2

Now, maybe you start to see why we can t put a login on the last hop: not only does it make no sense for nc, but also the main ssh process that runs the ProxyCommand and will actually talk to the remote host won t have any knowledge of it. From the above examples, it also appears obvious why we replace login%host:port with login@host -p port : so that the ssh command in the ProxyCommand gets the proper arguments for login and port. Note we could also replace with host -p port -l login for the same effect. With even more hops, this is what happens:

host1+host2+host3

ssh host1+host2 nc host3 %p

login1%host1:port1+host2:port2+host3

ssh login1%host1:port1+host2:port2 nc host3 %p

Each of these ProxyCommands will trigger another ProxyCommand quite looking like our first few examples. In the first hops, if we d just replace all % with @ , in cases where logins are given everywhere, we d end up with a ProxyCommand like the following:

ssh login%host1+login%host2 nc host3 %p

As we saw above, we can t use a login on the last hop, which means this ProxyCommand wouldn t work as expected and is why we have to rewrite login1%host1+login2%hop2 as login%host1+host2 -l login2 with the last instruction in the first sed. In the end, this is what happens:

login1%host1+login2%host2+host3

ssh login1%host1+host2 -l login2 nc host3 %p

login1%host1:port1+login2%host2:port2+host3:port3

ssh login1%host1:port1+host2:port2 -l login2 nc host3 %p

In this last example, you see the :port2 part is not changed into -p port2 . Actually, it would still work with the latter form : the ProxyCommand ssh login1%host1:port1+host2 -p port2 -l login2 would itself have ssh login1@host1 -p port1 nc host2 %p as ProxyCommand, in which %p would be replaced by port2, given as argument to -p. Based on this and with further small optimizations, we can slightly shorten our ProxyCommand to the following :

Host *+*
ProxyCommand ssh $(echo %h sed 's/+[^+]*$//;s/\([^+%%]*\)%%\([^+]*\)$/\2 -l \1/;s/:/ -p / ) PATH=.:\$PATH nc -w1 $(echo %h sed s/^.*+//;/:/!s/$/ %p/;s/:/ / )

sed instructions decryption is left as an exercise to the reader ;). Update : modified the shortened version to fix the issue spotted in the comments.

Oh well.... after the last weekend I ended up with a bunch of Nikon RAW images (*.NEF) from a different camera (Nikon D40X). I made my pictures as RAW+FINE with stores as well RAW files as JPEGs together. But for the images of that other camera I need to convert the images to JPEGs all by myself. Usually I use digikam to manage my images, which comes with some sort of RAW converter.

But needless to say that the results are disappointing. Very dull, dark and greyish JPEGs were generated from the RAW files, although digikam displayed the RAW images just fine and as bright as expected. So I wonder why there is this difference between what digikam displays and what you get out of RAW conversion?

For the time being I now convert the RAW files manually by invoking dcraw and convert as follows:

dcraw -w -b 1.2 -q 3 -T DSC_0*.NEF
for i in *.tiff ; do j= echo $i sed -e 's/tiff/jpg/g' ; convert $i $j ; done

The result is not the optimum, but fairly OK. It's still a little bit dull, dark and greyish, but not as worse as the standard conversion from digikam as you can see here:

This is a really slim and really incomplete description of the events turning really bad for Romania and severely bad for Moldavian civilians.

• Moldavia calls his ambassador from Romania
• the peaceful protests turn violent on Tuesday (while Voronion recognised for Ria Novostni that they allowed the protesters to control for a day the Presidential building and the Parliament (in Romanian)
• the Moldavian public is kept in the dark by the authorities: the national radio-television doesn't cover the events, the independent media is not allowed to report the events
• in the rural areas of Moldavia no other media is accessible except the national television which is controlled by the Communists
• Moldavian police men tear down a wall close by the Moldavian Parliament during the violent turn of the protest
  
• Moldavia accuses Romania without any proof of implication in the events in Moldavia
  
• The Romanian ambassador Filip Teodorescu, and the Counseling Minister of the Romanian Embassy in Moldova, Gabriel Gaboran, are declared persona non grata in Moldavia; Romania can't do the same since the Moldavian ambassador is not in the country
• Vladimir Voronin announces visas will be introduced for Romanians (an abussive measure since the EU should have been announced of such a measure with 48 hours in advance)
  
• Romanian authorities respond by:
• a ferm denial of the unproven accusations - from what I have seen CNN and Euronews simply don't accurately represent the declaration, and the texts put under quotes - the word "provocation" - imply that the Romanian position is not honest
  
• they declared that they will NOT declare the Moldavian ambassador as a persona non grata and that the regime of gratuitous visas for the Moldavian citizens will not change
• EU DOES NOT ANSWER AT ALL TO THIS ABUSE AGAINST EU CITIZENS
  
• Wednesday protests are quiet but media access is highly restricted and inaccurate information is aired through the national Moldavian Television
• Romanian journalists from Antena 3 were harassed, followed and, fearing for their safety, they appealed OSCE authorities in Moldavia to be able to leave the country safely
  
• Wednesday and Thursday many arrests occur in Moldavia and the Romanian officials requested the Moldavian officials to tell them if among the arrested people are any Romanian citizens; no answer was provided by the Moldavian authorities
• France, Sweden and Czehia and some other EU members declare that they "understand the complexity of the relation sensible issue between Romania and Moldavia" and appeal to Moldavia to rebound the diplomatic relation with Romania - WHAT? In Diplomatic lingo that seems to mean something like "we know Romanians are idiots, but please, Moldavian authorities, let's get back to dialogue"
  
• Today, Friday 10, the Moldavians are called to protests and the authorities are authorized to use real ammunition
• Teachers, university administration personnel, opposition representatives announce that people should not attend the protests and measures will be taken against anyone allowing people (mostly youngsters) to attend the protests
• Russia declares that "the EU has a member state that hasn't clearly delimited its borders" which obviously refers to Romania, thus attacking Romania and alleging that Romania has unionist expansions; EU DOESN'T DO ANYTHING!

Are we living in a parallel world? What interests would Romania have to ever consider that it would be involved in this? Moldavia is the poorest country in Europe, has huge problems with justice, corruption, criminal activity, social security, border security (Transdnistria) and other issues which Romania does NOT want to import since we're already in a bad position by ourselves!


WHAT THE HELL IS EU DOING? Romania is a member of the EU and it seems all the attacks against Romania are not addressed in any way by the EU, on the contrary, EU members either assist passively, look the other way or even allow us to understand that indeed they think Romania is involved.

New York City is home to fantastic set of libraries, including the the mothership of libraries, the NYPL. My favorite library is one that is much smaller, located in the north of Manhattan on the campus of Columbia University: East Asian Starr. Quiet and compact, there are tall ceilings, dark wood, stained class windows, and reading nooks. When I enter, it feels like I step back in time (if I ignore the laptops, at least). When I was an undergraduate there, it was my library of choice and I logged many hours there and have yet to find a library I so love. I also was fond of the place because of their discreet open door policy. When I was a student there, over a decade back, they were supposed to ask for you ID but they never did. When I returned back to NYC in 2007, I was thrilled to find this practice of ethical inattentiveness was still followed by the student works behind the desk. I always wanted to find out more about what sustained this practice but I never dared or was too lazy to do so. So I was pretty saddened when I popped in the library on Friday to discover a new security system that requires you to scan your ID to get in. There were large signs also indicating such, clearly seething with some passive aggressive anger expressing disbelief that this library had managed to evade security for over decade. So while it is a real shame that this gem is no longer a place that one anyone can enter at will, it is nice to know that it took so long for someone to notice.

Today over lunch, I and 6 others went to visit Grandma. She was in her room, looking better physically today than yesterday. When I walked in, I said brightly, Hi Grandma! She took my hand warmly, and said to me, Now I don t think we ve met. Who are you? I knew that moment would happen someday, but still was surprised when it happened. Mom told her that we were there to tell her we love her. Grandma counted out the seven of us, and said, All these wonderful people here to tell that to ME? That s Grandma still there! It was a difficult moment. Many of us were tearful, and Grandma was sick enough that she sometimes lost the battle to stay awake. But we were all glad it was happening. Mom thought it right (me too) that she should tell Grandma about the latest word from the doctors. The conversation went, in part, something like this, with my mom addressing her mother: My Mom: Mom, the doctors say your heart is probably wearing out. My Grandma: *shrug* Yeah. My Mom: And your body is probably tiring out too. My Gramdma: Yes it is. I m 94. It s not that Grandma was depressed or anything. Just that she had long ago been at peace with the idea of death, and actually told us more than once that the was rather surprised that she has lived to be as old as she has. So it wasn t frightening or surprising to her to confront her own mortality. To her, it was a fact, and an obvious one at that. My brother brought along Grandma s old Bible. She had given it to him a few years ago. It was filled with highlights and handwritten notes from cover to cover. She had carefully analyzed it, and when she bought a new Bible, had carefully copied the notes to it. My brother read to her two passages that she had highlighted:Psalm 23: (which she has noted as a favorite of many Christians ) and Numbers 6:24-26. He then held her hand and said a prayer with her. Then he said a blessing for her, and as he was getting to the end, she interrupted, saying the last word for him: forever! She still had her sense of humor, and made us all laugh several times. She said, as she always has when any of us stop by to visit, how happy she is that we came by, and what a wonderful family she has. She said several times Danke sch n, Danke sch n, Danke sch n! (Thank you, thank you, thank you! she had made sure to teach a few German words to all of us as kids.) When it was time to go, she got lots of hugs from us, and made a point to tell each of us individually thank you for coming! As we walked down the hall, I was reminded of her old tradition going back many years. Whenever we would leave her house, she d wave to us from the porch. And, if it was dark, we d turn on the car light and wave back. Since she s been in the nursing home, she ll wave to us from her doorway as we walk down the hall. And, sure enough, today s visit ended with her waving to us from her doorway with both hands (as she always does) and a big smile. We all waved back with a smile as we walked away, too. I don t know if this will be the last goodbye with her, but if it is, I can t think of a better one.

And there we have done it - new sections in the archive. Read more about it in my d-d-a post and the whole set of affected packages is also available - and updated slightly, as some small things slipped through. The whole process was less painful than feared. After some discussion, started a few weeks ago we finally went and coordinated the change today. There was no reliable and good way for changes that large, so we had to create our own way. (The last section changes had been done ages ago). Mike provided a little python script which was able to parse our input file with the section matches against our database, so we ended up with a list we could possibly feed back into the database. Except, not really. Thanks to various things the list generated needed some manual work (which I also managed to fuck up a little), but after that was done we were ready to go. And while Mark told the wanna-build admins about the new sections (for some reason our buildd network cares about sections) and prepared a patch for our policy, I prepared the mail for the d-d-a list. Then we all had a break, as a dinstall was still running, and we did not want to break that. The actual changes then took only a few seconds, sending the mail to d-d-a took longer, thankyou gnupg. :) I wonder how much afterwork this will generate. There are at least the policy changes, lintian, the packages control files to adjust. Also some websites, where Frank already changed packages.debian.org to know about the new names. (Before he fixed up me forgetting some PHP packages to move. This way all of the active ftpteam members did something in it :) ).

---

In other news, we are currently stress-testing our ftp-master host a little. Well, not really, it could take more, but right now we are scanning the full archive once, importing all filenames from all our Debian packages. The goal is to get projectb larger. Err, no, actually we do want to get rid of apt-ftparchive, as that is one of the worst tools in our dinstall runs. It is slow as hell in the mode it has to run for us, making up a large part of our runtime. Step 1 to get rid of it is to generate the content files on our own, and before we can do that we have to import them all Which means we have to unpack every binary package once, reading in all filenames to store them in our database. Thats a lot of data. Currently the associations table, which is nothing more than 4 integers per row, takes up 972MB, and import is still running. (The index on the binary pkg id in this table alone has 700MB). And this table does nothing more than linking the binary package to the filename and filepath. Which are two extra tables, with the filenames having only 87MB, the path entries only taking up 30MB. Yes, anyone using databases will tell me thats not much, and it really isn t that much. It s just that the largest table before this import had 120MB in it. And some of the others used more space for their indexes than for the actual data. :) Anyway, this is only step one on our way to get rid of the slowness apt-ftparchive is. When the contents all work as we want them, the next step will be to get rid of it generating Packages/Sources files. Which means we have some more data to store in projectb. And after that - we also intend to do a Contents file for our source. Each and every source file listed. Guess we really will end up with some more large tables. :) The final goal and main reason why we do this is to save us at least 60 minutes runtime per dinstall run. My own guess is that Contents alone will get us about 20 minutes down. No, before someone wants to yell at me - apt-ftparchive can t be changed as much to make this possible and keep using it. Not in the way the generate option works. It has the disadvantage that it scans the files after they are installed in their pool/ location and stores that in its own database . This approach has to negative points:

• first, it is using another database (BerkeleyDB). We do have one database already, projectb, thank you.
• second, and more important, the files only end up in pool/ on dinstall time. Which means the whole scanning is done during dinstall. Which eats time. The new way will surely use the same amount of cpu time in total - but it will spread it over the whole day. 4 runs each hour. Not 4 runs each day. And during dinstall it will boil down to one sql query per installed package moving data from a pending table into the real contents table. Which is a cheap operation.

---

The title says some more, so here is some more: We aren t only working on what I wrote above. There is also work on changing our database access. Historically dak used the pg module to do database work. While the module itself does work, the way its used isn t really the best one. Way too much groundwork everywhere. Not nearly as much of the nice things used people say python could have. So another largish work area is getting rid of pg. Migrating to psycopg2. And putting a proper OO class around it, which in the end hopefully hides those dark details from the application. And to achieve even more sanity, we (well, honestly, Mark) are looking into SQL Alchemy usage. (We also considered storm from Canonical, but that sucks when you take a look at its documentation. (Which documentation?). SQLA won there easily.) While I m ranting: WTF are people neglecting their docs that much? Ever tried looking at the documentation for psycopg2? Forget it. No, a db api compliance test suite is NOT documentation. If there would be any better python module to connect to postgres, I would veto the usage of psycopg2, but there doesn t seem to be one. Sucks.

So I know, I know. There are a lot of annoying memes circulating on the net/facebook/[pick your spot] and well, sometimes the best thing to do is stop the damn thing by ignoring its existence. But I have to say, I have completely, totally and absolutely have loved reading the 25 things notes on facebook. It just provides a window into the extraordinary character that emerges from the collection of mundane events/likes/pleasures that make up people s lives. Reading them took me back to elementary school when and where I was an avid slam-booker. I have very fond of collecting and reading the tidbits of thoughts and information people left behind in my notebook. What also struck me about these recent 25 thingie notes is how odd or not as interesting, I think, it would be to have them recited to you over coffee or at dinner. Now I could be wrong here but I think the list-like quality would just not fly in a conversational context. A few facts with more embellishment would work but a whole string of thoughts would just sound flat and out of place. A reminder that format does matter for engendering certain types of knowledge. So here is my list.. 1. I looked like a boy for the first three years of my life (but a cute one, I think).
2. I love dogs so much that I often think that if I had a criminal streak, I would be a dog nabber.
3. I would routinely bring home dogs from the street when I was a kid. Many of them ended up dead after being hit by cars (they liked to return to the street and this only happened to 2 of them).
4. My favorite concert was Jimmy Cliff in a parking lot. There was lots of rain and lots of weed.
5. I like have a penchant for tall men. My mom claims it was because I lacked a strong father figure in my early childhood years. Who knows if that is the case.
6. I knew the instant that I took an anthropology class I wanted to be an anthropologist. I guess I made it happen (and it was one of the few moments of clarity in life).
7. I started the first environmental club in my high school and my car was called the recycle-mobile because there were always cans in the back seat.
8. I love the ocean; swimming, snorkeling, diving, sailing, and just watching it, though the cold ocean is not always by cup of tea.
9. I had terrible insomnia for a year and slept for a good chunk of it on a couch. I like the snuggle feeling I got from sleeping against a back.
10. I am very indecisive person. It is annoying as i tend to worry a lot about whether I made the right decision.
11. I was a shy kid and barely talked until meeting my best friend in kindergarten, Yael
12. I also failed the psychological test to get into school but my mom managed to get me in anyway.
13. I often wonder if I would have been a nun if I had lived prior to 1900s to escape various domestic obligations. I think it might have been fun to be a nun.. Fun to be a nun also has a nice ring to it.

14. I am currently struggling with an Etsy addiction and the UPS delivery guy asked if delivering packages would be a daily occurrence.. I just find all the stuff amazing and it brings me back to my artist days when I used to paint and spend a heck of a lot of time in the darkroom.
15. I have recurrent issues with nausea and have been hospitalized for it a few times. It is gut wrenching and really uncomfortable when it happens.
16. When I left NYC in 1997 I thought i would never come back. I am here now and quite happy.
17. I was obsessed with Ultimate Frisbee for many man years. I played more than I studied and finally gave up as it was ruining (or perhaps ruined?) my neck. I still miss it.
18. I LOVE tostones and would eat them everyday if they were not, you know, so bad for you.
19. I have been using IRC for 7 years and consider some of my best friends to come from there.
20. Some folks have asked how I became interested in politics and I am pretty sure it was Doonesbury.
21. When I was like 4 we went to some fair and I swore that I almost got flung off some rickety ride but this could be totally wrong. It was really frightening.
22. The coldest night of my life was spent in Los Nevados, Venezuela and the scariest car ride I ever took was leaving Los Nevados in a jeep and something that resembled a road.
23. A few sucker fish once tried to stick on me while I was doing a decompression stop after a dive. My dive buddy smacked the fish, which had devil faces, with the regulator, saving me from what would have been a really sort of sucky situation.
24. I am trying to learn how to surf and am really sorry that I did not do so when a teenager since I don t exactly have the time to really learn.
25. Since I am not sure how to end, I will do so with a

So I know, I know. There are a lot of annoying memes circulating on the net/facebook/[pick your spot] and well, sometimes the best thing to do is stop the damn thing by ignoring its existence. But I have to say, I have completely, totally and absolutely have loved reading the 25 things notes on facebook. It just provides a window into the extraordinary character that emerges from the collection of mundane events/likes/pleasures that make up people s lives. Reading them took me back to elementary school when and where I was an avid slam-booker. I have very fond of collecting and reading the tidbits of thoughts and information people left behind in my notebook. What also struck me about these recent 25 thingie notes is how odd or not as interesting, I think, it would be to have them recited to you over coffee or at dinner. Now I could be wrong here but I think the list-like quality would just not fly in a conversational context. A few facts with more embellishment would work but a whole string of thoughts would just sound flat and out of place. A reminder that format does matter for engendering certain types of knowledge. So here is my list.. 1. I looked like a boy for the first three years of my life (but a cute one, I think).
2. I love dogs so much that I often think that if I had a criminal streak, I would be a dog nabber.
3. I would routinely bring home dogs from the street when I was a kid. Many of them ended up dead after being hit by cars (they liked to return to the street and this only happened to 2 of them).
4. My favorite concert was Jimmy Cliff in a parking lot. There was lots of rain and lots of weed.
5. I like have a penchant for tall men. My mom claims it was because I lacked a strong father figure in my early childhood years. Who knows if that is the case.
6. I knew the instant that I took an anthropology class I wanted to be an anthropologist. I guess I made it happen (and it was one of the few moments of clarity in life).
7. I started the first environmental club in my high school and my car was called the recycle-mobile because there were always cans in the back seat.
8. I love the ocean; swimming, snorkeling, diving, sailing, and just watching it, though the cold ocean is not always by cup of tea.
9. I had terrible insomnia for a year and slept for a good chunk of it on a couch. I like the snuggle feeling I got from sleeping against a back.
10. I am very indecisive person. It is annoying as i tend to worry a lot about whether I made the right decision.
11. I was a shy kid and barely talked until meeting my best friend in kindergarten, Yael
12. I also failed the psychological test to get into school but my mom managed to get me in anyway.
13. I often wonder if I would have been a nun if I had lived prior to 1900s to escape various domestic obligations. I think it might have been fun to be a nun.. Fun to be a nun also has a nice ring to it.

14. I am currently struggling with an Etsy addiction and the UPS delivery guy asked if delivering packages would be a daily occurrence.. I just find all the stuff amazing and it brings me back to my artist days when I used to paint and spend a heck of a lot of time in the darkroom.
15. I have recurrent issues with nausea and have been hospitalized for it a few times. It is gut wrenching and really uncomfortable when it happens.
16. When I left NYC in 1997 I thought i would never come back. I am here now and quite happy.
17. I was obsessed with Ultimate Frisbee for many man years. I played more than I studied and finally gave up as it was ruining (or perhaps ruined?) my neck. I still miss it.
18. I LOVE tostones and would eat them everyday if they were not, you know, so bad for you.
19. I have been using IRC for 7 years and consider some of my best friends to come from there.
20. Some folks have asked how I became interested in politics and I am pretty sure it was Doonesbury.
21. When I was like 4 we went to some fair and I swore that I almost got flung off some rickety ride but this could be totally wrong. It was really frightening.
22. The coldest night of my life was spent in Los Nevados, Venezuela and the scariest car ride I ever took was leaving Los Nevados in a jeep and something that resembled a road.
23. A few sucker fish once tried to stick on me while I was doing a decompression stop after a dive. My dive buddy smacked the fish, which had devil faces, with the regulator, saving me from what would have been a really sort of sucky situation.
24. I am trying to learn how to surf and am really sorry that I did not do so when a teenager since I don t exactly have the time to really learn.
25. Since I am not sure how to end, I will do so with a