I read a translated poem about Russia being "the Motherland" and its vast bosom years ago. Having driven through a significant part of it, I can agree on the "vast" part... Also, as I am on a train and without access to the Internet, I will refrain from linking to a lot of pages; sorry. (Turns out I am posting this a week later, but I will still not link to stuff now; no time). Russia in general

• All receipts you receive are torn before you get them; this is most likely due to the old Soviet voucher system, more on that below.
• Russia was hot with temperatures ranging from 27 to 32 degrees Celsius between Moscow and Ulan Ude.
• There aren't a lot of pedestrians bridges, but a lot of pedestrian tunnels. The sides of those tunnels are packed with tiny shops, often only two meters wide and 50-70 cm deep. Everything from stockings to candy over glasses to flowers and watches is being sold through a tiny window by some poor woman who somehow managed to get in there.
• Toilet brushes stand in water. In Germany, that's a sure sign of a really dirty toilet; in Russia, it's the thing to do. If you are lucky, there's blue cleaning stuff added to the water. If not, it will still have color. You are free to guess which.
• Queuing is war.

Moscow Sights Kreml Our remaining time in Moscow was spent with touring the usual suspects; the Kreml is a lot less impressive in real life, the Red Square is tiny when compared to the stories I heard about it and the Chapel ofi St. Basil is even more colorful and impressive in real life. Lenin's body was inaccessible because workers built seats for the May 9th parade to the left and the right of it and they apparently thought it would be a good idea to block access to one of the main tourist attractions while doing so. A river tour of Moscow was a nice cool-off and we got to see quite a few things. We managed to see the weekly military parade within the Kreml grounds, but it was mostly pomp and little substance. The National Treasure which you can access with an extra ticket within the Kreml grounds is nice, but less impressive than the tourist guides would make you believe. That being said... There's another museum within the museum and.... Whoah... Tourists pay extra, visitors go through the only non-security-theater check I encountered in Russia, guards are armed, people can only enter and leave in batches, and the stuff which is presented is mind-boggling. Disregarding the fist-to-calf-sized chunks of gold and platinum which are still in their original form directly from the mine, there is real, actual treasure galore. Little heaps of uncut and cut diamonds, an outline of Russia filled with cut diamonds and other random "we have this stuff" displays can be found as well. Then, you have various tiaras and other jewellery made from various gems. Not incorporating, but largely made of. All that pales in comparison to the crown, royal apple, scepter, etc. It's hard to put the amount of tiny multi-colored light points that shine at you into words. I was just standing there, swaying back and forth to catch the moving pattern of pinpoints. It's said that this collection is equalled only by the ones in the Tower of London and the one Shaw of Iran had and boy do I believe it. TV Tower Getting up there was funny. The old-style Soviet queuing system was used:

• Go to a counter to tell an attendant what you want; receive stub
• Go to another counter, hand over stub to another attendant, pay for what you want; receive voucher
• Go to third counter, hand over voucher; receive ticket for tower The whole thing was made even more absurd by the fact that counter one was in the middle, counter two to the right and counter three to the left. As Russians do not believe in queuing and everybody just tries to get in first, this made for a nice little exercise.

"Security" for approaching the tower was multi-level, the guards see you approach along a long walkway way in advance and the main guard shed had several small cabins separated by thick glass. So good so menacing. But in a twist that would make Bizarro and Garry Larson proud, I was required, by means of metal detector gate, metal detector wand and even an x-ray machine to remove every shred of metal and other hard objects from myself and the camera bag and put them onto a table. Once I was without anything except my clothes and the bag was completely empty, I could pass. Everything I had had to remove was just laying there, not inspected in the least, for me to stuff back into pockets and bag and to take with me. This "everything" included a Spot Messenger 2 with lots of green and red blinky lights. The guard did not even glance and it. Security theater? Security theater. The view from 364 meters down on Moscow was nice, but there was a lot of Smog so I couldn't see very far. Jumping on the glass floor while looking down was a lot of fun, though. Subway to Thiefing I bet Christopher Nolan rode the subway in Moscow at least once. That unnerving sound you hear during several key scenes in "The Dark Knight"? Two thirds of all subways make the same sound while moving. Also, I had an encounter with a pickpocket down there; very classical, too. Guy approaches quickly, talks loudly and sounds as if it's really important (in Russian... duh... that's sure to keep me interested). His approach made me turn and protect my left leg pocket automatically, most likely marking the target for the tiny woman standing behind me. Now, I have to tell you something about my usual travel layout. As my normal pockets are very deep, it looks as if their content was in the leg pocket. Plus, there's an extra, hidden leg pocket where I keep the passports and train tickets. The outermost leg pocket is protected by a velcro flap, but it contains nothing of value; usually the appropriate phrasebook, local map, maybe a tissue or chewing gum. Due to this layering, the outermost pocket looks as if it's full to the brim with stuff. Also, I took pains to make it a habit to protect said leg pocket with my hand, nothing else. This looks as if that's the target, but what I am actually doing is protect my normal pocket with my forearm. The right side is different, but the most easily accessibly pocket always holds some small change. I pay from that stash but my actual wallet is well out of reach. Anyway, once the guy ran off, talking to several others, most likely marking all them for the actual pickpockets, I wanted to enter the subway. While the Russian-style queuing took place, I felt an unusual tug at the velcro flap. I looked down and saw a tiny woman to the left of me with a jacket held over her right side with the left arm; I look up to check no one is trying to steal from my permanently assigned female, feel another tug, look the woman into the eyes, look up again and around me, look down again and she is gone. All that took maybe three seconds and I had boarded the subway after an additional two. In hindsight, it makes sense to choose the time of entry for attack. It's crowded, you are being pushed around, and once you are in the subway, it will start moving more or less immediately while the thief remains in the station. In this case, she would only have gotten a grubby map of Moscow's subway and an English-Russian phrasebook, but she got nothing at all. Moscow-Novosibirsk Where to begin... If you think a few hours on a train are a long time, try over fifty hours. Things get so bad, you start getting land-sick while not in a moving train. You even start missing the familiar tunk-cachunk, tunk-cachunk, tunk-cachunk... of driving over rails with gaps in them when you are not moving. The defining element of the Trans-Siberian Railway are birch trees. And birch trees. And then more birch trees. You would not believe how many birch trees there are. This is made "worse" by the way the Russian Railway protects their rails. Left and right of the track, there's a cleared area of maybe ten to twenty meters, sometimes as little as three. Outside of that, they plant ten to twenty meters of birch trees, presumably to catch snow during winter. Beyond that protective perimeter, there's the normal landscape.As a result, on top of the near endless stretches of birch woods, you see most if not all scenery through a layer of birch trees. You get sick sick of birch trees after a few hours and you see them for days on end. Bullet points to save myself some typing and you some reading...

• More than a thousand kilometers without a single hill. Flat as a pan.
• The whole route is powered by electricity. No diesel engines in sight.
• Many stations are little more than a heap of smoothed gravel, bordered by some wooden planks. Some stations have obviously been built by locals and are even less well-defined.
• You can see people in the middle of nowhere, walking along the railway tracks. At first this seemed counter-intuitive, but most if not all roads out there are dirt tracks. As there seems to be standing water across a third of Russia, this dirt is turned into mud. After walking maybe twenty meters across a parking lot, I had to scrape a heavy, thick cake of black earth from my soles. The railway is the only functioning footpath those people have. Many people even build shoddy bridges towards the tracks from their homes, obviously preferring to walk along the tracks over walking through the village.
• Railway crossings along the Trans-Siberian route, no matter how tiny, have a small cabin beside them. While the train passes, there's one guy or gal standing in said cabin, holding a yellow stick vertically out towards the train. Sometimes, you have not seen any living thing, other than birch trees, for twenty minutes and there, in the middle of nowhere right beside a dirt track, there's someone holding a stick out towards the train. Weird.
• Railway crossings of paved roads will always have two steel plates coming out of the ground, angled towards oncoming traffic on each side. This may not stop a heavy truck at full speed, but a car will disintegrate on these barriers without touching a passing train.
• The railways is important for Russia. Two parallel tracks cut across the whole country, transporting everything back and forth. Where "everything" means mostly coal and birch wood, I guess.
• All freight trains are usually 70 tanker waggons or 100 box waggons long, but you see the odd 100 tanker waggons, as well. You have more than enough time and opportunity to count them and then some.
• There are supposedly women at every station, selling what they cook at home. Unfortunately, this was only true for two stations. The things we did manage to get were very nice; I do wonder why anyone would offer (or buy) cooked and peeled potatoes, though.
• Every waggon has its own hot-water stove. They are powered by coal. Yep, you have a coal fire burning in every single waggon on the Trans-Siberian.

Novosibirsk The non-existent hostel We arrived at ~0200 local and made our way to the hostel we had booked a room with. Walking to the correct address, we saw several signs but they all turned out to be for a police station and some other state agency. We walked back, forth, double-checked, triple-checked: no hostel. We then walked around the building through some not-quite-nice back alleys, but other than a few entries to private flats, there was nothing. Thankfully, the booking slip included a number which we called and after at least twenty rings (no kidding), when I had given up and wanted to hang up, it stopped ringing. Dead silence. After maybe ten seconds, someone started talking in Russian. I asked him if he spoke English and told him that we could not find the hostel. He mumbled something about being sorry and that we should wait, he would come down. Fast forward a minute or two and someone walked towards us. Again, he mumbled about being sorry, that the hostel "did not work" at the moment and that we would need to sleep in his private apartment. He ushered us into some back alley entrance, into his flat, and proceeded to remove the sheets from the couch on which he had slept; after putting on new sheets, we had our "hostel" bed, ready to sleep on. We briefly considered if he would murder us in our sleep, but him and me even got to talking a bit. Over cheese, sausage and rum (at 0300), he admitted that the hostel did not exist and he merely planned to turn his flat into a hostel for the summer while he and his family moved into their summer house (the Russian term of which escapes me, at the moment) in the countryside. He had accepted our reservation as he thought he would be finished by that time. He did not even get started, though. While he sent us an overbooking notice through booking.com two days before, we were on the train at that time, so... booking.com even called him to check what happenend to us as we did not book another place through them. Good customer service/protection, that. Next morning, he didn't even want to take our money (we paid anyway) and, as a means of compensation, drove us into the city in the morning and to a train museum well outside the city limits, one of the fabled scientist cities, and a large lake which everyone in Novosibirsk claims is an ocean, in the afternoon. Foreigners, foreigners! All in all, Novosibirsk was relatively uneventful, safe for one bizarre episode. We took our lunch in a local fast food joint (why do all the good stories happen there, and not at the various truly local places?) and threw the cashier our well-rehearsed "Niet Russkie; anglisky?" with phrasebook in hand and he actually understood a few words of English (beef, chicken, fries). We told him, in our worst Russian, that we are from Germany wished him a nice day and went to sit down. A few minutes later, a girl approached us, literally hopping from one foot to the other and wringing her hands. She told us that the cashier had told her that we spoke English and if it would be OK if she talked to us. We suspected some sort of elaborate ruse, but went with it. Turns out, she had English at school and really wanted someone to practice English on. Two young men passed our table and exchanged a few words with her, sitting down out of sight. When she told us that she had to leave now but if it would be OK if the two boys joined us we suspected a ruse yet again. But those two were law students, one with a minor in English and one with a minor in German; both of them also extremely nervous, asking us if we would talk to them. When they had to leave, they told us that the three of them worked at the burger joint and that their shift was just about to start when the news that foreigners were here spread amongst staff like wildfire. The girl stopped by several times in between cleaning tables, getting in a sentence or two before being cussed at by her supervisor. All in all, this took about twenty minutes and seeing three people so nervous and grateful to talk with us felt beyond absurd. On the other hand, not a single traveller we met even considered stopping in Novosibirsk during their transit so there really does seem to be a shortage of non-Russians there. Weird, and memorable. Novosibirsk-Irkutsk

• Birch trees.
• Lots of burnt underwood, presumably to prevent larger fires.
• Birch trees.
• Sticky, stuffy, 30+ degree waggon with windows that could be opened but which were locked (this is why I always carry a Swisstool with me).
• Birch trees.

Irkutsk / Listvianka / Lake Baikal Listvianka Aah, lake Baikal... the oldest and deepest lake on Earth which holds a fifth of the global non-salt water reserves; a must-see in my book. Quad tours at break-neck speeds, dry-suit diving with Russian regulators, walking barefoot in between and across drift ice that made its way onto the shorei, and extended hiking around the lake's coast... All of which I could not do because I was ill and had to spend two solid days in bed. The draft from the open window in between Novosibirsk and Irkutsk was enough to give me a rather bad cold which peaked at Lake Baikal. Still, the area was lovely and we were glad to be out of a train and able to unpack our stuff without having to repack immediately for once. I am not sure where my current losing streak with regards to diving is coming from (Grimsey, diving north of the Arctic circle with birds that plummet into the water and hunt fish: Only guy who does this is on the Icelandic mainland that day; Svalbard, diving north of the Arctic circle in permanent darkness: The few people who do this privately did not reply while I was there; Baikal, oldest, deepest, largest lake on Earth: ill), but I will most likely return to Russia for a week of ice diving in Lake Baikal next winter or the one after that. As an aside, I saw several people walking to Lake Baikal with buckets to get their water. Other people got it from a well which was still half frozen. If you have running water consider yourself lucky... Irkutsk Nice city, largely uneventful. The farther east you get within Russia, the more normal women look. In Moscow, just as in Paris, they are way over-dressed and even service personnel will walk with high heels. Thankfully, I don't have to wear heels, but for the other males out there: Walking and standing in these things hurts and thus most if not all people who stand and walk for a living have flat shoes. We happened upon preparations for a military parade, complete with cordon, viewing podests, at least half a dozen TV cameras etc, but were not sure if it would start soon enough for us to catch our train.We asked someone who told us it would start at 2100 local, at 1945 local it seemed about to start, and sure enough at 1955 sharp, the whole thing went under way. About a dozen groups of 50-100 people each, all in their own, respective uniforms stood against one side of a cordoned-off street and several higher-ups on the other side. Two highest-ups shouted into microphones and the throng of people on the other side shouted back answers. Then, the two highest-ups stood in the back of a jeep each and drove past said throng, stopping in front of each group, shouting into microphones mounted in the back of the jeeps and the groups shouted back once again. After that, all groups marched around the make-shift plaza once, saluting the higher ups. Once they were done, and they took ages, two trucks drove by with soldiers jumping out of the moving trucks and moving into crouching positions. They ran around in a circle a few times and engaged in pretend hand-to-hand combat. I am sure they are skilled at whatever style they wanted to show, but they were overdoing things so badly, they were funny, not imposing. When they jumped over some barriers, the barriers fell to pieces and everyone scrambled to make it look as if that was part of the show. While carrying off the gear, it fell into further pieces which was even more funny. An armoured personnel carrier ended the show; several tougher looking guys jumped off of that one and their mock combat involved fully automatic fire (of blanks), several flashbangs, smoke grenades and, to top things off, the machine gun mounted on the APC moving down the opposing team with blanks. I never witnessed a "real" military parade in person but this one was somewhat disappointing. On the one hand, there was a distinct lack of ballistic missile carriers and tanks like you see in movies, documentaries and games, on the other hand, the whole thing had a make-do feeling to it. The cordoning police had designated spots to stand on, yet walked around. They were standing to attention, yet checking their cell phones. Several people in one uniformed group were wearing track suits and jeans. Another uniformed guy had a grocery bag with him; yet another one was carrying a huge water bottle. Bikers zig-zagged through the cordon and when the whole show was just about to wrap up the police finally started putting up barriers around the unmoving pedestrians, not blocking the bikers. One little girl was standing well within the cordoned area, watching with big eyes and after she did not react to the police talking to her, they just built the barriers in a curve around her. And to top it all off, some guy with a cane walked all through the parade with his personal camcorder, trying to direct the whole show while being ignored by everyone. Still, I am sure he managed to mess up some otherwise perfectly good TV scenes. Irkutsk-Russian border

• Diesel-powered trains.
• Single track most of the time with frequent stops to let other trains pass.
• Distinctively less developed cities, stations, streets, and other infrastructure along the road.
• 32+ degrees in our waggon.
• The train attendant was extremely unfriendly and just generally miserable even by Russian standards.
  • No toilet paper or towels at all on toilets.
  • While the other attendants made a point of presenting themselves well, he shuffled around in slacks all the time (not bad per se, but Russia is big on uniforms, so...)
  • He took all our tickets and stubs (including the ones not from this part of the journey) and kept them without comment. After we asked for them several times, he barked at us that we would get them back before Ulan Bator. Why? No idea...
  • He refused to let us exit the train during the very few stops. We were unable to exit through other waggons as the connecting door was locked. Being stuck in a train sucks.
• Border and customs took NINE HOURS!!! Stuck in blistering heat without a breeze, without access to a toilet, just waiting for bureaucracy to go its way. I checked all doors, we were locked into said waggon and there were no 'break glass to leave in emergency' windows. Especially nice as there's a coal fire burning in the hot-water stove and the whole train is plastered with warning signs about fire and what to do. In our case, presumably, burn to death; preferably without disturbing the attendant.
• The Russian stamp for entering Russia (by plane) has a plane on it, the departure one a train.
• The Russian side of the border is built like a fortress. There are several towers and bridges over the rails so trains can be checked from above, and reinforced holes dug into the ground in which soldiers stand and check the train from below.

TL;DR 3000 kilometers of birch trees

I'll start here a small series of posts about ganeti, xen and puppet. For my work I run few servers sitting on xen and it has always been a bit of a pain to create a new instance and keep it up to date. Up to now I've used the excellent xen-create-image tool to create my VMs, but I wanted to try something new and more sexy... Last week I finally found some time to learn (and a spare box to run my experiments) how to use ganeti. Ganeti is the only tool I tried out, but it seems to fit the bill for my use and it seems polished and mature project to me... Moreover I've seen a presentation about it in every FLOSS conference I've attended in the last few years and I thought it was time to give it a try. Installing and configuring ganeti is fairly easy, there is a lot of documentation available and this post is not going to be about installing it, but rather how to create a new bare instance with ganeti-deboostrap-instance. There is also a way to create a new instance from an image, but I didn't go that way yet. This first post is about the first problem I've encountered, that is, how to automatically assign a network address and a name to each new instance created by gnt-instance add. Since all my instances should be able to communicate together on the same subnet, I've decided to configure xen to create a NATted private network and add each new instance to this network. The first step is to create an interface in /etc/network/interfaces . This is the standard debian way but since xen uses a different naming convention (here I'm using ganeti naming convention xenbr0 vs xen-br0), I need to convince tell xen what I intend to do by adding these lines in /etc/xen/xend.config : Next I have to connect my real network interface to the private network using few iptables rules in /etc/rc.local (probably there is a better place to do this...): The xen setup is complete and every new image should have a vif connected to the subbet 10.0.0.0. The xen setup corresponds to the physical wiring of the network. The next step is to configure each instance so to allow them to communicate on this subnet. Since I build my VMs using ganeti-debootstrap-instance, and by default debootstrap does not configure the network, we need to add a new hook in the directory /etc/ganeti/instance-debootstrap/hooks. This hook will do two things. First it will configure the interfaces of the new instance to get configured using dhcp, second, it will add an entry to the dnsmasq configuration file to make this instance known to the world. This basically boils down to add a file in /etc/dnsmasq.d/ with the mac address of the new instance and its designated name. Dnsmasq will then provide an ip address for this instance and add it to the dns. dhcp-host=aa:00:00:24:6c:8a,node1 Configuring dnsmasq is pretty easy as well. First I want it to answer dhcp queries only on the internal network, second I want to configure my clients passing 10.0.0.1 as nameserver and gataway. You can just add the following lines in /etc/dnsmasq.d/general to get it going. To create your new instance you can just run the following command : I use --no-ip-check and --no-name-check to skip ip and dns check performed by ganeti and to avoid a sort of chicken-egg problem, where the name and address of this new instance is yet unknown to dnsmasq and that node1 is the name that will be used by the hook to add an entry in the dnsmasq configuration. debootstrap+unstable is a variant of the default configuration and you need to add it to the list of variants used by ganeti-deboostrap-instance. That should be it. The new instance should come up with a dynamically assigned ip address, able to talk to the outside world and automatically known by all the other machine on the subnet via dns. Next post will be about how to add a swap hook for ganete-debootstrap-istance.

I'll start here a small series of posts about ganeti, xen and puppet. For my work I run few servers sitting on xen and it has always been a bit of a pain to create a new instance and keep it up to date. Up to now I've used the excellent xen-create-image tool to create my VMs, but I wanted to try something new and more sexy... Last week I finally found some time to learn (and a spare box to run my experiments) how to use ganeti. Ganeti is the only tool I tried out, but it seems to fit the bill for my use and it seems polished and mature project to me... Moreover I've seen a presentation about it in every FLOSS conference I've attended in the last few years and I thought it was time to give it a try. Installing and configuring ganeti is fairly easy, there is a lot of documentation available and this post is not going to be about installing it, but rather how to create a new bare instance with ganeti-deboostrap-instance. There is also a way to create a new instance from an image, but I didn't go that way yet. This first post is about the first problem I've encountered, that is, how to automatically assign a network address and a name to each new instance created by gnt-instance add. Since all my instances should be able to communicate together on the same subnet, I've decided to configure xen to create a NATted private network and add each new instance to this network. The first step is to create an interface in /etc/network/interfaces . This is the standard debian way but since xen uses a different naming convention (here I'm using ganeti naming convention xenbr0 vs xen-br0), I need to convince tell xen what I intend to do by adding these lines in /etc/xen/xend.config : Next I have to connect my real network interface to the private network using few iptables rules in /etc/rc.local (probably there is a better place to do this...): The xen setup is complete and every new image should have a vif connected to the subbet 10.0.0.0. The xen setup corresponds to the physical wiring of the network. The next step is to configure each instance so to allow them to communicate on this subnet. Since I build my VMs using ganeti-debootstrap-instance, and by default debootstrap does not configure the network, we need to add a new hook in the directory /etc/ganeti/instance-debootstrap/hooks. This hook will do two things. First it will configure the interfaces of the new instance to get configured using dhcp, second, it will add an entry to the dnsmasq configuration file to make this instance known to the world. This basically boils down to add a file in /etc/dnsmasq.d/ with the mac address of the new instance and its designated name. Dnsmasq will then provide an ip address for this instance and add it to the dns. dhcp-host=aa:00:00:24:6c:8a,node1 Configuring dnsmasq is pretty easy as well. First I want it to answer dhcp queries only on the internal network, second I want to configure my clients passing 10.0.0.1 as nameserver and gataway. You can just add the following lines in /etc/dnsmasq.d/general to get it going. To create your new instance you can just run the following command : If you are running your dom0 on debian squeeze before running this command you should configure ganeti to pass the right xen parameters to the newly created instance : I use --no-ip-check and --no-name-check to skip ip and dns check performed by ganeti and to avoid a sort of chicken-egg problem, where the name and address of this new instance is yet unknown to dnsmasq and that node1 is the name that will be used by the hook to add an entry in the dnsmasq configuration. debootstrap+unstable is a variant of the default configuration and you need to add it to the list of variants used by ganeti-deboostrap-instance. That should be it. The new instance should come up with a dynamically assigned ip address, able to talk to the outside world and automatically known by all the other machine on the subnet via dns. Next post will be about how to add a swap hook for ganete-debootstrap-istance.

Firstly, please excuse the ugly formatting of this ultra-wide image on my blog, and I m sure the respective planets that I m on. po to is my first real node project, and I m planning to get it to a point where it makes email less painful then it otherwise is. The code s on my github, and I d like to thank Rick Waldron for his work in helping me, as well as Scott Wells for the name. For those wondering, po to is an Esperanto word meaning (according to wiktionary):

1. a public institution (usually government-run) to deliver mail, post
2. mail distribution in general
3. one s mail, collectively

The front-end s name is po tilo the suffix denotes tool such that po tilo means something like mail tool fitting! More to come

I have a Seagate GoFlex Net with two 2TB harddrives attached to it via SATA. The device itself is connected to my PC via its Gigabit Ethernet connection. It houses a Marvell Kirkwood at 1.2GHz and 128MB. I am booting Debian from a USB stick connected to its USB 2.0 port. The specs are pretty neat so I planned it as my NAS with 4TB of storage being attached to it. The most common use case is the transfer of big files (1-10 GB) between my laptop and the device. Now what are the common ways to achieve this? scp:

scp /local/path user@goflex:/remote/path

rsync:

rsync -Ph /local/path user@goflex:/remote/path

sshfs:

sshfs -o user@goflex:/remote/path /mnt
cp /local/path /mnt

ssh:

ssh user@goflex "cat > /remote/path" < /local/path

I then did some benchmarks to see how they perform: scp: 5.90 MB/s rsync: 5.16 MB/s sshfs: 5.05 MB/s ssh: 5.42 MB/s Since they all use ssh for transmission, the similarity of the result does not come as a surprise and 5.90 MB/s are also not too shabby for a plain scp. It means that I can transfer 1 GB in a bit under three minutes. I could live with that. Even for 10 GB files I would only have to wait for half an hour which is mostly okay since it is mostly known well in advance that a file is needed. But lets see if we can somehow get faster than this. Lets analyze where the bottleneck is. Lets have a look at the effective TCP transfer rate with netcat:

ssh user@goflex "netcat -l -p 8000 > /dev/null"
dd if=/dev/zero bs=10M count=1000   netcat goflex 8000

79.3 MB/s wow! Can we get more? Lets try increasing the buffer size on both ends. This can be done using nc6 with the -x argument on both sides.

ssh user@goflex "netcat -x -l -p 8000 > /dev/null"
dd if=/dev/zero bs=10M count=1000   netcat -x gloflex 8000

103 MB/s okay this is definitely NOT the bottleneck here. Lets see how fast I can read from my harddrive:

hdparm -tT /dev/sda

114.86 MB/s.. hmm... and writing to it?

ssh user@goflex "time sh -c 'dd if=/dev/zero of=/remote/path bs=10M count=100; sync'"

42.93 MB/s Those values are far faster than my puny 5.90 MB/s I get with scp. A look at the CPU usage during transfer shows, that the ssh process is at 100% CPU usage the whole time. It seems the bottleneck was found to be ssh and the encryption/decryption involved. I'm transferring directly from my laptop to the device. Not even a switch is in the middle so encryption seems to be quite pointless here. Even authentication doesnt seem to be necessary in this setup. So how to make the transfer unencrypted? The ssh protocol specifies a null cipher for not-encrypted connections. OpenSSH doesnt support this. Supposedly, adding

  "none", SSH_CIPHER_NONE, 8, 0, 0, EVP_enc_null  

to cipher.c adds a null cipher but I didnt want to patch around in my installation. So lets see how a plain netcat performs.

ssh user@goflex "netcat -l -p 8000 > /remote/path"
netcat goflex 8000 < /local/path

32.9 MB/s This is far better! Lets try a bigger buffer:

ssh user@goflex "netcat -x -l -p 8000 > /remote/path"
netcat -x goflex 8000 < /local/path

37.8 MB/s now this is far better! My Gigabyte will now take under half a minute and my 10 GB file under five minutes. But it is tedious to copy multiple files or even a whole directory structure with netcat. There are far better tools for this. An obvious candidate that doesnt encrypt is rsync when being used with the rsync protocol.

rsync -Ph /local/path user@goflex::module/remote/path

30.96 MB/s which is already much better! I used the following line to have the rsync daemon being started by inetd:

rsync stream tcp nowait root /usr/bin/rsync rsyncd --daemon

But it is slower than pure netcat. If we want directory trees, then how about netcatting a tarball?

ssh user@goflex "netcat -x -l -p 8000   tar -C /remote/path -x"
tar -c /local/path   netcat goflex 8000

26.2 MB/s so tar seems to add quite the overhead. How about ftp then? For this test I installed vsftpd and achieved a speed of 30.13 MB/s. This compares well with rsync. I also tried out nfs. Not surprisingly, its transfer rate is up in par with rsync and ftp at 31.5 MB/s. So what did I learn? Lets make a table: </dr></dr></dr></dr></dr></dr></dr></dr></dr></dr>

method	speed in MB/s
scp	5.90
rsync+ssh	5.16
sshfs	5.05
ssh	5.42
netcat	32.9
netcat -x	37.8
netcat -x tar	26.2
rsync	30.96
ftp	30.13
nfs	31.5

For transfer of a directory structure or many small files, unencrypted rsync seems the way to go. It outperforms a copy over ssh more than five-fold. When the convenience of having the remote data mounted locally is needed, nfs outperforms sshfs at speeds similar to rsync and ftp. As rsync and nfs already provide good performance, I didnt look into a more convenient solution using ftp. My policy will now be to use rsync for partial file transfers and mount my remote files with nfs. For transfer of one huge file, netcat is faster. Especially with increased buffer sizes it is a quarter faster than without. But copying a file with netcat is tedious and hence I wrote a script that simplifies the whole remote-login, listen, send process to one command. First argument is the local file, second argument is the remote name and path just as in scp.

#!/bin/sh -e
HOST=$ 2%%:* 
USER=$ HOST%%@* 
if [ "$HOST" = "$2" -o "$USER" = "$HOST" ]; then
        echo "second argument is not of form user@host:path" >&2
        exit 1
fi
HOST=$ HOST#*@ 
LPATH=$1
LNAME= basename "$1" 
RPATH= printf %q $ 2#*: /$LNAME 
ssh "$USER@$HOST" "nc6 -x -l -p 8000 > $RPATH" &
sleep 1.5
pv "$LPATH"   nc6 -x "$HOST" 8000
wait $!
ssh "$USER@$HOST" "md5sum $RPATH" &
md5sum "$LPATH"
wait $!

I use pv to get a status of the transfer on my local machine and ssh to login to the remote machine and start netcat in listening mode. After the transfer I check the md5sum to be sure that everything went fine. This step can of course be left out but during testing it was useful. Escaping of the arguments is done with printf %q. Problems with the above are the sleep, which can not be avoided but must be there to give the remote some time to start netcat and listen. This is unclean. A next problem with the above is, that one has to specify a username. Another is, that in scp, one has to double-escape the argument while above this is not necessary. The host that it netcats to is the same as the host it ssh's to. This is not necessarily the case as one can specify an alias in ~/.ssh/config. Last but not least this only transfers from the local machine to the remote host. Doing it the other way round is of course possible in the same manner but then one must be able to tell how the local machine is reachable for the remote host. Due to all those inconveniences I decided not to expand on the above script. Plus, rsync and nfs seem to perform well enough for day to day use.

tldr: counting without cumulative timing errors Sometimes I want just a small counter, incrementing an integer each second running somewhere in a terminal. Maybe it is because my wristwatch is in the bathroom or because I want to do more rewarding things than counting seconds manually. Maybe I want not only to know how long something takes but also for how long it already ran in the middle of its execution? There are many reason why I would want some script that does nothing else than simply counting upward or downward with some specific frequency. Some bonuses:

• the period should be possible to give as a floating point number and especially periods of a fraction of a second would be nice
• it should be able to execute an arbitrary program after each period
• it should not matter how long the execution of this program takes for the overall counting

Now this can not be hard, right? One would probably write this line and be done with it:

while sleep 1; do echo $i; i=$((i+1)); done

or to count for a certain number of steps:

for i in  seq 1 100 ; do echo $i; sleep 1; done

This would roughly do the job but in each iteration some small offset would be added and though small, this offset would quickly accumulate. Sure that cumulative error is tiny but given that this task seems to be so damn trivial I couldn't bear anymore with running any of the above but started looking into a solution. Sure I could just quickly hack a small C script that would check gettimeofday(2) at each iteration and would adjust the time to usleep(3) accordinly but there HAD to be people before me with the same problem who already came up with a solution. And there was! The solution is the sleepenh(1) program which, when given the timestamp of its last invocation and the sleep time in floating point seconds, will sleep for just the right amount to keep the overall frequency stable. The author suggests, that sleepenh is to be used in shell scripts that need to repeat an action in a regular time interval and that is just what I did. The result is trivial and simple but does just what I want:

• the interval will stay the same on average and the counter will not "fall behind"
• count upward or downward
• specify interval length as a floating point number of seconds including fractions of one second
• begin to count at given integer and count for a specific number of times or until infinity
• execute a program at every step, optionally by forking it from the script for programs possibly running longer than the given interval

You can check it out and read how to use and what to do with it on github: https://github.com/josch/periodic Now lets compare the periodic script with the second example from above:

$ time sh -c 'for i in  seq 1 1000 ; do echo $i; sleep 1; done'
0.08s user 0.12s system 0% cpu 16:41.55 total

So after only 1000 iterations, the counter is already off by 1.55 seconds. This means that instead of having run with a frequency of 1.0 Hz, the actual frequency was 1.00155 Hz. Is it too much to not want this 0.155% of error?

$ time ./periodic -c 1000
0.32s user 0.00s system 0% cpu 16:40.00 total

1000 iterations took exactly 1000 seconds. Cool.

Just set up Synology Audio Station for music playback at home. I originally thought about some more open solution, but this was the easiest choice which might still change. Main motivation was that I got really cheaply Synology Remote, what is wireless remote and USB sound card and I was too lazy to set it up with something else. It works quite nicely but now I'd like to control it from my N950 phone. Synology provide app for iPhone and Android, but others are left in the dark (there is also no documentation about the protocol). The options for me are to write native client or emulate some well know protocol, where I could use existing clients. Quick googling have shown me python-mpd-server, what looks exactly as half of program I need, so I would have to implement only bindings to (not documented) Audio Station protocol (there is open-syno, which seems to implement this protocol, so at least there will be some hints).

Filed under: English Life Linux 0 comments Flattr this!

I'm blogging about this small configuration issue as it took me some time to figure out how to configure cupt and smart to solve this problem. The reason I'm playing with cupt and smartpm is that I'm working to compare again a number of package managers in debian against the state of the art cudf solvers using mpm, and I'm suffering quite a bit to configure my virtual environment. Last year I promised to revise and fix our results. I didn't forget my promise, but it seems it took longer then expected. Anyway, back to the main topic. The release-problem arises because the key used to sign sarge (that I'm using as baseline for my experiments) is long expired. If you try to retrieve sarge from archive.debian.org you will find that sarge is signed with the key A70DAF536070D3A1 and apt-get will complain loudly if you try to use an archive signed with an expired key.

cupt For cupt this is documented in the man page and there are a number of options to add either to /etc/apt/apt.conf to the the cupt own conf file. Then cupt will happily accept the sarge Packages file and let you run update.

cupt::cache::release-file-expiration::ignore "true";
cupt::update::check-release-files "false";
cupt::update::keep-bad-signatures "true";

smartpm For smart I could not found this information anywhere, but reading the source code (tnx good it's python !). To cut it short, you need to set the key trustdb to an empty value for a specific channel. On the command line you get something like : smart channel --set aptsync-614482cb2c7e08d5722af3498232ba52 keyring= --config-file=/root/var/lib/smart/config where aptsync-614482cb2c7e08d5722af3498232ba52 is the channel name corresponding to sarge in my conf. Since I'm using a simulated environment, I save the result of this option in a non-default config file in my chroot.

The bug webinterface of the Ultimate Debian Database currently knows about the following release critical bugs:

In Total:	1515
Affecting Wheezy:	1078
Wheezy only:	235
Remaining to be fixed in Wheezy:	843

Of these 843 bugs, the following tags are set:

Pending in Wheezy:	70
Patched in Wheezy:	169
Duplicates in Wheezy:	62
Can be fixed in a security Update:	23
Contrib or non-free in Wheezy:	12
Claimed in Wheezy:	2
Delayed in Wheezy:	12
Otherwise fixed in Wheezy:	71

Ignoring all the above (multiple tags possible) 504 bugs need to be fixed by Debian Contributors to get Debian 7.0 Wheezy released. However, with the view of the Release Managers, 968 need to be dealt with for the release to happen. Please see Interpreting the release critical bug statistics for an explanation of the different numbers.

In january, I discovered the ColorHug, a USB dongle from Hughski to calibrate the color on a computer screen. The software required is included in Debian, and I decided back then to preorder from the next batch. Yesterday I finally heard back from them, and got the opportunity to order. Today I ordered mine, and eagerly await the delivery. I hope it arrive next week, as I got a confirmation that it should go in the mail on monday. :) If you want to ensure the colors on the screen match the intended colors, I suggest you check out this cheap tool with free software drivers. :)

At the recent Ubuntu Developer Summit, I managed to convince a few people (after assurances that there would be no permanent damage) to plug a USB stick into their machines so we could watch Xorg crash and wedge their console. What was this evil thing, you ask? It was an AVR microprocessor connected to USB, acting as a USB HID Keyboard, with the product name set to %n . Recently a Chrome OS developer discovered that renaming his Bluetooth Keyboard to %n would crash Xorg. The flaw was in the logging stack, triggering glibc to abort the process due to format string protections. At first glance, it looks like this isn t a big deal since one would have to have already done a Bluetooth pairing with the keyboard, but it would be a problem for any input device, not just Bluetooth. I wanted to see this in action for a normal (USB) keyboard. I borrowed a Maximus USB AVR from a friend, and then ultimately bought a Minimus. It will let you put anything you want on the USB bus. I added a rule for it to udev:

SUBSYSTEM=="usb", ACTION=="add", ATTR idVendor =="03eb", ATTR idProduct =="*", GROUP="plugdev"

installed the AVR tools:

sudo apt-get install dfu-programmer gcc-avr avr-libc

and pulled down the excellent LUFA USB tree:

git clone git://github.com/abcminiuser/lufa-lib.git

After applying a patch to the LUFA USB keyboard demo, I had my handy USB-AVR-as-Keyboard stick ready to crash Xorg:

-       .VendorID               = 0x03EB,
-       .ProductID              = 0x2042,
+       .VendorID               = 0x045e,
+       .ProductID              = 0x000b,
...
-       .UnicodeString          = L"LUFA Keyboard Demo"
+       .UnicodeString          = L"Keyboard (%n%n%n%n)"

In fact, it was so successfully that after I got the code right and programmed it, Xorg immediately crashed on my development machine. :)

make dfu

After a reboot, I switched it back to programming mode by pressing and holding the H button, press/releasing the R button, and releasing H . The fix to Xorg is winding its way through upstream, and should land in your distros soon. In the meantime, you can disable your external USB ports, as Marc Deslauriers demonstrated for me:

echo "0" > /sys/bus/usb/devices/usb1/authorized
echo "0" > /sys/bus/usb/devices/usb1/authorized_default

Be careful of shared internal/external ports, and having two buses on one port, etc.

2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

A new release 0.2.4 of RProtoBuf is now on CRAN. RProtoBuf provides GNU R bindings for the Google Protobuf data encoding library used and released by Google. This release once again contains a number of patches kindly contributed by Murray Stokely, as well as an added header file needed to build with the g++ 4.7 version which has become the build standard on CRAN. The NEWS file entry follows below:

0.2.4   2012-05-15
    o   Applied several patches kindly supplied by Murray Stokely to
         - properly work with repeated strings 
         - correct C++ function naming in a few instances
         - add an example of ascii export/import of messages
    o   Suppport g++-4.7 and stricter #include file checking by adding unistd
    o   Made small improvements to the startup code

CRANberries also provides a diff to the previous release 0.2.3. More information is at the RProtoBuf page which has a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. Questions, comments etc should go to the rprotobuf mailing list off the RProtoBuf page at R-Forge. Updated to show NEWS rather than ChangeLog

Way before Debian, Launchpad bug tracking system just reached 1 million bugs reported with one "bug" reported against Edubuntu basically mentioning it should invade schools. What to say about this? Hard without being harsh towards my friends working in the Ubuntu "world", indeed. Still, I really think that here, too much noise kills signal and the LP BTS is often hardly usable. I counted up to 217 bugs reported against samba4 (which is, after all, not so widely used yet) just because it apparently has upgrading issues between pre 12.04 versions of Ubuntu and Oneiric. It indeed seems that some automated bug reporting is now active and whenever a user encouters an upgrade issue with a package, a bug is being reported. I guess this is somehow an opt-in system (I hope so..:-)) but the default is very clearly using it. This feature is apparently what caused the recent bump in number of bugs reported in LP, making them even less useful, particularly to Debian package maintainers. I'm sure there are tools to help dealing with that and I was already answered that work is in progress to change this (and use a dedicated website for such reports or something like this). But, still, that seems to be the scary side of popularity...the very same popularity that is slowly but constantly hiding the work we're doing in Debian to indirectly make Ubuntu popular. (moving to more general concerns) I know that things are not all black or all white, but it always saddens me to feel that slowly....but, again, constantly, more and more people tend to forget that Debian is behind Ubuntu, is the ground on which it is built and Ubuntu wouldn't exist without it. When doing work, a human need is to get reward for it...and we are getting less of it...slowly, but constantly. Don't take me wrong. I have many friends working directly for Ubuntu. Some paid by Canonical for this. Some really involved up to "top level" (yes, including the very very top level even if I killed him once). I don't want to throw offense on them. I don't even know if they can do something about what I'm expressing below. I would just have them (and others) know. Let's take an example. I recently activated a few languages in D-I (Burmese, Tibetan, Uyghur). I'm happy with that, this is something I'm doing for 8 years now. But all these new translators were indeed only interested in one thing : "have Ubuntu translated in their language". No offense intended, but they didn't really care about *Debian* being translated in their language. I think that some didn't even know what Debian is. In the same field, I am more and more "fighting" to keep the level of translation completeness in Debian (see my regular spa^W reports). In some way, I still succeed, but the price to pay is more and more and more personal investment and work. That's still working for the strong set of languages we support. That works much less for most others. When someone "disappears" (or just switches to some other priorities), it's more and more difficult to find someone else popping up. And, for the "strong set", something else is happening : work duplication. There are "strong" French, German, Italian, whatever, l10n teams in Debian.....and there are similar teams for Ubuntu. And, mostly, those do not really work together. And sometimes, this is kinda discouraging. So, seeing the explosion happening on what is, whatever we think or write, the "other side", is not somethnig that can make one entirely happy. And this is why I won't celebrate Launchpad's millionth bug report. Particularly when I see that millionth bug report not even ack'ing that this Edubuntu marvel is based on the grounds set by some pionneers many years ago in a few schools in Norway (hello, Petter and others). Yeah, sometimes sad. To balance this, let's release wheezy and have millions of people benefit from it without even knowing.

I ve been noticing a number of odd things happening surrounding my blog lately, and I thought it s about time to figure out what s going on and how to stop it. The first problem is that people are illegally copying my posts, probably using RSS scraping, and putting them up on their own ad-infested sites. It is trivial to find them using Google for any somewhat unique word or phrase in one of my posts. Lately one of them, linux-support.com, actually sends me pingbacks announcing the fact that they ve scraped me! Most of these sites seem to be nothing but content farms for selling ad impressions, and almost none of them have any identifiable names for the owners. (There is an exception: I have specifically set up sites like Planet Debian and Goodreads to copy my blog posts.) I m obviously an advocate of open content, but I do not feel it right that others should be profiting by putting photos and stories about Free Software, or photos of my family, on their ad farms. While I release a great deal of content under GPL or Creative Commons licenses, I have never done so with my blog an intentional decision. What should I do about this? Is it worth fighting a battle over, or is it about as useless as trying to block every spam follower on my twitter account? So that s the first weird thing. The second weird thing just started within the last few weeks. I have been getting a surprising amount (a few a week) of email addressed to me. It does not bear the appearance of being 100% automated spam, though it is possible that it is. It s taken a few forms:

• Someone wanting to buy an ad on my blog
• Someone wanting to send me a story hyping their product (and intending me to pretend that I wrote the story)
• Someone wanting me to write a story about their website and link to it

The profit motive in all of these is high, and in at least the second and third, so is the sleaze factor. I ve gotten two emails lately of this form:

Hi John, I am curious if you are the administrator for this site: changelog.complete.org/archives/174-house-outlaws-fast-forwarding-senate-pres-next I am a researcher / writer involved with a new project whose mission it is to provide accurate and useful information for those interested in the practice of law, whether as a lawyer or paralegal. I recently produced an article detailing the complex relationship between law and technology and the legal implications on personal privacy and free speech. I would love to share this resource with those who might find it useful and am curious of you are the correct person to contact about such a request? Thank you! All my best,

The details vary the URLs appear to be random (the one cited above was little more than a link to an article), the topics the website claims to discuss range from law to schizophrenia (that one actually came with a link to the site, which again seemed to be a content farm). I am slightly tempted to reply to one of these and ask where the heck people are getting my name. It seems as if somebody has put me into a mailing list they sell containing sleazebag bloggers. Frankly, I am puzzled at this attention. I guess I haven t checked, but I can t imagine that my blog has anything even remotely resembling a high PageRank or anything else. It s not high-traffic, not Slashdot, etc. Either people are desperate, naive, failing to be selective, or maybe working some scam on me that I don t know yet. In any case, I m interested if others have seen this, or any advice you might have.

It's not too obvious how to test email receive handlers in Appengine. The important observation is that the handlers take HTTP POSTs with multipart/form-data encoded data. In python you can build an email to be handled with the following code:

from email.message import Message 

def test_email(self):
  body = Message()
  body.add_header('to', 'test-unknown@other-app.com')
  body.add_header('from', 'test@app.com')
  body.add_header('Content-Type', 'multipart/alternative', boundary=self.boundary)
  text = Message()
  text['content-type'] = 'text/plain'
  text.set_payload('I am I! Don Quixote!  The man of La Mancha!')
  body.attach(text)

  post(payload=body.as_string())

This year, the annual Debian conference will be held in Managua, Nicaragua. And I'll be lucky enough to spend two weeks visiting the country after Debconf, along with Elisabeth. Yes, I'll arrive in Nicarague on July 2nd, spend nearly the entire Debcamp, then Debconf, then we'll spend 16 days around the western part of Nicaragua, trying to discover the magic of this country. So, this post is about sharing our plans with my readers. Of course, I do not know the country so we may have made mistakes and bad choices. We'll see. Immediately after Debconf, Elisabeth will join in Managua. She'll be landing on July 15th. We'll then spend a night in an hotel near the airport and immediately leave the day after for Matagalpa, in North Nicaragua. We rented a car for the entire trip indeed, and will be on our own on wild Nicaragua roads..:-) We'll spend two nights in Matagalpa. We plan to visit some coffee or cigar plants, probably have a trip to Lake Apanas and Jinotega. Then, we'll have a short road trip to Esteli where we spend again two nights. We'll be visiting a coffee growing place (beneficio seco de caf ). A full day visit is planned at Miraflor natural reserve to enjoy te beauties of hundreds orchids and some local natural marvels. The next move will be to Leon, where we'll spend 4 nights, visiting a cigar factory (tabacaleras de puros?) on the way, as well as San Jacinto, a place with hydrothermal sources and "Hervideros" (geysers). Four nights in Leon leaves plenty of time for several activities *and* enjoying the colonial city. We'll have a full day at Juan Venado Island reserve with boat trip from Las Penitas (on a fisherman's boat from what the travel agency mentioned), then another full day climbing on the Cerro Negro volcano. Indeed, I was originally considering climbing the Momotombo, but our travel agency warned about the high difficulty. I would have loved that myself but maybe not the two of us...and this is a trip for both of us! So, we played the safe option..:-) After these 4 nights in Leon, we'll move to Granada for 3 nights, through Leon Viejo (the former site of Leon). From Granada, one day will there be used for a visit in the Masaya National Park and see the beauties of Masaya volcano (this is indeed something that could be done for Debconf day trip, IMHO, as it doesn't seem that far from Managua). Another day will be spent to Las Isletas on lake Nicaragua and others visiting the colonial city of Granada. Or, of course, whatever things we don't even known about now..:-) Then we'll move to what I personnally consider the peak of the trip: 3 nights on Ometepe island on lake Nicaragua. Just check Wikipedia to see why Ometepe is, in my opinion, THE place to go in Nicaragua. Here, I'll have my volcano..:-). Indeed, Elizabeth "authorized" me to book a local guide and then climb Concepcion Volcan, if the weather allows for it. 1600m height, that doesn' seem to be a big issue....except when starting from a little bit above sea level and are climbing a volcano that looks like s postcard volcano : nearly a perfect cone shape. So, let's cross fingers for having good weather that day. I promise myself I'll record the GPS track of that one and, even if I'll probably be walking most of the climbing (except if I have a very trained guide...), I'll add it to my run tracks! We might also be going to climb Ometepe's other volcano (Maderas) the day after so that Elizabeth also enjoys these beauties. There also seems to be great places around Maderas such as San Ramon Cascade, Finca el Porvenir, etc. Then, at the end of all this, it will be time to come back to Managua in the final day and fly back to Paris in the early morning of July 31st. All over, I'll be in Nicaragua from July 2nd until July 31st! Full month away, yay! Hurrah for the crazy number of holidays those lazy French people have..:-) During this trip, we might find it interesting some local geeks (not too many as Elizabeth is not that deeply interested in beersigning!) and share a few nice things in local places which are only known by locals. In case you're interested, out (very clever) travel agency is named Nicaragua Adventures and they're definitely worth contacting if you want to travel around .ni, particularly if we prefer booking things in advance as we do. They speak Spanish (of course!), English and French. They're very responsive to e-mail as well.

CRAN now tests packages against g++-4.7 (as this version has become the default on Debian's testing variant. This compiler switch once again triggered a set of build failures, mostly from include files now deemed missing. For RcppSMC, it came down to a five-character patch of explicitly stating one max() call as std::max() No other changes were made at this point. The NEWS entry is below:

0.1.1   2012-05-14
    o   Version 0.1.1 
    o   Minor g++-4.7 build fix of using std::max() explicitly

Courtesy of CRANberries, there is also a diffstat report for 0.1.1 relative to 0.1.0 As always, more detailed information is on the RcppSMC page,

The first Beta of the upcoming PostgreSQL 9.2 was released yesterday (see announcement). Your humble maintainer has now created packages for you to test. Please give them a whirl, and report any problems/regressions that you may see to the PostgreSQL developers, so that we can have a rock solid 9.2 release. Remember, with the postgresql-common infrastructure you can use pg_upgradecluster to create a 9.2 cluster from your existing 8.4/9.1 cluster and run them both in parallel without endangering your data. For Debian the package is currently waiting in the NEW queue, I expect them to go into experimental in a day or two. For Ubuntu 12.04 LTS you can get packages from my usual PostgreSQL backports PPA. Note that you need at least postgresql-common version 0.130, which is available in Debian unstable and the PPA now. I (or rather, the postgresql-common test suite) found one regression: Upgrades do not keep the current value of sequences, but reset them to their default value. I reported this upstream and will provide updated packages as soon as this is fixed.

You can get inspired from the most unlikely of places, and this time it was a twitter background image. I came across the twitter profile of @Nitish today, a fellow GSoC student from India. One thing that held my attention, apart from the fact that he tweets crazy, was the background picture in his profile. Yes I had not seen something like that before and I absolutely loved the concept. The image was large grid and each tile carried the profile picture of one of his twitter followers. It was beautiful. The background image was generated by twilk, a very popular application that I had never heard of. I wanted to try it, but I figured my follower count might be a little too low for the application to work. Twilk works only with twitter and I wanted it for Facebook. Instead of looking for something similar, I decided to write it myself. I had worked with the Python Imaging Library some time back and knew it would not be very difficult task. The first thing I needed was the profile pic thumbnails of my friends. Facebook API supports a call which returns a small 50x50 pixel profile pic of a user, if we send a valid user ID. I wrote a quick script that returned the user ID s of my friends on Facebook as JSON data. I grepped the user ID s from the JSON and wrote a small wget script to download the profile pics of all my friends to a directory. Once the download was in progress, I started my work with the imaging library. I knew the image count beforehand and after some quick math, I fixed the dimensions of the output image that I wanted. Once the dimensions were fixed, the work was fairly easy. I knew before hand how many images will be there in each row and how many rows will be there in total. All I had to do was to randomly pick one of the images from the directory, and arrange it on the output image canvas. Few lines of code later I get the result that I wanted and it is on my twitter profile. Yay! :) I have uploaded the python code in here. If you are running it, make sure you change the values according to your requirements. The image below is a cropped out from the output image to fit this div. Click on it to see the actual output. When I showed this to @Jaseeemabid, he asked me to put this up on Google App Engine and start a service so that even non geeks could use this. I liked the idea but I am slightly busy(lazy) now. I ll do it when I am free or when I see that there is some demand for this! :) Update As few Redditors pointed out, this may not be the best way to do it. This was just a quick hack to get something done. The JSON could be processed in Python itself and files can be downloaded as well. I modified an old PHP script to get the JSON data and since the structure was relatively simple, I grepped the relevant information and fed it to wget as I prefer wget over any other tool for mass downloads. Update 2 Redditor Bio Lazarus improved upon this and made an end to end version that does all of the above in one single python script and the code is available here. And if you like this post, you should follow me on twitter :)