We had a great weekend this last weekend here in Montreal. Yesterday, June 24, was Saint-Jean-Baptiste Day here in Quebec -- the national holiday of this cool and fun nation. St. Jean Baptiste is important to me in a lot of ways. Of course, it's cool that it's the national holiday of our adopted new home. But John the Baptist, the patron saint of Quebec, is also our family's name saint. ''Prodromos'' means "forerunner" in Greek, and it's one of the titles of St. John. Most of all, it's the day that Maj and I moved to Montreal. We came in on St. Jean Baptiste Day, 2003, and we've been here ever since. We drove my beautiful 1972 Citro n DS from San Francisco, where my niece Elena had just been born, all the way out here. We arrives on June 24th and wandered around the McGill Ghetto looking for our sight-unseen sublet. It's hard to believe it's only been 4 years -- it seems like so much longer. We had a good day yesterday. The three of us went to Piknic in the afternoon and hung out with our friends Jeremy and Lina and their two great kids Clea and Sylvie. We had a lot of hummus and pasta salad and bread and cheese and generally had a great time. Meg and Catherine came and hung out for a while, and also our friend Kristen, who's 7 months pregnant and has a 10-month-old. Good luck on that one. Most of all a pretty day enjoying the outdoors in our new home town, as well as fun dancing to house under a big Calder statue. How great is that? tags: montreal quebec stjeanbaptiste prodromou piknic jeremy lina

72 hours from Flickr I forgot to mention that we had a really nice time on Friday evening going to the 24 Hours of Flickr party at the SAT. Heather Champ, community leader at Flickr and welcomer to partiers, put it well: "We have three requirements for Flickr parties: the food must be good, the drinks must be copious, and the schwag must be ample." And that's exactly what happened. All the usual Montreal suspects were out in force: YULblog types, Web entrepreneurs, Web pranksters. It was great seeing m-c and c-c, who wore their awesome Flickr Socks -- although they each had a matching pair. I also talked to Allen Morris, which was really fun. He's a developer living now in Saranac Lake, one of the towns in the Adirondacks I find strangely compelling. We exchanged GPG key signatures and talked a lot about the Bay Area (California), where we both once lived. Probably my best talk was with Heather Champ and Derek Powazek. We talked about their joys of creation of JPG Magazine, the despair of leaving it, and looking forward to the future. We also chatted a lot about print on demand technologies, which we're evaluating for Wikitravel. Heather took the photo of my friend Ben Cerveny at Burning Man that's become his avatar icon on about every social software platform I know of. It must be good to capture someone in the way they want to think of themselves. I took my own self-concept picture by putting the camera on the dashboard of my Citroen. Heh. All in all a good party. One thing I was interested to find out was that there's a Yahoo! Quebec, which launched last month and I guess just slipped by me. Sadly, if you click on the "English" button, it takes you to... Yahoo! Canada. Hey, Yahoo! There are actually a lot of anglophones here! Just in case you were wondering. tags: yahooquebec 24heuresflickrmontreal 24hoursflickrmontreal flickr montreal party

Montreal Web Development Book Club Montreal Web scene stalwart Heri Rakotomalala -- the man with the magic name, originally from Madagascar, along with Montevideo and Marfa, one of the M-places on my travel wishlist -- recently started a Montreal Web Development Book Club. It's a meeting for people interested in the practical side of the Web to talk about the books they read and let that drift into talk about the work we do and ideas we have. I think the book club is an awesome idea. Book clubs are a great grass-roots intellectual meetup that we don't often celebrate in the post-flash mob BarCamp-ized world. The great thing about a book club, for Web people, is that it filters out the bullshitters and dilettantes. If you're going to a book club, you're going to have to know what you're talking about, and you're going to have to care about the subject. The book gives focus that keeps the conversation from drifting to tangential topics. Flakes and hangers-on just don't have the stamina for it. Also, we met at La ka, so it was nice to have some beers in a good place on a hot night. Why not? We managed to talk about a lot of good books, too. Jean-Fran ois <something> (I didn't get his last name) talked about In Search of Stupidity, a look at the last 25 years of marketing and business blunders in the high-tech realm. Our industry is subject to a lot of decisions that seem absolutely self-defeating in hindsight, and the book goes in-depth on 15 case studies -- like Wordstar, Ashton-Tate and Borland. It sounds like a great cautionary tale for Web 2.0 startup mavens. Heri talked about the book Information Architecture for the World Wide Web -- the infamous "Polar Bear Book". It sounds like a really good book for doing initial planning for a Web site, but I'm really interested in tools for profiling an IA effort. We get about 2 million unique visitors a month at Wikitravel, for example, which is a huge amount of usability data I could be mining. But I don't really have the tools for it, so I don't do it much. I'd love to know, for example, about people who land on a Wikitravel page from a search engine and then leave. What were they looking for? Why couldn't they find it? Also, users who browse through our pages clicking on links, then resort to a full-text search. What did they think they were clicking on, and what were they looking for instead? Did they find it? How can we put that in front of them in the first place? Most of all, I want to know how and why people start editing Wikitravel pages, and why they stop. Of people who click the "edit" button, only 1 in 17 click "save". (That is one bit of clickpath analysis I've done.) Why? I've often been tempted to put a survey up when someone navigates away from our edit page. "Why didn't you save? Tell us so we can fix it for other people." I talked about the book I just finished reading a few weeks ago, Everything is Miscellaneous. It's the latest book by Dave Weinberger about how we impose order on information to make it easy to understand and retrieve. It's an awesome book -- really fun -- especially for someone married to a knowledge management expert. I don't think I gave a good enough description of the book to ignite the club's interest, but I hope they check it out, because it's really readable and good. At the end of the meeting Heri took a poll and decided that we'll continue doing the book club on the last Monday of the month. I've already picked out my book for next month: RESTful Web Services. I've heard good things about this book (admittedly mostly from the co-author Leonard Richardson) and I want to give it a look-see. So book-club fodder it is. ''Update: Heri has his own book club report here.'' tags: bookclub montreal heri webdev montrealwebdevbookclub

And so DebConf7 is over.. And as always it was quite an experience.
In the two weeks I’ve pushed my alcohol limit to a new level, invaded Edinburgh castle at 5AM, mounted a plastic cow, had pancakes at the Scott monument also at roughly 5AM, roamed the city, seen the awful clubs UK has to offer, tasted some wonderful beers and whiskies, survived the food, met interesting people, fucked up a talk, driven all the way from Edinburgh to Norwich, and here I am. With a wonderful set of videos and photos.
Thanks to everyone who attended DebConf and organized this wonderful event for making it yet an experience I couldn’t dream of being without.
SEE YOU ALL IN ARGENTINA!!

A few months ago, the Washington Post featured an article about Joshua Bell playing his violin (which happens to be a Stradivarius) in a Washington metro station. Contrary to their expectations, nobody stopped by to listen. The article tries to find reasons for why this happened, digging into theories by a number of famous philosophers. While those are undoubtedly not far from the truth, let me dig up another theory: Joshua Bell may be one of the best musicians in the world; that doesn't mean he's one of the best street musicians in the world. You see, if you play your Stradivarius in a concert hall, you don't have to do much to get people's attention. If your name is Joshua Bell and you play in the New York Opera House, people pay good money to get a seat months in advance, they circle the date on their calendar, and they look forward to it for weeks. When the big day arrives, they dress up, they drive up to the opera house, they queue for the privilege of being allowed to show their tickets and being allowed in, they take their seats, and they wait in excitement and anticipation until the big moment arrives. If your name is Joshua Bell and you play your Stradivarius in a concert hall, people will clap their hands you before you've even started. You'd better make it be good, or they'll demand their money back. If, however, you play on the street, then that's a completely different matter. You don't get to have people circle the date of your street concert months in advance. You have exactly one second to let them know you're playing. You don't get to have an audience that will clap their hands the very moment they see you turn the corner; instead, you'll see an audience that would rather you did not stand in the middle of the way like that, so that they would not have to look out for you. In short, you don't get their attention on a silver platter; instead, you have to earn it. As someone who's done street music in the past, I can tell you that earning someone's attention isn't something you will do by playing the most difficult music that can be found. You should remember that you've got just a few seconds to catch someone's attention; from the moment they vaguely hear someone's playing, until that time when they're getting annoyed at the volume your, no doubt wonderful, instrument is making. If you want people to stop and listen, you need to do it in that very short timeframe. A wild and fast virtuoso piece, while sure to get you a great reaction by an audience in an opera house, is only a very noisy thing if you just get a few seconds of it—even if those few seconds are by the best violin player alive. The best way to get someone's attention is to give them something they recognize. The article notes the reactions from a few passers-by. It did not surprise me that of the three or four people who actually stopped and listened, one recognized the violinist as Joshua Bell, and the other had been a violinist himself in a previous life. Both had something they recognized, which made both stop for a few moments to check what they were seeing. That's what gave them more time to appreciate how truly beatifull this music was, and that's why they waited. The real way to get people to stop, to get a crowd around you, and to have them to listen to you, is not to play what gets you the best results in an Opera House or other concert hall. Instead, you should play something that will catch people's attention. A brilliant piece that is unknown will not do that. A funny variation on "Oh When The Saints" will, because people recognize the music. That's not to say that street musicians should only play stuff like "Oh When The Saints". Getting money from playing on the street has more to it than just the music you're playing; just as important is the place where you're playing, and the time when you're doing it:

• If you're playing in a busy hallway, then people will just hurry along. Stopping gets them angry other people walking in their back, so that doesn't happen. If, on the other hand, you play on a large market square or some such, then people are much more likely to stand and listen—people walk in all directions there at the same time, so you have to look out anyway.
• If you're playing in a closed metro station, you'd better not use an instrument that's too loud. If you do, that will make people move away from you, rather than towards you, because they'll notice they can not talk on their cell phones or some such. If they have to pass by where you're playing to get to where they're going, they'll increase their speed so that they're faster past you. Both options give you less time to catch their attention.
• If you're playing on a monday morning during rush hour, you're not going to be getting a load of success. Everyone is going to work, and you're just something in the background, not much more interesting than the guy who's changing the billboard. If, OTOH, you're playing in a busy shopping mall on a sunny saturday afternoon, you'll get a large crowd anyhow.

A good street musician knows these things, and applies them even without knowing. He'll instinctively know whether it's a good day to do street music, and he'll rather go to a bar and have a hot drink if it's not. So why didn't Joshua Bell get a crowd around him? Easy: he isn't a very good street musician.

Stephen Touset commented:

"The worst offender in my experience is the "Verified by VISA" program. They restrict you to a subset of characters (alphanumeric, if I recall correctly) and even restrict the maximum length! You know what the worst part of it is, though? If they were using a widely-published, widely-analyzed, and widely-reviewed secure hash standard like SHA-1, Whirlpool, or any number of alternatives, these restrictions wouldn't be in place. This tells me that they're likely not using a well-reviewed standard, and instead have probably come up with their own bastardized password "encryption"."

I'm a bit worried by this, but I'm moving my customers to Verfied by VISA and Mastercard 3DSecure because of the carrot-and-stick used by the payment gateways. Am I sleepwalking off the cliff here?

Following my previous entry about playing around with LVM, LUKS and the device mapper, I documented myself about internals involved in pvmove, which is more of a challenge, considering there is no such documentation. I could find no useful documentation for either the device mapper or LVM. A bit of good old UTSL later, I could elaborate how to do what I wanted, and realized it was even possible to do in shell script. So here you are : a shell script to transform an LVM physical volume to a LVM over LUKS physical volume,. I’ll detail in between how it works. As the previous script, use at your own risk, it comes with no warranty.
Note you theorically can still access the filesystems underneath without problems. It’s an in-place and live conversion. Also note this is more a proof of concept than a proper, risk-less and well-written solution.


set -e
dev=$1
luks=$(mktemp)
cryptdev=$(basename $dev)_crypt
pvsize=$(pvs -o pe_start,pv_size --units s --noheadings --nosuffix "$dev" awk ' print $1 + $2 ')
devsize=$(blockdev --getsz "$dev")
mchunk=8


The script takes the physical volume device to convert as an argument. Note there is no check for the validity of its value.

• luks will be the temporary file where we are going to create a temporary LUKS volume.
• pvsize is the full size of the LVM physical volume, i.e. the size of all the physical extents (pv_size) + the LVM headers and metadata (pe_start).
• devsize is the raw device size.
• mchunk is the size in blocks of chunks for the mirror target, see below.


dd of="$luks" seek=$devsize count=0 bs=512 2> /dev/null
luksdev=$(losetup -f)
losetup "$luksdev" "$luks"
trap "losetup -d \"$luksdev\"; rm -f \"$luks\"" EXIT


Next, we create a sparse luks file the same size as the device (in case luksFormat would use the size somehow, but I believe it doesn’t), and a loopback device on this file. The trap is here to avoid leaving the loopback device and the file when an error occurs later (though during the conversion itself, it will be pointless).


cryptsetup luksFormat -q "$luksdev"
cryptsetup luksOpen "$luksdev" $ cryptdev _real
read start length crypt format key IVoff cdev offset <<EOF
$(dmsetup table $ cryptdev _real)
EOF


We create a LUKS device, so that we can get the encryption key ($key), and the size of the LUKS header ($offset). Note you need to add –showkey to the dmsetup table command on sid.


if [ $(expr $devsize - $pvsize) -lt $offset ]; then
  echo Not enough free space after LVM physical volume
  cryptsetup luksClose $ cryptdev _real
  exit
fi


Check we have enough space after the LVM physical volume to offset everything by the size of the LUKS header. If not, you can still try again after you reduce the size of the LVM physical volume by an extent.


if [ $(expr $devsize % $offset % $mchunk) -gt 0 ]; then
  echo Last
  cryptsetup luksClose $ cryptdev _real
  exit
fi


This is another check to avoid surprises at the end, when dealing with the last chunk. As the script is written for the moment, it doesn’t support cases where this last chunk is not a multiple of $mchunk. So we need to abort in these.


read major minor <<EOF
$(stat -t "$dev" awk ' print $10,$11 ')
EOF
maps=$(dmsetup deps awk -F: "/\($major, $minor\)/ print \$1 ")
dmsetup create $cryptdev <<EOF
0 $length linear $dev 0
EOF
dmsetup reload $ cryptdev _real <<EOF
$start $length crypt $format $key $IVoff $dev $offset
EOF
dmsetup resume $ cryptdev _real
for map in $ maps ; do
  dmsetup table "$map" sed s,$major:$minor,/dev/mapper/$cryptdev, dmsetup reload "$map"
  dmsetup resume "$map"
done


Here, we create the dev_crypt device mapper that is our fake LUKS device, which starts as a simple linear mapper and will end as a complete crypt mapper. This fake LUKS device is inserted as an intermediate mapper between the LVM device mapper and the real device. So, the LVM device mapper will have this fake LUKS device as backend, and, at the beginning, the fake LUKS device maps linearly to the real device. Note we look for all device mappers using the real device as backend before creating the fake LUKS device to avoid finding the fake LUKS device in the list. Also note dmsetup reload only loads a new table in the INACTIVE slot, and dmsetup resume makes this inactive table LIVE.


cursor=$length
chunk=$offset
while [ $cursor -gt 0 ]; do
  cursor=$(expr $cursor - $chunk)
  if [ $cursor -lt 0 ]; then
    chunk=$(expr $chunk + $cursor)
    cursor=0
  fi
  (
  [ $cursor -ne 0 ] && echo 0 $cursor linear $dev 0
  echo $cursor $chunk mirror core 1 $mchunk 2 $dev $cursor /dev/mapper/$ cryptdev _real $cursor
  [ $cursor -lt $(expr $length - $chunk) ] && echo $(expr $cursor + $chunk) $(expr $length - $cursor - $chunk) crypt $format $key $(expr $IVoff + $cursor + $chunk) $dev $(expr $offset + $cursor + $chunk)
  ) dmsetup reload "$cryptdev"
  dmsetup resume "$cryptdev"
  chunks=$(expr $chunk / $mchunk)
  while ! dmsetup status "$cryptdev" grep "$chunks/$chunks"; do
    true
  done
done


This is where the main work is done : moving the data around. We actually just let the device mapper deal with the data duplication, $offset blocks by $offset blocks ($offset being the LUKS header size), using a mirror target for the chunk being moved. So our disk looks like the following:

We use the extra dev_crypt_real device (the previously remapped LUKS device) as the encryption backend for the mirror.
I haven’t figured a better way to wait for the end of the mirroring than to do a loop checking with dmsetup status, dmsetup wait doesn’t seem to be very helpful here.
Anyways, this is the part of the script where you don’t want a crash to occur. Because if it does, all you can do is start on a rescue system, and try to find where the encrypted part of the disk start to setup a device mapper by hand.
And you’d better have the luks temporary file in a directory that is neither in RAM (think tmpfs) nor in the LVM you are converting (/tmp in a default etch install is, for instance ; note the script works nevertheless fine in this case). Also note the trap will remove the luks temporary file if the script exits…


dmsetup reload "$cryptdev" <<EOF
$start $length crypt $format $key $IVoff $dev $offset
EOF
dmsetup resume "$cryptdev"
dmsetup remove "$ cryptdev _real"
dd if="$luks" of=$dev count=$offset bs=512 2> /dev/null


Final steps of the conversion : our dev_crypt device becomes a full LUKS volume, so we can remove dev_crypt_real and add the LUKS headers at the top of the device we converted. At this moment, the LUKS volume is setup just as if it had been setup by cryptsetup. For LVM to recognize the change properly, you need to run pvscan. Once you ran it, you can do whatever you want with LVM. Now, you may want to add the following to your /etc/crypttab file:


$cryptdev $dev none luks


i.e. hda5_crypt /dev/hda5 none luks if the device was /dev/hda5. And if the LVM volume you converted contains your root filesystem, you should run (for Debian systems):


update-initramfs -u


I tested this successfully under qemu and will give it a shot on my laptop some time soon. Now, because I had a hard time not finding much about the mirror target of the device mapper, here is what I could gather about it. The target syntax is as follows:


<logical_start_sector> <num_sectors> mirror [ core disk ] <num_params> <param> ... <num_mirrors> [ <destination > <start_sector> ] ...


logical_start_sector, num_sectors, destination and start_sector have the same meaning as in other targets.
core and disk are two different log types (to track differences between mirrors), respectively in memory and on disk.
num_params and params depend on the log type:

• for core, num_params can be either 1 or 2.
  The first parameter is region_size, which is the size (in blocks) of the chunks the mirror log tracks for synchronization. It can’t be less than the page size divided by 512, i.e. 8 on x86, must be a power of 2, and must not exceed num_sectors. Apparently, it is not a problem if num_sectors is not a multiple of region_size.
  The optional second parameter is either sync or nosync, meaning of which I’m not sure. I think it determines whether the mirror should do the initial synchronization (sync) or not (nosync).
• for disk, num_params can be either 2 or 3, with params being log_device (device where the logs are kept), region_size, as above, and an optional sync or nosync, as above too.

num_mirrors is the number of mirrors and for each mirror, we have a pair destination and start_sector. [ Update: after a quick look at the device mapper source code in Linus’s git tree, updated the mirror target description ]

Those of you that look at photos on my blog may have noticed that I recently switched from a self-hosted Gallery2 installation to the popular photo-sharing site Flickr.

I initially decided to try Flickr for a simple reason: digikam's image upload to Gallery2 has never worked for me, and I hoped that its Flickr upload would work (it did).

I had heard good things about Flickr from friends and colleagues, so I gave it a try.

First Impressions

Almost the first thing I noticed about Flickr was that there are people there that are interested in the same things I'm interested in. I discovered the Anabaptist History group. I've been contacted by a person in North Dakota that is interested German-speaking Russians, as my ancestors were. There are groups about Debian, classic-style black & white photography on digital cameras, and really just about any topic. These groups contain both discussions and a photo "pool" to which any group member can post.

In one sense, Flickr is social networking via photography.

Flickr also seems to be a company with a clue. It has tons of open APIs, permitting everything from flickrfs (mount flickr as a FUSE filesystem) to countless uploaders, mashups, scripts, etc. Despite a couple of recent high-publicity controversies, they seem to be laid back overall and haven't been turned into a corporate drone by Yahoo. There's an RSS feed for just about everything, too.

What is Flickr?

Depends on what you want it to be. It could be that place where you post your cell phone's snapshots, or a place to share photos with family in private, or just a place to find other's work.

I found sharing and talking with others to be infectious. I went from never getting comments on photos on my Gallery2 site to wondering what people will think -- and looking at their photos too.

Let's look at its different aspects in more detail.

Uploading Photos

You can upload photos manually with a HTTP form, or you can use any of the numerous uploading tools. Most of these will let you scale down your images on the fly, should you wish to. Most will also let you assign tags and write descriptions as they're being posted, as well as define who can view the photos. Uploaders are available for every common platform.

Flickr has a AJAX tool called the Organizr that helps you assign tags, add photos to groups, put them in sets, etc. It's fairly nice.

Sharing Photos

Making photos available to the public is easy. You can just them them a link to your photostream. You can also tag photos and give people a link to photos with a specific tag. And you can create sets -- photos around a specific theme.

Security

Each photo can be made visible to: just you, just your contacts marked "family", just your contacts marked "friend", any of your contacts, or anyone at all. This can be defined per photo.

On a global basis, you can define whether you make EXIF information visible on the site, default security for new uploads, who can post comments, who can post photo notes or add tags, and who can use the "all sizes" button.

You can also control how much of your profile to make public. Settings range from making everything public to concealing even your email address.

For sets, you can also send out a "Guest Pass" (GP). A GP lets someone that is not a Flickr member see non-public photos. I use GPs to share family photos, so that family doesn't have to create a Flickr account to see them. GPs can be tied only to sets.

Groups

Flickr groups appear to have been designed for discussions, but I find them most attractive because each group has its own dedicated photo pool. Members of the group can add their photos to the pool. Some groups have tens of thousands of photos, and others a few dozen. It's an interesting way to find images. There are a lot of group management tools as well.

Free vs. Paid Accounts

Free account holders can upload up to 100MB of photos per month. There is no cap on bandwidth consumed by views of photos or management of them. Free users can create up to 3 sets and put photos in up to 10 groups.

The photostream for free account holders will show only their 200 most recent photos. Additionally, the "All Sizes" button will max out at the Large size, which generally will be 1024 pixels along the longest side.

Paid accounts are $25/year and include unlimited upload bandwidth (10MB max per photo though), unlimited photostream size, unlimited sets, and removal of all ads on the site. Additionally, the original upload is made available to you. If you have enabled the All Sizes button, it's also made available to the public. One nice thing is that with a free account, all that data is saved, even if it isn't available. So if you later upgrade to Pro, it's all there.

Prints

Flickr sells prints of your photos for $0.15 each. As with viewing, you can designate who can purchase prints of your photos. I ordered a test batch and they turned out very nice -- approximately the same quality as Shutterfly, which is one of the better consumer-quality labs. They are a bit pricier than Shutterfly when buying in bulk, though the convenience of all the Flickr uploaders may make up for that.

Annoyances

There are a few things that bug me.

One is that the "All Sizes" button is either on or off. Once I upgraded to pro, suddenly my original size images were available. There is no way with a pro account to limit image availability to the 1024xwhatever size like there is with free -- either you give people the medium size only (500xwhatever) or you give them everything. Those that want to have the originals on Flickr generally upload twice: once in original size, kept private; and another resized to 1024xwhatever, made public.

It is also complex for some people to join Flickr. Several of our family and friends had trouble with the signup process, since you first have to create a Yahoo account, then a Flickr one, which can use a different username. It's a nice flexibility, but overly complex for people that just want to sign up quickly. Guest Passes have received rave reviews however -- with the one caveat that the guest pass URLs aren't bookmarkable (they set a cookie and then redirect to the real target).

There is also no built-in way to see all the new photos in the groups you're a member of. However, each group's pool has an RSS feed so I have just subscribed to them with bloglines.

Overall

Overall I am happy with Flickr. I will be migrating my Gallery2 install to it, and will not miss having to maintain Yet Another PHP Script on my server. The best part about Flickr is the community, which has already taught me about mpix.com, which I'll write about tomorrow.

After my brother's wedding, a bunch of us Goerzens got together out at our creek. We had a hot dog roast, some smores, and then eight of us remained to camp out at the creek. That included Terah, Jacob, and me.

My uncle loves camping and has organized camping at the creek many times over the years. This was the first time it worked out for me to join them in quite awhile. Jacob slept well overnight, but woke up 5 minutes late -- 6:35. Never mind that he got to bed hours later than his usual time, he still woke up at the same time as always. He's funny that way.

My uncle brought with him supplies for making pancakes over the campfire in the morning. They were great. After breakfast, it was time to take down the tent and get back home to shower before going to church.

Now, a word about Terah and camping here. Terah had only gone camping with me once since we were married, and that was two years ago. This time, we had a larger tent, warmer weather, and most importantly, a complete lack of raccoon fights all around us.

I had visited a Coleman Factory Outlet store in Wichita in the morning, before the wedding, and bought a new Coleman Crestline tent (the one on the left above) and some other supplies. It's a very nice tent with a lot of ventilation options and plenty of little pockets to store things. Jacob slept in his car seat (which he likes to do) in the tent with us.

I think we might not have to wait two years before going camping again. Though I'm sure Terah will correct me if I'm wrong on that.

One feature I’ve always found terribly useful for team maintained packages are commit emails. They help you know what’s going on without need for extra communication (eg. that somebody started packaging a new releases), and they are useful when there are new maintainers, because their work can be easily reviewed. Centralized version control systems like CVS or Subversion make possible and easy to have such messages sent automatically upon commit, so all fits nicely. Enter distributed version control systems; particularly those like Bazaar that can push over dumb protocols like SFTP: it is not possible to have the server do something (send an email) when new revisions are commited. One has to resort to solutions like bzr-email, where each individual commiter configures a plugin to send out the email from their machines when they commit. The result IME is that, either nobody does that, or not everybody does. Either way, no cookie. So I decided to bite the bullet here at DebConf, and write a tool to solve this problem: a program that monitors the branches in the machine where they are hosted, and sends emails for new revisions. The result is here. If you’d like to use it for an Alioth project, you just need to do:

 alioth% /var/lib/gforge/chroot/home/users/adeodato/bzr-hookless-email/alioth_helper.sh $PROJECT_NAME -d

(And ensure that the process does not die, or gets restarted when it does. I guess I could add code for that in the helper script, hm.)

Thanks to everybody involved in the Video Team recording the talks and BoFs so the people who can’t be there we feel closer. Maybe the streaming is not as successful as expected because the same obligations that keep out being there, don’t let you watch the videos on live but anyway, those records are the link between the Debconf and the rest of the world.

Pete Nuttall writes about the fnreduce function:

fnreduce :: [(a->a)]->a->a
fnreduce [] value = value
fnreduce (a:as) value = freduce as $ a value

A shorter, although not necessarily clearer, expression of this function would be

fnreduce as value = (foldr (flip (.)) id as) value

This just says that to apply a list of functions to a value, we first compose all the functions by folding down the list, then apply the result to the value. The important thing here is (flip (.)), which says to compose backwards; it means that fnreduce [f, g, h] returns (h . g . f . id) instead of (f . g . h . id). An interesting side note is that it doesn't matter (from a semantic point of view) whether I use foldl or foldr, since function composition is associative:

f . (g . (h . id)) = ((id . f) . g) . h

But "flip" here feels a little confusing. A clearer approach would be to reverse the list instead of reversing the operator:

fnreduce as value = foldr (.) id (reverse as) value

This might be less efficient, but it says in a much clearer way what I'm doing.

Yet another option is to eschew the use of function composition:

fnreduce as value = foldl (flip ($)) value as

Here ($) is the application function; (f$x) applies f to x. So if we reverse it (call this $$), we get (x$$f) applies f to x. Folding this from the left down the list [f, g, h] gives us (((value$$f)$$g)$$h), which is exactly what the original fnreduce did.

I think this is maybe the best fold-based solution to the original problem, because it models what's happening (repeatedly transforming a value with a series of functions) directly. To use a right-fold, you have to remove the flip and reverse as (the reason this is necessary is left as an exercise to the reader).

---

Pete also asks whether this can be done for heterogenous lists in a type-safe manner. The core problem is, what is the type of our list? It can't actually be a list: in order to check that it's OK to combine the output of one function with the input of another, we would need the elements of the list to have different types. You can't do this in Haskell, even a crazy ghc-extended Haskell.

So, we could try something like this:

module Test where

data TransformList a b where
    Nil   :: TransformList a a
    Cons  :: (a -> b) -> TransformList b c -> TransformList a c

apply :: TransformList a b -> a -> b
apply Nil         = id
apply (Cons f fs) = (apply fs) . f

This lets us do stuff like

> apply (Cons (\x -> x + 1) (Cons (\x -> show x) Nil)) 5
"6"

This is cute, but I don't know that it's really that useful. After all, how is it any better than the following?

> ((\x -> show x) . (\x -> x + 1)) 5
"6"

Everything the above code lets you do with TransformList could just as well be done with the humble compose operator. The only benefit would be if we could somehow do "list-like" stuff with a transform list. For instance, say we want to write the following function:

traceIntermediates Nil        x = (x, [])
traceIntermediates (Cons f l) x = let x'        = f x
                                      (x'', ss) = traceIntermediates l x'
                                  in
                                    (x'', (show x'):ss)

This won't compile, because we don't know that x' is a member of the Show typeclass. And there is no way to fix this in a reusable way using any technique that relies on building a type parameterized on the input and output types: the problem is that we need to say something about the types of the intermediate values, but their type variables are inaccessible.

In other words, there isn't any way to write down a constraint on the type "TransformList a b" that constrains the intermediates. So I've just wasted a lot of typing duplicating the compose operator. Yay.

On the other hand, it's entirely possible to do this if you don't mind some boilerplate code.

data ShowTransformList a b where
    Nil   :: (Show a) => ShowTransformList a a
    Cons  :: (Show a, Show b, Show c) => (a -> b) -> ShowTransformList b c -> ShowTransformList a c

traceIntermediates :: ShowTransformList a b -> a -> (b, [String])
traceIntermediates Nil        x = (x, [])
traceIntermediates (Cons f l) x = let x'        = f x
                                      (x'', ss) = traceIntermediates l x'
                                  in
                                    (x'', (show x'):ss)

apply :: ShowTransformList a b -> a -> b
apply Nil         = id
apply (Cons f fs) = (apply fs) . f

Now I can do this:

> traceIntermediates
    (Cons (\x -> x + 1)
          (Cons (\x -> x * 2)
                (Cons (\x -> x - 5)
                      (Cons show
                            (Cons (read :: String -> Double)
                                  Nil))))) 5
(7.0,["6","12","7","\"7\"","7.0"])

Did some work on d-i stuff, went to Yuri's mind-blowing Gentoo/Debian mashup talk. Very proud that "squeezed joeyh brains" went into that. Lots of other tech stuff, too much to blog about. But I spent a couple hours this afternoon on tourist stuff. Went up to Edinburgh Castle. Here's the view for the people at home: (Some more pics from there are in Maykel Moya's gallery Tomorrow: Geeks On A Train
(150 of them. In the Scottish countryside.)

Today I mostly familiarized with Ruby On Rails coding, after an absence of several months. I wrote a quick multi-user online DVD catalog program to replace my current online DVD list. This is pretty neat as it allows me to actually edit the films via the web GUI, and should also allow me to create per-user lists of films. Of course the hard part will be importing all my "old" data, then adding all the movies that Megan brought to the flat when she moved in ..

First Draft of the Vim Packaging Policy Available Yay again for DebConf7!! Today the talks I attended were really interesting and in the remaining slices of time I managed to complete ... suspense ... the first draft of the Vim packaging policy \o/. The policy covers packaging of Vim addons and start to tell people about vim-addon-manager. The policy will be shipped in the next upload of the Vim package and is also available on the web. If you're maintaining a package which ships an extension for Vim please have a look, since soon it will be enforced on your package Also an eye from native English speakers will be appreciated, no matter if you speak Vim or not. PS please do not tell bzed about that, he will probably be hating me for "wasting" time on Vim rather then on polishing the first Zenoss package ever

4:30am and I can't sleep... which in a way is good since it gave me some time to think about the script I posted yesterday and how it could be generalized into a simpler and far more powerful dcmd script which just expands any .dsc/.changes argument found in the command line:

elegiac$ dcmd scp -C /tmp/1/dasher_4.4.2-1.dsc romain@yeast:/tmp
dasher_4.4.2.orig.tar.gz                      100% 8728KB   4.3MB/s   00:02    
dasher_4.4.2-1.diff.gz                        100% 5231     5.1KB/s   00:00    
dasher_4.4.2-1.dsc                            100% 1259     1.2KB/s   00:00    
elegiac$

Its only limitation is that the .dsc/.changes must be local in order for Python to parse it.

elegiac$ dcmd sha1sum dasher_4.4.2-2_amd64.changes                                                   
87cd6eb29d5f54c1547b2bf7531a0e303dbc2fa3  dasher_4.4.2-2.dsc
70d0729c8955309efa230832cf5e82bd5e28cf7c  dasher_4.4.2-2.diff.gz
a5c15d3e6c3b297897651e9a5aa40451c73b2420  dasher-data_4.4.2-2_all.deb
68383058289fb41cdf19d57225afafafd4032719  dasher_4.4.2-2_amd64.deb
481e2008ad78be0721335634954dbe3a0439914b  dasher_4.4.2-2_amd64.changes
elegiac$

Sound useful?

PS: Christoph, I know that dput supports scp and rsync, but it requires a host definition in ~/.dput.cf, and works only on .changes files.

hacking at debconf Zenoss Stefano reported about the work on the Zenoss packages yesterday, now I m proud to be able to present the first screenshot from Zenoss, freshly installed by using dpkg -i. There s still a lot of work waiting for Fabio, Stefano and me, but it s great to see the first results. It s really great to work together with them, here on the Debconf. I m really happy that I can be here. At the moment we re waiting for Sean Finney no arrive, as we have some questions about dbconfig-common. hacklab pano 3 Again, here s another pano from the hacklab 1, taken yesterday evening. Sorry for the bad stitching, didn t have the time to fix it, but I hope it ll still give am impression of the great atmosphere here in the Teviot.

Several of my workmates and I are regular listeners of technology-related podcasts like Diggnation, TWiT or FLOSS Weekly (which BTW is as “weekly” as the Halley comet). Some days ago we realized that there weren’t similar podcasts in Spanish (or at least we didn’t know about them), so we thought that creating one would be cool. During this week we recorded our pilot, we had some problems with sound and I spoke a bit too fast, but I think it’s nice overall, at least I would enjoy listening to it Trying to emulate people like Kevin Rose or Leo Laporte (in the media business since 1991), who create podcasts professionally, is quite hard for amateurs like us, but we’ll do our best. Be sure to check it at Foocast (sorry, Spanish only, that’s the whole point )

So I'm off travelling again. I've done way too much lately, and I'm looking forward to getting home and staying there for a while. I flew from San Francisco to London yesterday for my first ever business class trip. United business class is whole different experience to United economy, that's for sure. Aside from getting ridiculous amounts of legroom, and much better cabin service, the thing I like best about the whole experience was the preferential treatment. I arrived at SFO, and there was an unholy amount of people at the economy check-in counters, so it was really nice to breeze around to the business check-in counters and wait in a much more manageable line. At Heathrow, I got to used the priority immigration line, and my bag came out pretty quickly, so I'd breezed through Heathrow and onto the tube faster than you could say "Pussycat, pussycat, where have you been?". Very happy with all of that. I caught a Piccadilly line tube from Heathrow to Acton Town, then changed to the District line to Victoria station. The tube (particularly the first one) was smaller than I expected. The ceiling was pretty low, the aisle narrow, and it just felt pretty squishy. The second one seemed a bit wider and taller. It didn't feel like it could really deal with peak-hour commuting, but I guess maybe the frequency makes up for the capacity... Buckingham Palace Road turns out to have pretty wacky numbering. Just because you're opposite number 80, doesn't mean the odd numbered 80's are anywhere near you. So it took me longer than it should have to find the hotel, and I wandered up and down Buckingham Palace Road for a while before I'd managed to calibrate myself to the numbering. More frequent numbers on the buildings would have helped immensely. So I've hardly seen anything, I'm really just passing through on the way to Edinburgh for Debconf, but I love what I've seen so far. The red double-decker buses. The taxis. The post boxes. The phone booth's. It all feels really really old, but still really really alive. We've got to come back and do the place properly. The office is really nice as well. Very modern looking. Very open-plan like the Sydney one. I wasn't able to sleep on the plane unfortunately (I couldn't convince my brain to switch off), so I was pretty zonked by the end of the day (got into the office shortly after 9am). I had a fairly productive day, and went to the hotel and crashed at about 6:30pm, woke up at about 2:30am and couldn't get back to sleep. Hopefully if I just soldier on through today, I'll get to sleep a bit later this evening and do something closer to a normal night's sleep. I'm catching the train to Edinburgh today (I blame Mark (who doesn't appear to believe in permalinks) for putting the idea in my head). Looking forward to another tube ride and seeing some of the countryside. I've been really apprehensive about this trip ever since I booked it. Mainly because I'm going over to Dublin after Debconf to try and apply for a new visa. For some reason, I'm just really worried that's all going to go pear-shaped. I'm feeling better now that I'm here, but I'll feel even better once the trip to the consulate is behind me. Even better still when I get my passport back successfully. Best still when I'm home again.

Transcript of the call for participation on the Gathering of the Indigenous People of the Americas. If you don’t mind, feel free to skip it. Considering that 515 years after the first invasion of our ancestral territories, the war of conquest, the plunder, and capitalist exploitation have not altered their course, but rather have become a new war of extermination, a war designed for the destruction and utter plunder of all the original peoples of the Americas; Considering that the long history of wars of independence and numerous revolutions that have taken place on our continent have not yet changed the condition of colonized people, nor have they allowed for a full recognition of our rights in the nation states that formed in the last two centuries; Considering that despite a long history of domination, our resistance has sustained itself and that our people continue to live and struggle, and that this struggle has been encouraged by the uprising of the Zapatista National Liberation Front (EZLN) in the first days of 1994; Considering that the strategies of the national governments of the Americas have always sought the division of our people through the establishment of borders, reservations, and legislation designed to fragment us and to neutralize our efforts of autonomy; Considering that the strengthening and the unity of the struggles of liberation of the indigenous people of the Americas will be possible as our peoples meet and come to know each other; Considering that for our people it has proven necessary that their struggles be made known world wide in order to be supported by all honest movements that struggle for a society effectively just, free, and democratic; All indigenous communities, tribes, and nations of the Americas are called to

THE GATHERING OF THE INDIGENOUS PEOPLE OF THE AMERICAS Which will take place the 11st, 12nd, 13rd, and 14th days of October, 2007 in the community of V cam, Yaqui Territory, Municipality of Guaymas, State of Sonora, Mexico, to address the following

TOPICS 1 The war of capitalist conquest in the indigenous communities in the Americas.
2 The resistance of the indigenous peoples of the Americas and our defense of
mother earth, our territories, and our cultures.
3 Why do we, the indigenous peoples of the Americas, struggle?

ORGANIZATIONAL PRINCIPLES 1st Principal. Delegates, observers, and communications media workers attending the Gathering will abide by the regulations of the Yaqui Tribe devised by their Traditional Authorities in order to govern their presence while in attendance.
2nd Principal. All participating indigenous communities, tribes, and nations are invited to demonstrate their clothing, songs, dances, and other traditional forms of expression.
3rd Principal. Authorities and Representatives of indigenous communities, tribes, and nations of the Americas are invited to participate as delegates with voice and vote.
4th Principal. Individuals and members of social movements and civil organizations in solidarity with the struggles of indigenous peoples of the Americas invited by the organizing commission will be able to participate as observers.
5th Principal. National and international communications media workers that are accredited by the organizing commission will be able to cover the event.
6th Principal. All agreements adopted at the Gathering will be made by consensus of the participants and not through the method of voting.
7th Principal. Organization of the Gathering, the national and international distribution of its publicity, as well as accreditation of participating delegates, invitees, and any unforeseen issues will be resolved only by the organizing commission in conformity with the Traditional Authorities of the Community of V cam, Yaqui Tribe, The National Indigenous Congress (CNI), and the Sixth Commission of the Zapatista National Liberation Front. Information: 1. Office of the Sixth Commission of the EZLN, located on Zapotecos St. No 7 Col.Obrera, C.P. 06800 Mexico, D.F. Telephone: (01 or 011) 55 67 61 42 36 Monday thru Friday, 10 AM 8 PM., Saturday, 10 AM 6 PM.
webpage: www.encuentroindignea.org e-mail: informes@encuentroindigena.org
2. Official Precinct Office of the Traditional Guard of the Yaqui Tribe, Vicam Station ,Colonia Yaqui, Municipio de Guaymas, Sonora, Mexico. Telephone: (045 or 001) 64 49 98 94 08.
3. Office of the Organization of Indigenous and Campesino Communities, de Tuxpan, Jalisco, in Nicolas Bravo, No. 65 , Tuxpan, Jalisco,. Telephone: (01 or 011) 371 41 764 15 from Monday Saturday, 10 AM 8 PM. e-mail: comunidad_tuxpan@hotmail.com Written - April 22nd, 2007 in Tohono O Odham Territory of the Tohono O Odham Nation, Sonora, Mexico. Attentively,

Wa-a itom yo Lutu-ria yoribetchibo, intoka a Wiutinepo Amani
(Por el respeto a nuestros usos y costumbres, nuestra identidad y cultura y nuestra verdad que permanezcan en el tiempo y no se olviden)

Never Again, a Mexico Without Us ~ Liberty, Justice, and Democracy

 

Nde Cultural Historical Organization
(Organizaci n Cultural e Hist rica de la Naci n Nde/Apache)

 

Tierra y Libertad / Chicana Indigenous Organization, Tucson, Arizona
(Tierra y Libertad / Organizaci n Xicana Ind gena, Tucson, Arizona

 

Native and Immigrant Indigenous Development Organization
(Desarrollo de Los Pueblos Indios Inmigrantes y Nativos, A.C.)

 

Michelle Cook (Dene Nation)
(Nacion Dene/Navajo)

 

Tohono O Odham Nation Mexico United States
(Nacion Tohono O Odham M xico - Los Estados Unidos de America)

 

Traditonal Authorities of Vicam Community, Yaqui Tribe
(Autoridades Tradicionales del Pueblo V cam, Tribu Yaqui)

 

National indigenous Congress (Congreso Nacional Ind gena)

 

Clandestine Revolutionary Committee, General Command of
the Zapatista National Liberation Army (EZLN)
(Comit Clandestino Revolucionario Ind gena-Comandancia General del
Ej rcito Zapatista de Liberaci n Nacional)

debcamp hacklab Although we ve moved into a different room a few minutes ago, here s a little pano from yesterday, thanks to tokkee s cam, hugin, autopano-sift and gimp Update: Here s another hackish pano of one of the hacklabs we re using since today. The stitching is not perfect again, I ll probably fix this when I m home again - or not

By popular request (if you consider Lunar to be a pupil) I have packaged my old hack vlshow as screen-message to Debian. The small program (called sm on when installed) starts and displays a given text as large as possible, in big bulky black letters on white background. You can change the text while the program is running. This might be useful to quickly send a message across the room, say something to people who think they are reading your screen unnoticed, for make-shift signs, to marvel at nice unicode characters and all kinds of other funny purposes.
I used this as a chance to try out darcs-buildpackage, which is very nice, especially as it makes the separation of upstream sources and Debian changes possible in a non-troublesome way. I attended Enrico s tutorial on autotools and autotoolized the package during that talk, which meant that I could use cdbs for building the package nice experience as well. Darcs s dist feature is very nice when using autotools, although it makes darcs-buildpackage slightly inconvenient. (While dpkg-buildpackage uses darcs dist on the upstream sources to build the orig.tar.gz, it does not check the predist command in _darcs/pref/pref when building the debian package, which fails then. It works fine if I run that command, in my case predist autoreconf -i && rm -rf autom4te.cache/, before running dpkg-buildpackage.)The package is currently in the NEW-queue, until it was accepted, you can find packages on people.debian.org. Thanks to Lunar for some patches. I m keen to hear interesting stories about how and where you used the program.