Those of you that look at photos on my blog may have noticed that I recently switched from a self-hosted Gallery2 installation to the popular photo-sharing site Flickr.

I initially decided to try Flickr for a simple reason: digikam's image upload to Gallery2 has never worked for me, and I hoped that its Flickr upload would work (it did).

I had heard good things about Flickr from friends and colleagues, so I gave it a try.

First Impressions

Almost the first thing I noticed about Flickr was that there are people there that are interested in the same things I'm interested in. I discovered the Anabaptist History group. I've been contacted by a person in North Dakota that is interested German-speaking Russians, as my ancestors were. There are groups about Debian, classic-style black & white photography on digital cameras, and really just about any topic. These groups contain both discussions and a photo "pool" to which any group member can post.

In one sense, Flickr is social networking via photography.

Flickr also seems to be a company with a clue. It has tons of open APIs, permitting everything from flickrfs (mount flickr as a FUSE filesystem) to countless uploaders, mashups, scripts, etc. Despite a couple of recent high-publicity controversies, they seem to be laid back overall and haven't been turned into a corporate drone by Yahoo. There's an RSS feed for just about everything, too.

What is Flickr?

Depends on what you want it to be. It could be that place where you post your cell phone's snapshots, or a place to share photos with family in private, or just a place to find other's work.

I found sharing and talking with others to be infectious. I went from never getting comments on photos on my Gallery2 site to wondering what people will think -- and looking at their photos too.

Let's look at its different aspects in more detail.

Uploading Photos

You can upload photos manually with a HTTP form, or you can use any of the numerous uploading tools. Most of these will let you scale down your images on the fly, should you wish to. Most will also let you assign tags and write descriptions as they're being posted, as well as define who can view the photos. Uploaders are available for every common platform.

Flickr has a AJAX tool called the Organizr that helps you assign tags, add photos to groups, put them in sets, etc. It's fairly nice.

Sharing Photos

Making photos available to the public is easy. You can just them them a link to your photostream. You can also tag photos and give people a link to photos with a specific tag. And you can create sets -- photos around a specific theme.

Security

Each photo can be made visible to: just you, just your contacts marked "family", just your contacts marked "friend", any of your contacts, or anyone at all. This can be defined per photo.

On a global basis, you can define whether you make EXIF information visible on the site, default security for new uploads, who can post comments, who can post photo notes or add tags, and who can use the "all sizes" button.

You can also control how much of your profile to make public. Settings range from making everything public to concealing even your email address.

For sets, you can also send out a "Guest Pass" (GP). A GP lets someone that is not a Flickr member see non-public photos. I use GPs to share family photos, so that family doesn't have to create a Flickr account to see them. GPs can be tied only to sets.

Groups

Flickr groups appear to have been designed for discussions, but I find them most attractive because each group has its own dedicated photo pool. Members of the group can add their photos to the pool. Some groups have tens of thousands of photos, and others a few dozen. It's an interesting way to find images. There are a lot of group management tools as well.

Free vs. Paid Accounts

Free account holders can upload up to 100MB of photos per month. There is no cap on bandwidth consumed by views of photos or management of them. Free users can create up to 3 sets and put photos in up to 10 groups.

The photostream for free account holders will show only their 200 most recent photos. Additionally, the "All Sizes" button will max out at the Large size, which generally will be 1024 pixels along the longest side.

Paid accounts are $25/year and include unlimited upload bandwidth (10MB max per photo though), unlimited photostream size, unlimited sets, and removal of all ads on the site. Additionally, the original upload is made available to you. If you have enabled the All Sizes button, it's also made available to the public. One nice thing is that with a free account, all that data is saved, even if it isn't available. So if you later upgrade to Pro, it's all there.

Prints

Flickr sells prints of your photos for $0.15 each. As with viewing, you can designate who can purchase prints of your photos. I ordered a test batch and they turned out very nice -- approximately the same quality as Shutterfly, which is one of the better consumer-quality labs. They are a bit pricier than Shutterfly when buying in bulk, though the convenience of all the Flickr uploaders may make up for that.

Annoyances

There are a few things that bug me.

One is that the "All Sizes" button is either on or off. Once I upgraded to pro, suddenly my original size images were available. There is no way with a pro account to limit image availability to the 1024xwhatever size like there is with free -- either you give people the medium size only (500xwhatever) or you give them everything. Those that want to have the originals on Flickr generally upload twice: once in original size, kept private; and another resized to 1024xwhatever, made public.

It is also complex for some people to join Flickr. Several of our family and friends had trouble with the signup process, since you first have to create a Yahoo account, then a Flickr one, which can use a different username. It's a nice flexibility, but overly complex for people that just want to sign up quickly. Guest Passes have received rave reviews however -- with the one caveat that the guest pass URLs aren't bookmarkable (they set a cookie and then redirect to the real target).

There is also no built-in way to see all the new photos in the groups you're a member of. However, each group's pool has an RSS feed so I have just subscribed to them with bloglines.

Overall

Overall I am happy with Flickr. I will be migrating my Gallery2 install to it, and will not miss having to maintain Yet Another PHP Script on my server. The best part about Flickr is the community, which has already taught me about mpix.com, which I'll write about tomorrow.

I’m just sitting in the night venue of Debconf7 and do several tasks in the same moment, that includes: * Getting kind of a spanish course on IRC in #debian-qt-kde * Get some news from friends in .ch via MSN (there were several unweathers and other troubles) * Think about tdbackup which is the topic of this blog entry. So what is tdbackup? It is just another backup implementation but it should work kind of distributed. I manage several servers and in that farm are 3 backup servers with a tape drive, every sunday i mount parts of the other machines via NFS and write them with tar to the tape. Until I did that by hand because there was no implementation which fits my needs unfortunately. This is the point where tdbackup jumps in. I want to create a GUI which edits an ini file in the form of: [backupmachine 1] mount = bla1:/home mount = bla2:/home tdbackup will then distribute a specifically generated configfile to each backuphost and a little python script which is called by cron and mounts the file systems, write some information into mysql and generate warning messages via e-mail if the tape is almost full. So, I will be able to sleep better because the stuff is auto-documented in a way. I’m still thinking about how exactly i want to implement that, if there are any interested people who would like to join that project, please just tell me.

So, I'm back from Holidays, to find we now have a new Government in Ireland. (An unexpected choice of government, of which more later.) We were camping in Fouesnant, in Brittany; Family & Friends might like to see the holiday snaps here). Camping in France went well. With three children aged 7, 4 and 2, there is only so much you can expect from a family holiday. Like it or not, you have to mind the toddler fairly consistently, and the older kids would like your full attention too. This is a holiday ''for'' the kids; plan also a break ''from'' the kids later (a long-weekend, with kids at grandparents, for example). This was our first time camping abroad, and its well recommended. While we went touring with a camper van with the two older children two years ago, this is different in that we were at the campsite for almost two weeks: the kids got to meet and play with others in a way they couldn't before. Also, there were good facilities at the site in Fouesnant, and the swimming pools in particular were appreciated (Caveat emptor though: there were no lifeguards or supervision. We went off-peak, and there were apparently two accidents in the time we were there: a broken kneecap and chin when people came down the slides too soon after others and collided). It was the first time the children (a) had pocket money to spend, and (b) were able to walk to a (campsite) shop to spend it. Lessons were learnt in the finiteness of pocket money, especially by Caoimhe . The weather was fairly good: two wet days (though not overly so, compared to what we're used to in Galway); enough to give the 'authentic camping experience'. And for the children especially, just warm enough. Long drives South were not an option with young kids, and we wanted to see Carnac and Brittany in general. We went touring to two places: Mont St. Michel, on the way down, Carnac, and locally around Quimper and Concarneau. At seven, Cormac is beginning to be old enough to enjoy this, but Sadhbh at 2 is too young, and easily aggravated. Caoimhe can appreciate some of it, such as the well recommended Breton museum in Quimper, seeing the local costumes and clothing is just her style at the moment. We're looking forward to going back when the kids are older, or we can enjoy touring on our own. The camp site is dominated by English / Dutch / Irish. It is still term-time in Britain and Ireland, which meant the children on the site was full of 2-4 year-olds, with a shortage of playmates for Cormac. Next time, synchronise with English school times, etc. We went during Cormacs term-time because it was much cheaper (1000 Euro) than high-season. If Cormac was having difficulty in school things would have been different, as it is, we had no qualms in doing so. Things will probably be different next year with Caoimhe also in school at the same time, and we would go later. Suffice it to say I think synchonisation of school timetables is a bad idea, as it has led to these huge price differentials and parents taking their kids from school. As a friend with experience of the French school system points out, they've spent the last 40 years desynchronising their timetables ... So would we go again, or buy our own tents, etc? possibly. The brochure for next year arrived already, with large discounts for booking early ... Tags holiday, family, france, brittany, cormac, coimhe, sadhbh

At the Free Culture National Conference a few weeks ago, Kevin Driscoll initiated a project that I feel is hugely important: he's prompted the free culture community to state and share their vision. While I've talked a lot about definitions in the past, I probably should have been talking about goals or vision. Kevin has created an important opportunity for all free culture stakeholders to step back and imagine what the world will look like when we win. By doing so, we end up defining a set of implicit goals for our social movement and can then set to work on the hard part: figuring out how we get there. With thanks to Eben Moglen for much of the inspiration, here's mine:

People remembered that there is no scarcity in information goods except where they have created it. As evidence grew of the positive effects of free culture and the toll of information ownership, our communities decided that we were not well served by limits on the flow and development of knowledge. Accordingly, the gatekeepers and tax collectors for culture have withered away and were dismantled. We -- the consumers, creators, and re-creators -- have offered new, more ethical business models, have engaged in new methods of distribution, and have produced creative goods. Today, access to information is a simple matter of connecting someone to a network and a community: a technical problem that we know how to solve. Nobody pays for the "right" to hear music, read a book, watch a movie, or use a piece of software. Nobody is forced to choose between being a bad neighbor or friend and breaking copyright law. No artist, musician, or author sells a million copies of anything and no artist, musician, or author has a day job.

Now it's your turn. Eben Moglen tell us to not stop until we're free. Let's paint a picture of what that free world looks like. Most importantly, let's challenge ourselves to find ways to make it possible.

Web applications are in constant competition for a user's attention. Unlike shrink-wrap software, there's often no captive audience, and web apps must gently guide and woo users to help them to the best experience of the software. This is particularly true when considering the problem of collecting personal details from a user. Collect too little information, and your app might not be able to function effectively. Collect too much information, and you raise the barrier for participation with your application. Users may either not be bothered, be alarmed at the apparent intrusion into their privacy, or fill out bogus information.  One size doesn't fit all
I originally stumbled on the issue in the development of Expectnation. The amount of information a conference speaker needs to give the organizers is significantly more than that of, say, a reviewer. An organizer will often want postal and phone details for a speaker, which are largely irrelevant to the review process. All conferences and events rely on the willingess of participants, so the path to participation needs to be as frictionless as possible.
Thus, a one-size-fits-all personal details form didn't make sense for us. At best, we could have simply made more fields optional, but that would result in missing information that we really did need. Another solution would be to provide a dedicated personal details form per user type, e.g. different forms for speakers and reviewers. This is a non-starter, however, as users can take on many roles within a conference. So, we had a problem on our hands: different levels of interaction with our application requires different levels of personal information. We also had a second problem, which is that not all events and organizers want the same level of information. For example, a user group event is unlikely to need the same amount of information from speakers as an academic conference. Thus we have two axes on which the amount of information required varies: by role, and by event.  Our solution: progressive disclosure We concluded that we wanted to adhere to two principles in the collection of user information for Expectnation:

• collect as little information from users as possible, and
  
• allow organizers to determine the level of information they wish to collect.

The first objective was achieved by constructing a matrix of information items against user role, with the possible values of required, optional, unused for each item. We then achieved the second objective by allowing this matrix to be varied over each event. We've named this concept "progressive disclosure." What this means in practice is that a user is only ever asked for the information required to fulfil the roles they have within Expectnation. If they take on another role, they are asked for the additional information at that point, and not before. Implementation concerns
Extract from disclosure level editing UI It would be untrue to say that progressive disclosure is simple to implement. A user in our system will have disclosure requirements that are the union of all of their roles in all of the events they attend. Within Ruby on Rails, this means the conventional validation logic is useless to us, as it only deals with static requirements. Secondly, disclosure requirements are expensive to compute, so we need to cache them, with the inevitable addition of issues such as cache invalidation. Beyond the data model issues, the user interface issues are, if anything, more troublesome. From an end-user's point of view, the difficulty arises when we need extra information from them, and presenting this in a way that makes sense. From the event organizer's point of view, we have the challenge of presenting a controlling user interface for rather a complex concept. Organizers must be able to change the level of required information either globally, or per-event. We opted for a grid presentation, an extract from which is shown in the screenshot in this article. Over time we'll be able to see how effective this interface is, and what we can do to improve it. Finally, we have chosen sensible defaults for all disclosure levels, as the best result would be that organizers have little need to reconfigure the settings in the first place. Making it look easy Because Expectnation is a general application for organizing events, rather than targetting one specific one, we've had to construct a framework for minimizing user information collection, as opposed to simple implementation of known scenarios. At the end of the day we have added a fairly complicated subsystem and an extra degree of complexity to our codebase. However, the result from the user's point of view is increased simplicity and a streamlined experience. Every time you embark on something that wide-reaching and complex, you have to ask yourself whether the feature is really worth it. In this case, though, the answer is a resounding "yes". The user is the most important factor in our software, and making their life easier in an unobtrusive manner is a priority.

• Expectnation blog: Only ask users for the information you need

Since the release team update got the arm architecture names mixed, chances are that others are confused as well.

* arm - This is the current in-Debian, little-endian hard-float old-abi port.

This is somewhat inefficent port, as the hard-float code needs to be emulated
in the kernel. It is also depreciated by upstream.

* armeb - This was a effort to create an big-endian old-abi port.

Since the Linksys NSLU-2 got it's Ethernet driver reverse engineered and thus it became possible to run little-endian Debian on it, interest on this port has been weak. If interest in bigendian arm returns, it will probably be re-ported using EABI and softfloat.

* armel - This the shining new little-endian EABI (and thus soft-float) based architecture.

THIS IS SPARTA^W THE FUTURE.

The journey to Edinburgh on Wednesday went pretty well. The only problem I ran into was that after landing at Edinburgh airport we had to wait in the plane for a few more minutes because the airport's doors wouldn't open. I felt pretty beaten when I finally arrived at the venue and just exchanged a few greetings and took a look around before going to the hostel and finishing my night's sleep. When I got up again it was almost time to go for dinner and then pick up Tolimar who got here on a later plane, since his appclication for travel sponsorship got accepted much earlier. (He's giving a talk and a BoF, so he had a higher priority.) On Thursday I saw a few talks and attended the debian-women BoF, which I think was very productive and - most importantly - gave us back some of the momentum we'd lost over the last years. We've got a whole lot of plans for new things to do now and I am looking forward to getting to work on them when I get back. The day ended with the DebConf Ceilidh which was pretty fun altough it was quite obvious that geeks may not actually be the best dancers around. Lack of rhythm was easily made up with enthusiasm though and the slight lack of women was also easily compensated. I think that Ceilidh band will remember us. We even contributed a few musicians of our own! Once the Ceilidh was over I pretty much fell right into bed. Although it's good to be here and meet all the people, only being at DebConf for the last few days generally tends to be rather frustrating. Firstly it's always weird to join a group that has already been together for a while and also DebConf really just isn't something that can be condensed to a few days. I've just about started to get into the flow today (Friday) and tomorrow I'll already be leaving again. Shame. Anyway, only being here for a bit is still better than not being here at all and I very much hope to have more time next year when we meet in Argentina. For now I'll just enjoy my remaining last day at DebConf and hope for a better time next year.

A month ago I asked for help to localize reportbug-ng. To my surprise translations came in pretty quick. Junichi Uekawa was the first one, he sent his Japanese translation only a few hours after my request for help. Currently we have rng translated into the following languages: Catalan, Czech, English, German, Japanese, Spanish and Swedish. Thanks David Planella, Miroslav Kure, Junichi Uekawa, Javier Fern ndez-Sanguino Pe a and Daniel Nylander for your translations! But that’s not enough. If your language is missing, please consider translating rng to your language. Just take the source of the current version in unstable, grab the .pot and translate it. There isn’t really that much text to translate. The .pot contains only 42 messages and most of them are very short. When you’re done, please file a l10n+patch bug against rng with the translation attached. I’ll include your translation as soon as possible, thank you in the changelog and will probably name my first pony after you Thanks!

Last December, I went to the Delta de l'Ebre with a good number of friends to spend one of December's three-day weekends. Les Terres de l'Ebre are the southest region of Catalunya, right next to the border with the Valencian Country. The Ebre river forms a big delta as it meets the Mediterranean, and the great amount of wildlife, specially avian, in the Natural Park makes it a fantastic place to visit. We would sleep, more or less, in Cinta's appartment in L'Ampolla. The first night we managed to fit 17 persons for dinner, and about 14 stayed to sleep in any possible corner of the flat. As soon as the Valencian group arrived, we tried to meet the people coming from the North, but apparently they got lost and we stayed on a sightseeing spot admiring how easily the weather changes here. In an hour or so, we went from a sunny day to a quick but heavy shower and back to sunny. When we finally found each other, we decided it was a bit too late to do anything too special, given the short days in December, so we walked South along the beach, on our way to the Salines, the salt processing facility, in the Punta de la Banya.

Agua para todos... During the walk, we witnessed what the people from the Delta have been warning about for decades: the Delta is disappearing. The progressive overuse of the river's flow, excessive irrigation, damm construction and degradation is killing this unique place. The waters are slowly claiming what once was theirs. An electricity line, at the beginning of our trip being at the end of the beach starts approaching the sea, or viceversa, as you move southward, to end up many metres inside the water. The old people in the Delta still remember the 200m tall lighthouse in the Illa de Buda, which the sea managed to devour.

Sunset at the salines del Delta When we reached the Punta we could watch a beautiful sunset over the mountains of Els Ports, and when it got dark and cold we headed to Tortosa, the area's capital, which I had recently visited for the first time for one of Softcatal 's meetings. There, we had dinner at a bar and met Jordi and Anna for a while, before going back to Ampolla, going to bed relatively early due to everyone being very tired.

On Saturday, we decided to have a look at the Delta... from above. So we drove to the top of the Cim del Caro, where we could admire a great view over the flat farmlands of the Delta, and then found a nice trail which took us to a nice forest, where we had lunch. After a visit to Alfara de Carles, we went back to the house, had a great dinner and played fun games at the table. Unfortunately, Sunday morning was dedicated to cleaning up the mess in the appartment and packing, as part of the group, the Jordis from Alcoi and the twins, had to leave early. The remaining four Valencians decided to go to Deltebre to have a local paella. It was quite good, although the bar tender laughed at our request to eat directly from the paella, rejecting dishes. Small cultural clash. ;)

I remember downloading and viewing the pictures of the trip a few days after getting back in Val ncia, and then losing track of them. Last week, while testing an EPIA board with some random hard drive, I suddenly realised there was a big chance the pictures would be on that drive, and after some search, I found them; I had been close to mkfs the disk a few minutes before. I hadn't stopped blaming myself for misplacing these pictures, as they remind me of this wonderful weekend. Luckily, they are back! Going back to the state of the Delta, experts say it will be mostly gone in 200 years. All of the Terres de l'Ebre have fought against the previous government's "PHN" plan to divert part of the river's flow to the North and South, to supply Barcelona, and specially the South of Val ncia, where the developing of the coastal areas is out of control. I fear that whenever PP regains power in Madrid, they will retake this idea. There is a lot of money involved in building a 700 km long scar in our land. The Delta de l'Ebre is a fantastic place which doesn't deserve a terrible fate like this, only for the benefit of the usual suspects. As the good people of this land would say...

Lo riu s vida. No al transvasament!

After my brother's wedding, a bunch of us Goerzens got together out at our creek. We had a hot dog roast, some smores, and then eight of us remained to camp out at the creek. That included Terah, Jacob, and me.

My uncle loves camping and has organized camping at the creek many times over the years. This was the first time it worked out for me to join them in quite awhile. Jacob slept well overnight, but woke up 5 minutes late -- 6:35. Never mind that he got to bed hours later than his usual time, he still woke up at the same time as always. He's funny that way.

My uncle brought with him supplies for making pancakes over the campfire in the morning. They were great. After breakfast, it was time to take down the tent and get back home to shower before going to church.

Now, a word about Terah and camping here. Terah had only gone camping with me once since we were married, and that was two years ago. This time, we had a larger tent, warmer weather, and most importantly, a complete lack of raccoon fights all around us.

I had visited a Coleman Factory Outlet store in Wichita in the morning, before the wedding, and bought a new Coleman Crestline tent (the one on the left above) and some other supplies. It's a very nice tent with a lot of ventilation options and plenty of little pockets to store things. Jacob slept in his car seat (which he likes to do) in the tent with us.

I think we might not have to wait two years before going camping again. Though I'm sure Terah will correct me if I'm wrong on that.

The source-searching system I was talking about previously is progressing slowly. So far I've synced the source to Etch to my local machine, total size 29Gb, and this evening I've started unpacking all the source. I'm still in the "a" section at the moment, but thanks to caching I should be able to re-sync the source archive and unpack newer revisions pretty speedily. The big problem at the moment is that the unpacking of all the archives is incredibly slow. Still I do have one new bug to report aatv: Buffer overflow in handling environmental variables.. That was found with:

rgrep getenv /mnt/mirror/unpacked   grep sprintf

(A very very very slow pair of greps. Hopefully once the unpacking has finished it will become faster. ha!) The only issue I see at the moment is that I might not have the disk space to store an unpacked tree. I've got 100Gb allocated, with 29Gb comprised of the source. I'll just have to hope that the source is less than 70Gb unpacked or do this in stages.) I've been working on a list of patterns and processes to run, I think pscan, rats, and its should be the first tools to run on the archive. Then after that some directed use of grep. If anybody else with more disk space and connectivity than myself be interested I can post the script(s) I'm using to sync and unpack .. Failing that I'll shut up now.

Last December, I went to the Delta de l'Ebre with a good number of friends to spend one of December's three-day weekends. Les Terres de l'Ebre are the southest region of Catalunya, right next to the border with the Valencian Country. The Ebre river forms a big delta as it meets the Mediterranean, and the great amount of wildlife, specially avian, in the Natural Park makes it a fantastic place to visit. We would sleep, more or less, in Cinta's appartment in L'Ampolla. The first night we managed to fit 17 persons for dinner, and about 14 stayed to sleep in any possible corner of the flat. As soon as the Valencian group arrived, we tried to meet the people coming from the North, but apparently they got lost and we stayed on a sightseeing spot admiring how easily the weather changes here. In an hour or so, we went from a sunny day to a quick but heavy shower and back to sunny. When we finally found each other, we decided it was a bit too late to do anything too special, given the short days in December, so we walked South along the beach, on our way to the Salines, the salt processing facility, in the Punta de la Banya.

Agua para todos... During the walk, we witnessed what the people from the Delta have been warning about for decades: the Delta is disappearing. The progressive overuse of the river's flow, excessive irrigation, damm construction and degradation is killing this unique place. The waters are slowly claiming what once was theirs. An electricity line, at the beginning of our trip being at the end of the beach starts approaching the sea, or viceversa, as you move southward, to end up many metres inside the water. The old people in the Delta still remember the 200m tall lighthouse in the Illa de Buda, which the sea managed to devour.

Sunset at the salines del Delta When we reached the Punta we could watch a beautiful sunset over the mountains of Els Ports, and when it got dark and cold we headed to Tortosa, the area's capital, which I had recently visited for the first time for one of Softcatal 's meetings. There, we had dinner at a bar and met Jordi and Anna for a while, before going back to Ampolla, going to bed relatively early due to everyone being very tired.

On Saturday, we decided to have a look at the Delta... from above. So we drove to the top of the Cim del Caro, where we could admire a great view over the flat farmlands of the Delta, and then found a nice trail which took us to a nice forest, where we had lunch. After a visit to Alfara de Carles, we went back to the house, had a great dinner and played fun games at the table. Unfortunately, Sunday morning was dedicated to cleaning up the mess in the appartment and packing, as part of the group, the Jordis from Alcoi and the twins, had to leave early. The remaining four Valencians decided to go to Deltebre to have a local paella. It was quite good, although the bar tender laughed at our request to eat directly from the paella, rejecting dishes. Small cultural clash. ;)

I remember downloading and viewing the pictures of the trip a few days after getting back in Val ncia, and then losing track of them. Last week, while testing an EPIA board with some random hard drive, I suddenly realised there was a big chance the pictures would be on that drive, and after some search, I found them; I had been close to mkfs the disk a few minutes before. I hadn't stopped blaming myself for misplacing these pictures, as they remind me of this wonderful weekend. Luckily, they are back! Going back to the state of the Delta, experts say it will be mostly gone in 200 years. All of the Terres de l'Ebre have fought against the previous government's "PHN" plan to divert part of the river's flow to the North and South, to supply Barcelona, and specially the South of Val ncia, where the developing of the coastal areas is out of control. I fear that whenever PP regains power in Madrid, they will retake this idea. There is a lot of money involved in building a 700 km long scar in our land. The Delta de l'Ebre is a fantastic place which doesn't deserve a terrible fate like this, only for the benefit of the usual suspects. As the good people of this land would say...

Lo riu s vida. No al transvasament!

I will most probably not accept any further source packages for Debian Etch r1 after todays tonights dinstall run, unless someone really convinces me there is something very important missing. Update: Exception are granted for packages related to D-I or kernel, if not already uploaded. release

There are good chances that the Debian BTS will turn the #500000 mark at some moment during the etch->lenny release cycle. In order to make this more fun, a small contest has been set up. The principle is very simple: please place a bet (one per person) about the day this bug will be reported. The winner(s) will be the person(s) placing her his their bet as close as possible of the real moment bug #500000 is reported. Nothing to win, just fun and maybe a French kiss by the DPL at the next Debconf. To place you bet, just go to the contest wiki page and follow instructions there. Bets can be placed modified removed until July 15th 2007, 00:00 UTC.

Just to prevent people from thinking I’ve gotten old and stodgy, I’ve joined the Twitter bandwagon. Here’s my feed, which also appears in my sidebar. I’m reminded of a Utah Phillips quote, from a different context:

Well, and I’m open to all those things. If you live in California, you’ve got to be open; if you’re not they pry you open.

I’m not sure whether I really agree with Twitter–it reminds me of this interesting collection of postmodern thoughts from n+1 magazine:

The Decivilizing Process As the specific addressee of any set of remarks becomes less important, in the midst of more and more babble, it will become more and more difficult to remember the special status of listening human beings, in the confusion of shouted orders.

But, hey, it’s worth a try.

Share This

I finally decided myself to switch my laptop to using a LVM over LUKS setup instead of LVM alone. The problem it involves is that there is currently no way that I know to do this in-place. I do have an external disk on which I could pvmove everything and back, but that would not be very challenging ;). So I started wondering how doable that would be. LUKS has a 1032 sectors (a sector being 512 bytes) overhead in which it store the LUKS header. Following the header, encrypted data just takes as much space as its decrypted counterpart, linearly. So all it would take to do in-place LUKS conversion would be to move data by 1032 sectors, and encrypt it. Which means the main problem is to have 1032 sectors free after the current volume. Fortunately, LVM doesn’t take up all the space on the partition a physical volume is setup, except if you’re unlucky or created partitions specifically so that it uses all the space. Because LVM physical volumes use the space by big chunks, 4MB by default), if the size of the partition is not 4MB aligned (modulo the LVM header), you have some free space between the physical volume end and the partition end. We thus need this free space to be greater than 1032 sectors. On my laptop, there is more than 1.8MB free there, so there would be no problem.
Note the debian-installer seems to do more than a simple cryptsetup luksFormat, as LUKS volumes it creates have an overhead of 2056 sectors instead of the 1032 sectors I got creating a volume by hand. Next problem is how to move the data and encrypt it in-place. Bonus points if that can be done without unmounting the filesystems. My first guess was to play around with the device mapper and a specially crafted FUSE filesystem to create a fake LUKS volume that could be half encrypted and half not, and would evolve, with time, into a fully encrypted LUKS volume. Since I didn’t know much about the device mapper, I looked at it a bit, and discovered a FUSE intermediate would be unnecessary: not only can you remap on-the-fly (which I already guessed was somehow possible, considering how LVM can do it, though I didn’t know much about the interaction between LVM and the d-m), but you can also mix dm-crypt and linear chunks in the same d-m table. It also appears LVM over LUKS has an overhead that is actually not necessary. How sad both use the device mapper but yet can’t do an efficient setup with it. When you setup LVM over LUKS, you open a LUKS volume, which will create a dm-crypt device mapper, and use this device mapper as the physical volume for LVM. Logical volumes created with LVM are actually device mappers using the physical volume as destination device. When you access a block on an LVM logical volume, your access is first mapped through the LV device mapper and then mapped again through the dm-crypt mapper. So it goes through 2 device mappers, but it could do with only one: the linear mapping used in most of the cases with LVM could be replaced by crypt mappings, that are just as linear, but also handle the encryption/decryption part. Playing around under qemu with a freshly installed etch system, I was able to implement this and came up with the following script. It should work with most setups, but use at your own risk, it comes with no warranty.


MAJOR=$(awk '/device-mapper/ print $1;exit ' /proc/devices)
dmsetup ls --target linear while read map extra; do
  TABLE=$(dmsetup table $map while IFS=" :" read start length linear major minor offset; do
    map2=
    if [ "$major" -eq "$MAJOR" ]; then
      IFS=":" read map2 extra <<EOF
$(dmsetup info -c --noheadings -j $major -m $minor)
EOF
    fi
    if [ -z "$map2" ] [ $(dmsetup table "$map2" wc -l) -gt 1 ] [ $(dmsetup table --target crypt "$map2" wc -l) -eq 0 ]; then
      echo $start $length $linear $major:$minor $offset
    else
      IFS=" " read cstart clength crypt format key IVoff dev coffset <<EOF
$(dmsetup table "$map2")
EOF
      echo $start $length crypt $format $key $(expr $IVoff + $offset) $dev $(expr $coffset + $offset)
    fi
  done)
  if (dmsetup table $map; echo "$TABLE") sort uniq -c grep -v "^ *2 " > /dev/null 2>&1;
  then
    echo Diverting $map
    dmsetup reload $map <<EOF
$TABLE
EOF
    dmsetup resume $map
  fi
done


[ Update: There was a dmsetup resume missing in the script ] After running this script, you should be able to run cryptsetup luksClose on your LUKS volume. Once you’ve done it, pvdisplay and other LVM tools will show you nothing, because the physical volume will just have disappeared… but everything will still be in place. Note that if you leave the LUKs volume open, you’d better be careful NOT to do operations with LVM (such as pvmove) because it could break everything. Note that on sid, you would need to add a –showkey option into the $(dmsetup table "$map2") line. This option is necessary for dmsetup to display the encryption keys. I wonder if LVM upstream would consider supporting LUKS volumes directly… Anyways, back to the in-place conversion to LVM over LUKs. The idea would be to:

• Create a file the size of the LUKS headers + one block.
• Create a loopback device for this file.
• luksFormat the loopback device.
• luksOpen the loopback device, so that dmsetup can give us the key to use. I wonder if there are ways to get the key not involving the creation of the device mapper.
• luksClose it, because we actually don’t need it
• Create a device mapper that linearly maps the whole LVM physical volume (i.e the real partition, but without the empty tail).
• Remap the LVM device mappers so that they use our own device mapper instead of the partition as physical volume.
• Here is the trickiest part, and I still have to take a look at how pvmove does to know exactly how this can be done (I guess it will require an additional device mapper): move data chunk by chunk (offsetting with the size of the LUKS headers), starting from the end of the LVM physical volume, and encrypt at the same time. Once a chunk is moved, adjust our device mapper so that it maps to the linear unmoved data followed by the moved encrypted data. As I said earlier, this is a setup that works.
• Once everything is moved, we are in a situation where our device mapper is a single crypt mapping, in the same shape as a luksOpen would have made it. We can now copy the LUKS headers at the beginning of the partition, and add the appropriate configuration to /etc/crypttab.

If I got it all right, both LUKS and LVM should be believing they have done this setup themselves, so unmounting, lvchange, etc. should just be fine. It might be necessary to regenerate the initramfs, though. I’ll tell you when I fuck up my hard disk try it. To be continued…

Does anyone fancy hacking for an hour on a neat little feature for Sound Juicer, as a way of learning the code base a little? This shouldn't take more than an hour or so, and would be good for the world. When Musicbrainz can't find the track listing for a CD, it automatically queries FreeDB and will attempt to return something from the garbled nonsense that FreeDB generally contains. The user cannot tell the difference, unless they notice that the track listings are badly formatted, the album artist is incorrect, or the encoding is wrong. The user will not notice any trivial issues, fix any obvious problems and then rip the CD, or just give up now if the encoding is wrong. This is bad. What should happen is that Sound Juicer should subtly point out to the user if the data is proxied from FreeDB, so that they can import it into MusicBrainz where more data can be added and verified. What I'd like to see is a little pane appear if a CD is from FreeDB (SJ knows this already) with a link to the relevant import URL, so the user can quickly import it once for the benefit of everyone else who rips the album in the future. Does anyone fancy hacking this? As I said, it sounds like an hour of coding for someone who can code GTK+ but hasn't touched SJ before. NP: This Book Is About Words, Ahmad Szabo

It's now shortly before 5am (Argh!) and I'll be leaving home now. If everything goes as planned this will be my journey:

5:20 - Bus to station
5:43 - Train to Hanover
6:21 - Train to Cologne
9:21 - Train to the Airport (CGN)
10:50 - Plane to EDI
Around noon - Arrival at Teviot.

If I don't show up, something went wrong.

Bye bye for now ... *yawn*

Debconf7 continues very well, with a nice trip to a local sauna this evening. Unfortunately our party had to split into two, with the ladies going to one section and the gentlemen to another. Still it was fun, and I'm glad I went. It was probably just as well that Megan didn't get to see me being terrified of the cold water! Apart from that things went well today. I saw half a talk on Xen, sufficient to see my name in lights, then half a talk on security, again managing to see my name on the big screen. If people, at debconf7, would like to learn more about security I've volunteered myself on the "skills exchange" page of the wiki to demonstrate the process which Moritz described as "manual and complicated". (ie. releasing a DSA and updating the webpage.) I think this will happen sometime on Friday. I also need to track down AJ and talk about debootstrap work. One more thing, before I go to sleep. During the security talk Moritz did mention the idea of being able to grep through the source code of the entire archive. This is a topic which has been raised before. Right now I'm keen to make this possible, so overnight I'm syncing the latest sid archive and I think I have a plan to make it work.

• Sync the sources of the given distribution.
• Once that has happened recursively unpack any archive we can understand. (.tar.bz2, tar.gz, etc).
• Either:
  • Write a simple script with "grep".
  • or import the unpacked root into a text indexing system.
  • Also it would be useful to recognize common files via SHA + MD5 checksums.
• make some simple GUI.

I think that the sync will take a while for me on my home connection, and I also believe that the unpacking will be grossly CPU-intensive. Still it should be a worthwhile job even if I can't get it done, because it will tell us the kind of machine which is required to actually do it. I'd like to use a project machine because I'm not entirely sure I have the necessary space but that should become apparent fairly quickly. (The archive I can handle, the unpacked tress might be a little bit much for me. Still I have 100Gb free and I guess that is a good starting point). If anybody has any tips for full-text-indexers that would be appropriate for fast queries against a large directory tree of source code then I'd be interested in hearing about them.

The Gnu Pricacy Guard seems to have the default option of setting an expiry date when the signed key has such a date. My gpg key used to have such an expiry day when I went to the first keysigning parties (LinuxTag, DebConf 3). I have removed the expiry date later, but still a lot of signatures are now no longer valid: While I have signed 310 keys, my key is currently (pre-DebConf 7) only has 170 valid signatures.Tonight I wanted to find out what keys are these exactly, and after quite some trial and error I found a way to get that data, using only real old unix tools such as awk, sort, join:

gpg  --list-options show-sig-expire --fixed-list-mode --with-colons --list-sigs nomeata > my-sigs-col
cat my-sigs-col  awk -F: '$1 == "sig" && $7 != ""   print  '   sort -k 5,5 -t : -u > expired
cat my-sigs-col  awk -F: '$1 == "sig" && $7 == ""   print  '   sort -k 5,5 -t : -u > non-expired
join -v 1 -t : -j 5  expired non-expired > only-expired

Maybe I will use that list some time after DebConf7 to mail all these people, asking them to remove or update the expiry date from their signature.