DebConf launched with a bang the day I arrived by bike I was up until 3am meeting and greeting in the basement lounge of the Carmen Columbia dormitory, where I was staying. No idea how I managed to be so awake for that. The rest of the week alternated between hacking like crazy on code for my talk and spending a lot of time socializing with Debian folks new and old. For the day trip to Coney Island, I joined the dkg-led bike expedition which ended up running to nearly 30 miles, which was a bit more than expected. The fact that this was all in actual dense city really drove home the scale difference between Boston and New York (I'd never been to NYC before this). We took several breaks to lounge around and eat and drink, so it took quite a long time even given the distance. I hadn't planned on seeing the baseball game that was a part of the trip, but I ended up going anyway and it turns out that a bunch of geeks at a minor league game is actually quite a lot of fun! I hope someone else will put some pictures from the bike ride and game online soon, since I didn't really take any myself. This was the first DebConf where I gave a talk, which resulted in me skipping almost all of the other talks, because my talk was on the last day and I reaaally wasn't ready at the start of the conference due to the rest of life being pretty crazy this summer. I missed some things I would have liked to see because of this, but ultimately I think it was worth it. The good news is: it went well! I was nervous until I actually started talking (never given a talk at a conference before), and then it was fine. If you missed it, the talk video is on the web in low and high quality; slides are here. The audience was great there were excellent questions and people were excited and interested in the project. I couldn't have asked for a better reception. After the talk finished I spent some time aisle-chatting with some folks, and totally failed to recognize Joey despite having met him before, because he'd shaved off his hair. DebConf was, like usual, both inspiring and exhausting. I haven't managed to follow up on much that happened during the conference yet. I definitely plan to do so, though, now that real life is calming down again. I'd hate to waste the post-conference buzz about SD. My todo list includes:

• Working more on the SD debbugs bridge to make it more stable.
  • I ran into Jesse soon after coming back and now have a better idea of how I'm going to handle a lack of history properly.
• Getting my patch to the Debbugs SOAP interface merged.
• Looking into the read-write SOAP interface work that was done as a Summer of Code project.
• After talking with Jesse I also kind of want to hack up a RESTful interface that could be used alongside the SOAP interface. It seems like doing so will make development of and using the Debbugs web API less painful in the future. This may be a rabbit hole that I don't actually want to jump down, but it's an idea.
• Maybe other help on Debbugs proper!
• Fixing SD bugs and generating more documentation.
• Thinking about and thanking people for talk feedback!
• Playing around with monkeysphere for authentication on my personal machines.
• Watching videos of talks I missed (this includes basically everything that didn't have to do with bugtracking).

I got tired of having to authenticate as root to change printer settings like paper size or printout mode via a GUI. Since system-config-printer uses PolicyKit things like allow myself to change printer settings when logged into the system should be possible without having to mess with cupsd.conf or sudo, are they? Which printer related actions are governed by PolicyKit:

pkaction   grep printer

shows about the following if you have cups-pk-helper installed. The later is used by system-config-printer to interface with PolicyKit:

org.opensuse.cupspkhelper.mechanism.printer-enable
org.opensuse.cupspkhelper.mechanism.printer-local-edit
org.opensuse.cupspkhelper.mechanism.printer-remote-edit
org.opensuse.cupspkhelper.mechanism.printer-set-default
org.opensuse.cupspkhelper.mechanism.printeraddremove

So org.opensuse.cupspkhelper.mechanism.printer-local-edit seems to be what I'm looking for. Let's change the policy for this action:

cat <<EOF >/etc/polkit-1/localauthority/50-local.d/99-local.pkla 
[EditPrinters]
Identity=unix-user:foo
Action=org.opensuse.cupspkhelper.mechanism.printer-local-edit
ResultAny=no
ResultInactive=no
ResultActive=yes
EOF

This allows user foo to perform the action org.opensuse.cupspkhelper.mechanism.printer-local-edit if the user is logged into a local interactive session. The pklocalauthority(8) manpage has all the details. PolicyKit will pick up the configuration file changes on the fly. Lets see if this worked by getting the process id of the running system-config-printer and checking via pkcheck if the process is authorized to use that action:

PID=$(ps a   awk '/[p]ython .*system-config-printer/   print $1  ')
pkcheck --action-id org.opensuse.cupspkhelper.mechanism.printer-local-edit --process $PID
echo $?

If pkcheck returns with a zero return value everything is fine and user foo won't need any password to modify local printer settings from now on. Check the output of pkaction(1) to list more actions that can be setup this way.

DebConf10 ended 2 weeks ago, I m back in Bosnia and I needed some time to recalculate everything that happened to me in last month This was my third time in NYC, and NYC as a city never left me empty handed, in every possible sense! And am seriously considering option of moving back one day, whether to get my Masters/phD or just do work we ll see what future holds for me DebConf10 I was in organization team for DebConf10, but if you put me aside, what s particularly interesting is that there was a total number of 7 Bosnians on DebConf10, two of which were government officials. I must note this wasn t our complete team, and what s most important of all is that they all pretty much volunteered as much as they could, of course, besides good will to help Debian, but also to at least somehow prepare themselves for organization of DebConf11. It s also worth noting that this was their first DebConf, and I as a leader of the team must say I m proud on how well they performed. I m sure by being highly active in volunteering they have learned as much as they could when it comes to how it s done and were able to see good and bad sides of DebConf10 organization team. Of course, I d also like to thank DebConf10 team for their superb work, perfect Venue selection, their hospitality and being incredibly friendly and helpful to all of us. As I said in one of my lectures, it s kinda unwritten rule that every next DebConf has to top the previous one, and that being said it s going to be almost impossible to compete with this year bid, but being Bosnians we re up for the challenge My DebConf10 Pictures: Semi-private album FaceBook public link (no registration required) DebConf11 DebConf10 was busy period for all of us, but I don t want to start complaining since even though we ve done enormous amounts of work so far, the real challenge lays ahead of us. With DebConf10 behind us, the thing we should process now is it s our turn now, we re next Before anything, I d like to thank Government of Republika Srpska for their full financial support, which was one of the preconditions to even get this bid. With their full support + sponsors, we ll be able to get great amount for sponsorships, for you to come to Banja Luka from the most remote places, and also in case you need visa for them to help you in getting one. Government support is just too precious in any way you want to take it. We would also like to invite you to DebConf11 in Bosnia (Banja Luka) next summer, we also invite all of you to our communication channels (mailing list, irc) since we would really like to get involved as many people as we can. Besides all of you enjoying DebConf11 next year, we would like you to help us in giving directions since we re a young team Besides Debian people, I d like to invite anyone who think he/she could help, or want to get involved, especially individuals/organizations/LUG from whole region! It s also important to stress that in all of our communication means, we do speak English, so please do not hesitate if you have any sort of question/suggestion/whatever. For those interested in next DebConf11, most important page you should follow is our wiki page, where you can find all information and which we keep updated in timely manner: http://wiki.debconf.org/wiki/DebConf11/BanjaLuka Communication/Links Languages we officially support are: English, Bosnian, Serbian and Croatian Mailing list:
http://lists.debconf.net/mailman/listinfo/debconf11-bosnia IRC:
#debconf11-bosnia on irc.debian.org / irc.oftc.net Twitter:
#debconf11 tag
http://twitter.com/absinthesyringe (shameless self promotion ) Facebook:
Official DebConf11 FaceBook group DebConf10 Adnan Hodzic DebConf11 in Bosnia (Introduction and Invitation):
Lecture video high quality (205mb)

See you next year!

(this is a post in two parts) At the BoF of GPG Key Management: Best Practices, dkg had two immediate recommendations for improving WoT resilience:

• Regularly refresh your local keyring from the key servers;
• If you have spare space and transit, provide a key server to take part in the pool.

Regular Refreshes This is important because unless keys are refreshed, changed expirations and (more importantly) revocations on keys are never made known to you. Making regular refreshes is easy, though I was surprised to hear people had experienced difficulty with it it s a simple entry to cron, like: 0 0 * * 1 /usr/bin/gpg --refresh-keys to run at midnight on a Sunday, or any other convenient time. Append > /dev/null to make it silent unless there are errors. Provide a key server This is nearly as trivial a refreshing the keyring, but it does take some disk space (on my machine, 4G for the decompressed dump from another server, 7.4G for the database itself and a few hundred MB for other stuff. Supposedly the dump can go after a full database build, but as I m not constrained I haven t bothered). The sks package in Lenny and Squeeze manages the database and answers to clients. However, take care to follow README.Debian carefully, since if the permissions on /var/lib/sks or /var/log/sks aren t set right it will silently fail and much head-scratching will ensue. Building the database for the first time takes some hours even on a reasonably quick machine, and takes a lot of CPU time, so it s probably best done overnight. Once sks is running, it needs to catch up from the date of the key dump to now (which is why a recent dump is always a good start). To do that, it should be listed in the pool.sks-keyservers.net and therefore needs gossip peers. http://code.google.com/p/sks-keyserver/wiki/Peering is an excellent guide, though the version of sks in Lenny does not support multiple listening addresses for either the recon or hkp processes. When a small enough delta between the server and its peers is reached, it s included in the DNS pool for use by other clients.

A DebConf doesn't just happen in a few months. It's not even a year, it's closer to two or two and a half. I've only been through two cycles, and I think this process has happened for the last three or four DebConfs. This post will talk about a "brief" summary of the general DebConf timeline, with elaboration on all the bits later. This post describes the preparation of DebConfN in year 20XX.
At DebConfN-2 (mid 20XX-2), there is a talk where interested groups can describe their location and how it would be suitable for a DebConf. This consists of a short presentation and discussion, but not many details by this point. No decision is made here.Throughout late 20XX-2, teams develop their bid as best they can. The last two years there has been a nominal deadline of December 31 20XX-2 for submitting a bid, but like all DebConf deadlines it's more for guidance than a real deadline. There are various checklists and templates to provide guidance here, which I'll describe later.Early 20XX-1 brings a lot of work to finalize bids, and then the actual selection process. There is a burst of mailing list traffic where people ask questions and help each other refine their bids. There are several meetings where bid teams give their status and people can ask questions. In February or March, there is a selection meeting.After the decision meeting, there is a big round of cheers but people soon refocus on DebConf N-1 since it is more urgent. Ideally, most of the people involved in DebConfN help with DebConfN-1 both because DebConfN-1 needs help, but also to learn how to make DebConfN go well. The DebConfN local team keeps working in the background doing whatever is needed to prepare for their conference.At DebConfN-1, there is a talk where the DebConfN team presents their progress to the DebConf attendees and tries to build support.Late 20XX-1 brings the end of the previous conference and time for N's localteam to really start getting stuff done. Throughout the last part of the year, we try to work out enough details for registration to begin: final conference dates, locations, accommodation, etc.Registration opens in early 20XX. The exact dates for opening/sponsorship deadline/reconfirmation/etc all depend on how early in the year the conference will be. There are nominal deadlines published, but they usually slip. With our limited volunteer base, it's hard to make things go according to schedule, but things always work out.Around this time, the fundraising team really starts working hard. Also, the travel sponsorship team needs to meet to decide who gets funded. The talks team needs to issue a call for events, read, and select them. The video team needs to make sure that they will be ready for the DebCamp setup period. There are many more things I'm forgetting, too.In the months before arrival, it becomes kind of crazy from a local perspective. There are many things to get organized before people arrive, mainly focusing on food, accommodation, and venue space logistics.Once DebCamp starts, things get even more complicated, but we have a flood of experienced DebConfers from around the world come in, and they all know a bit and can help out. DebCamp is effectively the DebConf preparation week to the organizers. DebCamp becomes DebConf, and much bigger, once all attendees arrive.Eventually, people leave and everyone goes home. This doesn't mean local stuff is over - I'm still engaged in DebConf10 wrap-up, making sure everyone gets paid and everything is returned.Mid to late 20XX brings a brief period of rest for most people.There is one last thing to do, though: the DebConf final report. This describes the events of DebConf, and is very useful for getting sponsors. It is also a valuable reference for future years looking back on the lessons learned. You can browse past final reports in our archive . Because everyone is tired, this too frequently gets pushed off until early 20XX+1, when DebConfN+1's fundraising team needs it. A frantic rush finishes up the missing pieces and it's done.
So, there you have it: a rough outline of the timeline of a DebConf. It extends from mid 20XX-2 to early 20XX+1, more than two and a half years. This post nowhere near describes all the details that go into DebConf, almost any of the paragraphs above could be made it's own post. Stay tuned for them.Tags: debconf, debian

As you may remember, our co-op is working on various RFID (radio tags instead of barcodes, basically) extensions for Koha. One of the main ethical concerns about RFID is privacy if done wrong, it could become quite easy to see what books from which libraries someone has in their bag, without their consent and without physical contact. If borrower cards themselves become RFID-enabled, you might even obtain their personal data pretty easily, although I m not aware of any libraries in practice who put any personal data onto the borrower card tags yet. On 12 May 2009, the European Commission Recommended privacy and data protection in all RFID applications. That was followed by EU Mandate M436 for the European Standards Organisations (ESO) to develop standards to support that recommandation. Phase 1 started last March and has delivered a document RFID-DTR07044v006-draft (PDF). Phase 2 will build on that to produce formal standards for signage, privacy impact assessments and so on. EDItEUR are replying to that draft on behalf of the library sector and have sent me their draft response RFID_LibrarySectorCommentsDTR-07044-1 as a PDF. It s worth opening those two side-by-side to read through. If you can help improve the combined response, please send feedback to the email address in the Library Sector PDF by 6 September. If you d like to make your own response, you have until 15 September to reply to the ESO PDF using their template. By the way, does the ESO PDF look jaggy to anyone else? It looks like ghostscript can t anti-alias the text.

My project. This part of this post assumes that the reader has read my last post. The Summer of Code is over. It was great to spend time working on GHC and getting money for it. Although the implementation is not mature enough to be included in the repository, I m happy with the state it is now. I think it s a good start, and I plan to keep working on it. It s good to see how my motivation increased now that the program is over, and that I m free to not work on it if I want to. So I m going to take a look at it again, as soon as I do the stuff I was postergating during the program. I ve created a wiki page for it, and my plans are to implement and measure the Remove partial lists patch, and then to debug Allocate in lines in minor GCs . Any help is very welcome.

People often want to know what an application looks like before they install it. This is one of the reason why Christoph Haas started with the screenshots.debian.net service. People can upload screenshots for their favorite applications for others to take a look at. Finally, as convenience feature, they are now added on packages.debian.org. When you click on the screenshot of a package you can see the list of all of the available ones. In the case when no screenshot is available yet the page will show a placeholder image which gives you a convenience link to the screenshots page of the package where you can submit one for the benefit of all. Please make sure that you are following the guidelines for submissions, otherwise your upload might get rejected. On a not totally unrelated topic, the packages.ubuntu.com site finally will also show you packages in maverick. It doesn't show screenshots yet if you think adding support for screenshots in there would be a good thing too, please let me know! Enjoy!

/debian permanent link Comments: 5

At DebConf10, there was a BOF entitled "DebConf & Debian", discussing about the similarities and differences of Debian and DebConf. At times this became a little bit heated as people wanted more integration and transparency.I don't think the differences are that great. I don't even see them as separate. I see DebConf as a Debian team, one that has to release every year on a short time frame. Things are rushed, things are not perfect, but we welcome anyone who wants to help.It's my goal to start a series of posts on DebConf, in the spirit of other teams documenting themselves and calling for contributors. I hope to let people know what's the current state of things, how you can get involved.If you want to get involved, join #debconf-team on OFTC, or subscribe to the debconf-team mailing list at http://lists.debconf.org/ . There are many people who would like to help you get involved. Right now (post-debconf) is a bit of an idle time, but one of the few times that big changes can be made. DebConf is a do-ocracy, if you'd like there to be a change, the best way is to help implement it. There may be some resistance, but we (at least I) will help to find a solution.These posts are my take on things, I can't claim to speak for the DebConf team. I don't know all of the history of DebConf, and I may get things wrong. I'll run these posts by the rest of the DebConf team for suggestions/corrections, but there may still be things wrong. You should consider anything useful to be thanks to the DebConf team, and anything wrong to be my mistake.I've only been involved in DebConf for two years. I first got involved as part of DebConf10, but spent most of 2009 working on global management things for DebConf9. This year, I spent more time with local on-the-ground tasks. I'd like to keep being involved next year - and hopefully keep improving, with your help.Tags: debconf, debian

When Lars and I started work on Branchable, the Ikiwiki-based wiki and blog hosting service that we first discussed a year ago, and launched two weeks ago, we didn't know what software to use to build it. Probably, we assumed, we'd need a relational database. (Or maybe Couchdb, we wondered?) To manage the sysadmin jobs of keeping a slew of Ikiwiki sites running, we investigated using Puppet or Chef. (We worried that scaling Puppet or Chef to handle an arbitrary number of ikiwiki sites could be a problem.) For the user interface frontend, we assumed we'd be writing code in some kind of web application framework. Nearly all of that conventional wisdom turned out to be false. Instead, all the frontend code for Branchable runs as part of Ikiwiki. www.branchable.com is just another Branchable site, though one with some custom plugins that can make other sites and display a user's control panel. Other plugins handle things like the DNS setup interface. Those plugins are part of the first public site hosted on Branchable, ikiwiki-hosting.branchable.com. That wiki's git repository holds all the code, as well as the wiki for documentation and bug tracking. Also included in ikiwiki-hosting are some backend programs, like ikisite, which handles the high-level site management (creation, deletion, branching, etc), and is what we used instead of Chef. (Though we may use Chef for server deployment later.) Rather than use a relational database, we did out own little noSQL thing, by using a special, locked-down wiki as our database. That wiki is so special-purpose that it is not even accessible via the web! But it's managed by Branchable just like any other site. Deciding to not use a database was a watershed moment, because at that point we were fully dogfooding everything for Branchable. It's worked out well so far, possibly because our database needs are minimal. We have over a dozen other Ikiwiki sites in Branchable that we use either internally, or for other special purposes. Like this parked site that is branched every day to refresh sandbox.branchable.com. So, no database, no Puppet, lots of dogfooding. Did we use a web app framework? Not really. The closest we came is when developing Branchable highlighted that Ikiwiki needed a more user-friendly openid login interface, so I went off and adapted one, and of course added it into Ikiwiki proper. And that uses the JQuery framework. (I might write sometime about all the other Ikiwiki improvements that Branchable spurred.) We've done a lot of eating our own dogfood at Branchable, but I think it's all been pragmatic, not due to Not Invented Here syndrome. And the result is that Branchable is increasingly flexible in what its sites can do, and runs on a very simple architecture, with no odd special-cases for database servers, etc. It's just a bunch of Ikiwiki sites that work together.

I have some mp3 files in my collection. Some years ago, back in the times, when I was a Windows-User, I used "The Godfather", to keep the chaos under control. This software, although not Open Source, is really good at what it does. And it does almost everything to organize mp3s. From (mass-)tagging to auto-renaming to auto-sorting your mp3s. But this wouldn't be a blog entry by me if this became... a praise to a windows software so..

When I became a daily Linux User I searched for alternatives. With GUI or without a GUI, but I did not really find an application that suited my needs. As far as tagging and renaming was concerned it wasn't that hard. There are really excellent command line tools (lltag, id3, etc.) and rumours have been heard that there are also GUI tools.
But what I still did not find is a simple, yet flexible, tool to sort a huge collection of mp3s into a flexible structure on the filesystem like "The Godfather" was able to do since.. ehh.. a lot of years.

So the day before yesterday I finally decided to write one on my own and came up with an ~ 280 lines perl script (POD-documentation included) which does exactly what I want and is simple.
It does no tagging.
It does no renaming.
All it does is sorting mp3s into a given template-based directory hierarchy based on their ID3 tag.

At this point a warning is due. I'm sharing that script with you under the terms of the GPL. BUT it still needs testing and therefore you probably do not want to use it without its safety measures (e.g. dry-run or copy instead of move) to avoid data loss. And notice that although it has been written with portability in mind, it has only been tested on a Debian GNU/Linux system.

The script is hosted at github. Here (or raw to download it directly).

After I've talked to a co-worker about the script, he told me that arename would do what I want. So you probably don't want to use my script, because arename is probably more tested and much more sophisticated. But on the other hand I had a quick look at the arename manpage and its so utterly feature-loaded, that it cannot avoid a certain complexicity. My tool is simple.
And there is another advantage. If you want arename to handle a certain amount of mp3s you have to use your shell magic to find the files and pipe it to arename. My script finds all mp3s recursively from where you let it start (default is $PWD so be careful) and will happily move it into a hierarchy under a directory you ask it to. For the simple job of sorting mp3s its probably easier to use.

(Oh and if its worth nothing, it might still be of use as a simple programming example, how one could solve this problem in Perl)

At some point I had to debug an issue with some code I worked on in the past. It was using OpenLDAP to connect to an Active Directory server to get some information. At some point I got a report that the authentication failed with an ugly error.

Username is stored. Authenticating as domain\user.
Enter password:
ldap_search_ext: Operations error (1)
additional info: 00000000: LdapErr: DSID-0C090627, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, vece

I looked for the meaning of that error and after some staring at different possible explanations, I ended up on a page dealing with some python-ldap code. My code was in C, but it was clear it was the underlying library issuing the error message, so I looked at what it said:

Normally, this error indicates that you're attempting to bind anonymously, which Active Directory (sensibly) doesn't allow by default. We were supplying credentials to bind, though, and changing the base DN on the search to a sub-OU was all that was necessary to get the search to work. It turns out that python-ldap was binding anonymously, so the error was only sort of a red herring.

This was really strange because the authentication was actually done, as it was obvious from the messages and the traffic (analyzed with wireshark). Later in that post there were some hints that indicated that parts of the data might be stored on another server and the suggested fix was to instruct the library not to try chase referrals.

ldap.set_option(ldap.OPT_REFERRALS, 0)

I tried to see what was going on with our server using a ldapsearch command and at the end of the output there were some referrals specified.

[..]
# search reference
# refldap://ForestDnsZones.domeniu.ro/DC=ForestDnsZones,DC=domeniu,DC=ro

# search reference
# refldap://DomainDnsZones.domeniu.ro/DC=DomainDnsZones,DC=domeniu,DC=ro

# search reference
# refldap://domeniu.ro/CN=Configuration,DC=domeniu,DC=ro

# search result

# numResponses: 5
# numEntries: 1
# numReferences: 3

Bingo! So, after looking at the man page, I added this bit of code:

+ /* do not chase referrals */
+ if (ldap_set_option (ld,LDAP_OPT_REFERRALS,LDAP_OPT_OFF)!=LDAP_SUCCESS)  
+  ldap_perror(ld,"ldap_set_option");
+  return NULL;
+  
+

And then it worked. I hope this helps others that might be in the same situation as I was.

Having swapped details with many, many people at Debconf, and then been away for a week after that, I found myself with an overflowing mailbox and a long task of open mail, provide pass-phrase, pipe to gpg import . I wanted a way to batch-import all these signatures (there are three times as many, because my key has three UIDs) in one or two goes, and tidy up the stragglers later. David Bremner wrote a small Perl script to do this from an mbox file, but I wanted to work in pure shell and with mutt. Just shoving the mbox at gpg resulted in it decrypting one message, then bailing at the fact the IDEA plugin is not present. Here was my eventual workflow, which only requires you to provide the pass-phrase once:

1. create a maildir, either with maildir-make or a directory with cur, new and tmp directories nested inside;
2. mark all relevant messages as read, and save them to here (it doesn t matter if others get caught up in it);
3. now change to the maildir /cur directory, and run the following bash (disclaimer: totally untested and used at your own risk):
   
for a in ls ; do mv $a $a.gpg; done
gpg --decrypt-files *.gpg
rm *.gpg
gpg --import *
rm *

I expect there are better/quicker/safer ways to do it, but this worked well for me at midnight on a Monday evening. 19/08/10: Yes, it turns out I am a numpty, and Mutt can handle this all by itself with Ctrl-K and a tagged list. This is still quite handy when the private key is not on the machine you re using to read mail, though. Thanks for the corrections.

This LWN article is perhaps a bit confused, but it does point out a truth: On modern hardware, with fast, local storage, and large, often static files, rsync is often unncessarily slow. That's probably true to some extent when using rsync across a LAN, but it's especially true when using it to copy files locally. rsync runs as a client/server pair, and both processes MD5 checksum the files as they are being transferred. That's nice, but it's slow too. Funny thing about rsync is that probably 50% of uses of it don't involve the core feature that it was written to provide: Updating files by transferring differences. Which, other than ensuring data validity, is the only reason it needs slow checksums. Instead, rsync is often chosen because of all the other awesome features that were glommed onto it over the past several decades. Compared with rsync, cp is pretty laughable -- it can't even exclude files from being copied by patterns. And unlike rsync, cp command lines are not often developed by repeated trial and error -- cp does not recover well from being ctrl-c'd in the middle, while rsync does. These kinds of things make a lot of us reach for rsync first, even if the situation does not involve incremental file changes. In most any situation, one of rsync's 120-some options is sure to be just what you need... So lots of scripts use rsync to synchronise directories, but the amount of speedup obtained by using the rsync algorithm is often low. (Correction: rsync turns off the delta-transfer algorythm by default for local to local transfers. It still does md5 checksums however.) Since rsync is reasonably fast, we generally don't care that it does these checksums that probably on average slow it down. But if larger files, like videos, are involved, this starts to change.. When rsync is run on a typical home NAS, with a slow (arm) CPU, the picture changes entirely -- now the checksum overhead is unbearable, while the IO overhead is minimal. So, here's local-rsync. It takes all the same options as rsync, with the caveat that SOURCE and DEST must be the first two options, and must be local directories.

% local-rsync huge/ /media/usb/huge/ -a -v --exclude '*~' --delete --max-delete=100

It speeds up rsync in these types of situations, by querying it to find what files need to be updated, and updating them the brute force way, with cp. At the end, rsync is run, to take care of the non-brute force stuff (like deletions and file permissions). On a fast CPU, local-rsync will probably speed up rsync of large, static files by a factor of 2. On a slow CPU, local-rsync is so fast, and rsync so slow, that I have not bothered to benchmark. :) This is just a hack (easily replaced by a --no-checksum option in rsync, of course). But I think it illustrates some interesting things about how a program's underlying assumptions about its environment can change over time, and how free software programs can accrete value until the original differentiating reason for their existence is not the most important thing about them anymore.

A while back I mentioned that I was going to be updating and overhauling the blogspam.net service. That process is now almost complete. A couple of nights ago I overhauled the website, and today I've finally commited my last (planned) change to the repository for the purposes of migration. I started reworking the code a week or so ago, but as of this evening the code in the repository is the code the server is actually running. The previous codebase was functional but a little hasty - and was implemented before I switched to per-UID server-hosting - so there was a need to clean things up and make sure permissions and similar niggles were checked. The new, modular, codebase requires no root access, and will store all state (logs & transient caches) in a clean extensible fasion. The code is also much more flexible making use of Module::Pluggable rather than Class::Pluggable. This allowed me to overhaul the API of the plugins (primarily to add an expire method such that each plugin has a well-defined means to expire any state they may maintain). Module::Pluggable is a great module - allows me to treat plugins as first class objects, which wasn't the case with C::P. Since all the code behind the service is Perl it is also now available on CPAN in addition to the mercurial repository where it is developed.. I see that the server is getting pretty popular these days, used by the likes of embedders.org, publiclive.com, & etc. It doesn't hurt that ikiwiki, identi.ca, and other people include support in their distributions these days. Me? I mostly use it on debian-administration.org where it does a great job. ObQuote: What's the name of that thing that if I eat it real fast, it's free? - Whip It.

Since I didn t actually come home from Debconf, but instead took a further week s holiday and lazed around by the beach, I haven t sat down to write anything about it yet. Most importantly, I should say thank you to Joey Hess, Steve McIntyre and Stefano Zacchiroli for arranging the Debconf Newbies programme, without which I would have been stuck at work watching the videos again (though this does have the advantage of seeing three talks at once). Every year I am astounded at the generosity of certain companies who sponsor the conference, since I ve experienced for myself the difficulty of trying to put a corporate price on free software even more so to specifically sponsor newbies, who by definition probably aren t as active in ways that interest sponsors than more established developers. Right up to the day I flew I was nervous (as in any social situation) at meeting so many new people in one go, on my own, and long way from home after all, it s not really easy to back out and go home after that But in fact, everybody I met was warm and friendly, and very supportive in all sorts of ways. I got to see and hear many talks; in the first two days alone:

• Supporting Debian machines for friends and family
• Debian at a large Dutch university
• Why is GNU/Linux Like a Player Piano?
• Bits from the DPL
• Packaging Python modules and applications
• The Unity desktop environment
• Collaboration between Debian and Ubuntu

And of course social events, like Cheese and Wine, Keysigning (handy for putting faces to names), many nights in the third hacklab, a day trip to Coney Island, a privileged viewing of the most nerdy organ in the world, and a trip up the Empire State building: The campus was ideally suited to such a conference, and apart from the mugginess we had excellent weather. I discovered that I rather enjoy watching baseball and that one should repeat the application of sunblock after paddling. But what I enjoyed most of all was being in the same area as so many eminent Debian people, and being so inspired by them to be even more involved technically and socially. I hope to see you all again soon!
<script type="text/javascript"> var flattr_wp_ver = '0.9.11'; var flattr_uid = '32970'; var flattr_url = 'http://www.jwiltshire.org.uk/content'; var flattr_lng = 'en_GB'; var flattr_cat = 'text'; var flattr_tag = 'blog,wordpress,rss,feed'; var flattr_btn = 'large'; var flattr_tle = 'jwiltshire.org.uk'; var flattr_dsc = 'The public face of jwiltshire'; </script> <script src="http://api.flattr.com/button/load.js?v=0.2" type="text/javascript"></script>

A while ago I was with my SO at a concert of a band I might cover at a later point. This time I want to cover the support act of that concert: Jerboa. It was a bit weird to see a single person on stage turning some knobs, but he quickly caught both our attention and heart. And he was a charm to talk with afterwards when we bought his CD. He's definitely worth your attention, and if you are able to see him live take the chance for it.

• Number 1: The obvious number 1 song in the list. Perfect groovy song, great video too.
• Running North (remix): Also interesting output of his creativity.
• Just Another Number: It starts off pretty smooth but then really kicks in. Hard to not headnick along to the song, it is far away from being Just Another Number.

Actually I would have loved to include What if in the songs but unfortunately that one wasn't available so I had to look for a replacement for it. Enjoy!

/music permanent link Comments: 2

It's been a couple of very interesting days. The thank.debian.net site was quite a success, and a lot of credit is due to Valessio's awesome balloons. Even though I didn't do much marketing about it, the site was linked on Slashdot, LWN, Ubuntu's and Debian's website, Planet, the BTS, and quite a number of blogs. We received almost 3000 thank you messages, from all over the world. Even though the site was in English, many people felt inspired to leave their thank you messages in their languages, if you browse over the site, you'll find messages in Turkish, Chinese, Russian and many other languages. On that matter, I was particularly amazed at this message in persian, which displays properly even inside mutt. Not too surprising, one of the things that people most value of Debian is the package management. The apt team got a lot of thank you messages, and many of the ones directed to the whole community also mention apt and upgrades as something very valuable I find it very interesting that many of the messages, came from Ubuntu users, stating that even though they use Ubuntu, they really value the work done by Debian. A number of posts also mention sidux, knoppix and mint among the many derivatives. Many people stated the many years that they use Debian, some of them have just started, some have been using Debian for quite a while.
I was happily surprised by a number of posts that included a thank you message to "guys/gals" or "guys and girls". And the image used in the "hackergotchi" of this message really brightened my day. Finally, I'm sorry that some people were annoyed by the thank you messages. For next year, we can try to do something a bit different and hopefully not annoy those who were annoyed this time. I do think that once in a while it's nice to be able to receive some of that much love that the users have for Debian and we rarely get to know about.

Today's fun idea was to put all my contacts stored into BBDB on a Google Maps' map, using my Google Maps extension for Emacs. With the help of a few lines of Lisp glue:

(google-maps-static-show
 :markers
 (mapcar
  (lambda (address-entry)
     ((,(concat
         (mapconcat
          'identity
          (elt address-entry 1) ", ") ", "
          (elt address-entry 2) ", "
          (elt address-entry 3) ", "
          (elt address-entry 4) ", "
          (elt address-entry 5)))))
  (mapcan
   (lambda (record)
     ;; We need to copy the returned list, because mapcan will modify it later
     (copy-list (bbdb-record-addresses record)))
   (bbdb-records))))

we can make that:

It's really simplistic, but I did not need more to have fun. :-) This could be extended to set a specific marker and/or color for each contact, with a legend. I'll let that as an exercise for my readers.

An old house, theoretically empty, but still filled with the cruft of its history. Here an old classroom-style pencil sharpener mounted on a wall. In a kitchen cabinet, a plastic bag of dried fish of unknown provenance. Old books. Somewhere, a moldering, out of tune piano. How does this make you feel? Maybe depressed, or urgently needing to wipe it all clean and repaint? Myself, as long as the previous inhabitants are people whose heads I don't mind getting into, I like this environment. I will carve out my own part of it, clean and reorganise that, but generally leave the rest as-is.

---

It's weird that, if we were talking not about houses, but software, I'd probably bulldoze it down, compost it, grow trees from it, and start over.

---

This house has the added complication of a solar power system that has accreted broken parts and workarounds over the years. So far I don't understand the original design, let alone what it developed into! I'm spending a few days here now just to try to learn how it behaves -- how long can I run my laptop and forgetfully leave lights on before the battery bank reaches unsafe levels? Am I even measuring the level of the right battery bank? (Why does this incoming solar power display still read 21, in the middle of the night?) I haven't been in this situation since spring of 2001. And then I only had to contend with no running water, and dialup. Now I also have to deal with limited power. Interesting times.