After (again) searching for some good photo album generator, I'm more and more inclined to writing yet another tool to do this. Maybe I just want too much from this, but I haven't found solution that would fit my needs:

• Generate static HTML (or XHTML).
• Design should be done using CSS.
• Ability to process whole tree of photos into single album with directory based sub albums.
• Generate data only for changed/new photos (timestamp checking).
• Support for including EXIF information in results.
• Clean and nicely looking default template.
• Support for image comments from EXIF, JPEG comment field and external file.
• Automatic generating of images suitable for web browsing as well as including links to full sized images from camera.
• Somehow active development. I fear of using code which has not been updated for 3 years.
• It should be fast. This excludes anything written in bash as spawning tons of external programs can not be fast.
• Written in some language I know. I'm pretty sure I will hit some bug and I would like to be able to fix it. In fact only suitable language for me seems to be Python.
• Command line interface is a must.

So far I'm using a bit customised Matew which really does not fulfil everything, but at least works reasonably good (but extremely slow). However the more and more pictures are in my album, the slower this beast seems to be, so it's probably time to switch to something new. So far I haven't found anything that would actually fit, but PyAC seems to be usable as starting point for my new project for generating albums. Or is there a working solution?

And no, I don't mean Mao.
How was Thursday 23 for me?
6:30 Wake up, knowing it will be a long day. No, don't get those delicious extra 15 minutes of sleep.
8:00 Arrive to work, ~90min earlier than usual, to finish the Rails app you have to demo at 10:00
10:00 Meeting with the Institute's Director, Academic Secretary and Technic Secretary to show them the system for following the internal inter-tematic colloquium information workflow. Thankfully, they were happy with it.
11:20 (20 minutes after scheduled): As several academics have said they are interested in getting to know Linux, I'm giving a series of introductory sessions on Debian to them. So, I went with some of them through the Etch installation. Of course, mostly everything was fine. What was not? Resizing a Windows partition is still a step that requires several jumps through hoops (i.e. booting Debian LiveCDs, some console-based interactivity which I wanted to avoid...), and... Well, d-i does not time out if the selected mirror is not responding. Will file a bug, yes. Anyway...
14:30 Managed to leave the lab working, with no damages. Went to the bank to check some data and to have a quick lunch.
16:00 Get back to the institute. Jos Luis, from Difusi n y extensi n was already waiting for me to show a set of components for the Comas-based system we use for tracking attendance to the Institute's activities
16:45 The people from AMECIDER arrived so I could show them my proposal for their new website. Can you believe most of the websites I host are still completely static HTML, and, what's even worse, generated by the $ despiseful_adjective Dreamweaver? Yes, I am proposing them to move over to Drupal
17:30 I finally received another person, one of the people that's attending my Thursday lessons, wanting me to help with some Linux questions. What's Linux's killer app for my Institute's researchers? Believe it or not, a not-brain-dead PDF reader. They are amazed that, using Gnome's Evince, they can just copy-paste bits of PDFs into their documents (and the fact that pasting tabular data into OpenOffice's Calc allows for an easy import feature - Yesterday I spent ~1hr with this researcher importing data generated by the International Monetary Fund into his work), and they are even more amazed with the gnome-screenshot they can directly include graphics from such documents without redrawing them.
19:45 (45 minutes behind schedule): Went to the Astronomy Research Institute, to meet with Sergio, H ctor and Eric, to decide on bits of the implementation of the new gear we will be putting in nisamox, our dear and faithful ftp.mx.debian.org mirror. Thanks to everybody! We will soon tell you what we got and what we did with it, it's mostly settled by now.
20:30 Had the first of some... Four calls regarding yet another system I have been working with for the ICME conference, to be held in Mexico in 2008. Several pushing details that need my attention and work, bah :-/ Have to work on them...
21:00 Got home. Did a couple housekeeping activities (i.e. take out the trash, buy some food...). I was too burnt out to start making my bags for going to Vienna (I'm leaving tonight!), work on the pending systems I have to work on, or even stay awake.
Leaving work for a week to go a conference is always fun and great... But man, it's stressing!

It's been a while since I've posted here; I've had a busy two weeks, and posting on my blog hasn't been a priority. Something I should do more often. Of course, as time backs up it seems harder and harder to actually get around to posting. I wanted to bring the attention of my friends and colleagues, though, to a notice that Maj put on the Wikitravel shared logbook. As of 15 August 2007, Maj is no longer working on Wikitravel. In addition, my time has been curtailed quite a bit, and I'm concentrating only on community and content issues -- no more programming or sysadmin work. This is all coming about for several reasons. The main one is that Internet Brands, the company that now owns wikitravel.org, wanted to have their own staff do the programming and sysadmin work. The end of our service contract seemed like an ideal time to effect this change. On our side, it's been challenging for us to work with a large organization not specialized in Open Content and wiki sites. We've gotten along really well with the IB people, but we haven't always seen eye-to-eye on the best way to grow and develop the site. As the decision was made to remove us from our technical roles, Maj decided it was a good time to opt out entirely. I'm going to miss her tremendously. As most people reading this already know, we started the Wikitravel project together, and over the last 4 years it's been a constant important part of our family life. Maj's information-architecture skills and dedication to travel have been crucial to the development of the site. She's been a strategizer, a spokesperson, a content organizer and a content builder. Probably most important for the future of the project, she developed almost single-handedly the Wikitravel Extra personal experience component of the site. We've got a lot to keep us busy, though. We recently launched Keiki, a project to create an Open Content parenting guide. For myself, I've also started Vinismo, an Open Content wine guide (sensing a pattern here...?). And we'll be working with our partner Jani P. on Wikitravel Press, an independent project to publish printed travel guidebooks based on Wikitravel articles. Despite the shakeups, I anticipate that the future of Wikitravel will be good. We have an incredibly strong and smart community, dedicated to an important and noble purpose. I'll remain involved on a part-time basis. We've got a lot of traction in public opinion and great Web traffic. I don't think that IB is going to make any bad decisions about the site, but if they do there's a local and global community of people that take Wikitravel and Open Content travel guides extremely seriously. I think things are probably going to be OK. tags: wiki wikitravel opencontent maj work ib

On the subject of wiki The South by Southwest 2008 Panel Picker app is up for public review right now. SXSW Interactive, in Austin, is one of the best tech conferences of the year. I've got a panel proposal up, on Wiki 2.0 -- I really want to talk about some of the interesting new wiki apps, new wikis, and new wiki business models coming up. There are a couple of other wiki-related panels proposed -- including one on commercial wikis, which I presented about last year (Talks/SXSW07). Folks who are interested in wikis, or in SXSW, should take some time to give their feedback on the proposed panels and improved the level of discussion for 2008. tags: sxsw wiki vote panel sxsw08

[Long post, sorry. If you're short on time skip to the end for the juicy parts.]

It all started about a week ago when I decided to find out how many files were unique in the whole Debian archive, and how many were duplicated in the same package or in other packages. I had done some work on duplication detection before, and I knew that the process involves getting some kind of hash value of every document, then finding duplicate values in the list of hashes.

I already had a local Debian mirror so the raw data was all there. I basically had two options: either compute the hash value of every file myself from the packages in my mirror, or find some other (ideally faster) way to determine if a file is unique. I quickly decided that the best option was to use the md5sums embedded in most Debian packages (in the DEBIAN/md5sums control file). That would give me MD5 hashes of all regular files, excluding conffiles.

So the first step was to check how many packages have embedded md5sums, and a simple script showed that less than 3% don't have them. This first check exposed a bug in python-debian, which was duly reported. Along the way, my post prompted a discussion on debian-devel about the state of md5sums, and I set up a daily check to keep track of things.

The next step was to make sure that all md5sum-enabled packages had usable information. It turned out that python-debian choked on 103 packages because of embedded spaces in filenames, and that it also had a blocker bug when used in Python 2.4. I filed two more bugs.

All that remained was the easiest part: write the program to find duplicate files. I did that, and I now have all kinds of funny statistics:

• The dataset consists of 20170 packages with md5sums, shipping a total of 2069830 files. That gives an average ratio of 102 files per package, excluding conffiles.
• There are 113732 duplicates in the archive. 1556 files are duplicated more than 10 times, and 14 files are duplicated more than 300 times.
• The empty file is present 8325 times in the archive, spread over 874 packages. This isn't surprising since it's used for all kinds of purposes like Python's init.py files, Perl's .bs files, etc. I also learned (among other oddities) that the python2.4-doc package ships a few zero-byte .png files. Uh?
• Also popular is the file with just one newline character in it: 343 occurrences. In the same vein, we have 461 occurrences of the "deny from all\n" file.
• Most of the hits are for Doxygen images in -dev and -doc packages, namely doxygen.png, tab_b.gif, tab_l.gif, etc (about 350 hits each). In the same category, gjdoc CSS files (149 hits).
• The partlibrary package is our worst offender. It ships a total of 9680 non-empty files, and only 4833 of them are unique. 6 files are duplicated more than 400 times each in the package.

If you're still reading so far, thanks! You can see the report file, and the program itself. If you want to run it, note that it's optimized for speed rather than memory efficiency; it runs in under 3 minutes but uses up to 1.5GB of memory (my home desktop has 4GB).

The grand conclusion is that all things considered, there is very little file duplication in the archive: the #1 duplicate represents less than 0.4% of the two million analyzed files, and it doesn't actually use any space since it's an empty file... :)

My supervisor pointed out a very annoying habit I had formed when writing technical documents, best summed up below:

"It would be difficult to find a page of governmental, military, or academic writing that doesn't have on it the word utilize. It must be one of the most over-utilized words in the world. It seems as though people out to impress people with the significance of what they're doing use utilize when they should use use. Utilize is not an elegant variation of the word use; it has its own distinct meaning. When you utilize something, you make do with something not normally used for the purpose, e.g., you utilize a dime when the bloody screwdriver is nowhere to be found. If the screwdriver were there, you'd use it, not utilize a stupid dime for the purpose. Use use when you mean use, and utilize only when it's properly used to mean--to use something not normally used. The computer went off-line, so they utilized Mr. Wang's abacus, the one he liked to use. Despite the temporary breakdown, the computer's use-rate was up (not its utilization-rate)." Cheney, "Getting the words right" (1983)

You will now probably notice this subtle verbiage in most everything you read!

I had a pretty good week last week in terms of doing some open source work. I had the opportunity to do some consulting regarding Hecl, which was pretty cool. Some of the work was fed back into Hecl itself, including the beginnings of a "media" API (video streams, taking pictures with the camera), and some other fixes and updates. I've also been working on it a bit on my own time, adding a Canvas demo to the MIDP2.0 demo application (works in the emulator, haven't tried it on a phone yet), and today I'm going to look at updating some of the documentation that's been lagging the code. In the course of my work, I also happened on a cool idea to speed up the launch of Hecl applications: by creating the widgets as regular widgets in Java, right away, in the startApp MIDlet method, it's possible to get them on the screen quickly. At that point, you can start loading Hecl, and create some Hecl objects to hold the already created Java widgets. It goes something like this:

Form mainform = new Form("Initial Form");
TextField search = new TextField("search:", "", 30, 0);
... load script ...
interp.setVar("mainform", ObjectThing.create(mainform));
interp.setVar("searchwidget", ObjectThing.create(search));
... evaluate script ...

This is a good tradeoff, because it gets something in front of the user right away, and it's not a throwaway like a splash screen. On another front, I've updated some of the code in DedaWiki, to enable email notifications in the comment system, which has percolated into the Linux Incompatibility List, and, with a bit of tweaking, into Squeezed Books. I don't think the code is quite ready for a real release, but by running the incompatibility list on it, it's a great way to give it a thorough shakedown, and I'll attempt to release it sooner or later. Of course, in the meantime, interested parties can get the code out of subversion on rubyforge.

As many readers of this blog know, I tend not to shy away from writing about my mother s struggle with Alzheimers. But in the last 6 months and especially since I paid my last visit to my mother, I have found it much harder to sit and write about her current state of affairs. If we think of Alzheimers as a journey to a new place, she is almost at the point of reaching that place of great loneliness and inhospitality, which is not only new and different both for her and those around her but is a virtual prison, for it rarely allows you to leave and visit the places of your past. More than ever, her memory and understanding of her life as she (as I and many others), once knew it, is fading fast; Of course, this was to be expected but it is quite difficult to imagine what it will be like until the actual experience unmistakably knocks on the door of present time and unfortunately, when it knocks, you can t do anything but open the door and let it in. Unsurprisingly, it is incredibly difficult to witness and interact with a person who is losing most all recollections, the stuff of which, you come to realize, is what defines a person and allows you to more or less have the opportunity to seamlessly interact with him or her. In the last number of months, I have perhaps been more silent than usual because there is only so much I can and want to think about when it comes to my mother s decline. I talk to her nearly every day and I have decided for now, that is enough. But I am retreating out of my silence after reading a refreshingly honest, though still somewhat timid piece on Alzheimer in the NYTimes, entitled Zen and the Art of Coping With Alzheimer s
The piece is striking because it offers a more realistic portrayal of the disease than most mainstream media accounts and it provides some really sound advice about the importance of just letting go and going with the flow when interacting with those with Alzheimers. At the same time, it lacks a certain window into just how disheartening and hard it can be to witness the decline, and how hard it can be to manage those conversations and interactions. On the whole, I try to go with the flow. For example, I recently returned from a visit and soon after I left, my mother forgot I was even there. She started to ask me over and over again when I was coming home for a visit and when I told her I was just there, her semi-humorous reaction was well, why didn t anyone tell me? (and then proceeded to castigate her caretaker for not telling her!!). Perhaps I did not stay long enough, or perhaps her lack of recall would happen no matter what. To hear she forgot shook me hard and deeply. The first time I realized she could not remember I had just visited, I was able to jog her memory by listing off all that we did together. Finally, when I mentioned that I bought a new refrigerator while in PR, something clicked. She is still worried about money, so buying a refrigerator was enough to remind her I had spent a lot of money. But after it was clear that she felt quite bad about not remembering, I knew that was the first and last time I would try to jog her memory. Instead, I will merge and mold my reality to her reality, as much as I can and assure her that I will soon visit.
(more…)

Something I often hear from people that talk about Linux on the desktop is this: people want to be able to go to the store, buy hardware, and be confident that it will Just Work.

I would like to point out that things are rarely this simple on Windows. And, in fact, things are often simpler on Linux these days.

Here's the example that prompted this post.

I have a computer that's about 4 years old. It's my main desktop machine at home. It was still fast enough for me, but has been developing all sorts of weird behaviors. Certain USB ports stopped working altogether a few months ago. Then it started hanging during POST whenever I'd try to reboot -- but would still boot OK about 80% of the time after a power cycle. Then it started randomly losing contact with my USB mouse until a reboot. And the last straw was when the display started randomly going out. I've told everyone that my machine has cancer and is slowly dying.

The case is a pretty nice full tower -- solid and sturdy. I have an 160GB IDE drive in it. So I figured I will upgrade the motherboard, CPU, RAM, and add a 500GB SATA drive since they're so cheap these days and I'm running out of space. I'd also have to buy a new video card since my old one was AGP and the new motherboard only has PCI Express for video. So about $700 later from Newegg (I got a Core 2 Duo E6750), the parts arrived.

I spent some time installing it all. The motherboard had only one IDE channel, and I didn't have any IDE cable long enough to connect both the IDE hard disk and the optical drive, so I popped in an old Maxtor/Promise PCI Ultra133 controller I had sitting around to use with the DVD burner.

Now, to recap, the hardware that the OS would see as new/different is: CPU, RAM, IDE controller, SATA controller, Promise IDE controller, integrated NIC, sound, video.

Then the magic smoke test.

I turned on the machine. Grub appeared. Linux started booting.

Even though I had switched from the default Debian "supports everything" kernel to a K7 kernel, it still booted.

And every single piece of hardware was supported immediately. There was no "add new hardware" wizard that popped up, no "I've found new hardware" boxes. It just worked, silently, with no need to tell me anything or have me click on anything.

Only one piece required configuration: the NIC, thanks to some udev design flaws (it got renamed from eth0 to eth1 by udev). That took 20 seconds. Debian saw the IDE HDD, the SATA drive, the Promise controller, the DVD burner, the video card, the sound, and it all worked automatically. And Debian is not even a distro that occurs to a lot of people when they think of great hardware support.

Now let's turn to Windows.

The Windows Nightmare

I have a legal copy of Windows XP Home that was preinstalled on the machine when I got it. I resized its partition down to about 20GB so that I could use 140GB for Linux. I use it rarely, primarily for gaming, and I've bought about 3 games in the last 4 years. I usually disconnect the network when I boot to Windows, though I do keep it current with updates.

I did some research on what Windows was going to do when I replace the hardware. The general consensus from people on the 'net is that you can't just replace a motherboard and expect everything to be happy. There were generally three different approaches suggested: 1) don't even try, just reinstall; 2) do a rescue install after you move over; and 3) use sysprep. The rescue install has to be done by booting from an XP install CD, then picking a rescue install option somewhere. It will overwrite your installed Windows with the version from the CD. That means that I'd have to re-apply SP2, though bits of it that didn't get overwritten would still be on the hard disk, and who knows what would happen to the registry.

Option #3 was to download sysprep (must have the Genuine Disadvantage ActiveX to get the free download from MS). Sysprep is designed to be used just prior to taking an image with ghost for replication. It removes the hardware-specific config (but not the drivers), as well as the product key, from the machine, but otherwise leaves it untouched. On the next boot, you get the "Welcome to XP" wizard.

One other strike against #2 is that Compaq "helpfully" didn't ship any install CDs with the machine. Under Windows, they did have a "create rescue CD" tool, which burned 7 CDs for me. But they are full Compaq-specific CDs, not one of them an XP CD, *AND* they check on boot to see if you're using the same Compaq motherboard, and exit if not. Highly useless.

So I went with sysprep. Before my new hardware even arrived, I downloaded the Windows drivers for all of it. I burned them to a CD, and installed as many as I could on the system in advance. About half of them refused to install since the new hardware wasn't there yet. I then took a raw image of the partition with dd, just in case. Finally, right before I swapped the hardware, I ran sysprep and let it shut down the machine.

So after the new hardware was installed came the adventure.

Windows booted to the "welcome to XP" thingy. The video, keyboard, mouse, and IDE HDD worked. That's about it.

I went through the "welcome to XP wizard". But the network didn't work yet, so I couldn't activate it. So I popped my handy driver CD in the drive. But what's this? Windows doesn't recognize the DVD drive because it doesn't have drivers for this Promise controller that came out in, what, 2001? Sigh. Downloaded the drivers with the imac, copy them to a CF card, plug the USB CF reader into Windows.

While I was doing that, about 6 "found new hardware" dialogs got queued up. Not one of them could actually find a driver for my hardware, but that didn't prevent Windows from making me click through them all.

So, install Promise driver from CF card, reboot. Click through new hardware dialogs again. Install network driver, reboot, click through dialogs. Install sound driver. Install Intel "chipset" driver, click through dialogs. Reboot. Install SATA driver. Reboot.

So the hardware appears to all be working by this point, though I have a Creative volume control (from the old hardware) and a Realtek one in the tray. Minor annoyance to deal with later.

Now I have to re-activate XP. I dutifully key in the magic string from the sticker on my case. Surprise surprise, the Internet-based activation fails because my hardware is different. So I have to call the 800 number. I have to read in 7 blocks of 6 digits, one block at a time. Then I answer some questions: have I activated Windows before, have I changed hardware, was the old hardware defective (yes, yes, and yes). Then I get 7 blocks of 6 digits read to me. Finally Windows is activated. PHEW! Why they couldn't ask those questions with the online tool is beyond me.

Anyhow. Linux took me 20 seconds to get working. Windows, about 2 hours, plus another 2 hours for prep and research.

I did zero prep for Linux. I made one config change (GUI users could have just configured their machine to use eth1).

Other cool Linux HW features

Say you buy a new printer and want to get it set up. On Windows, you insert the CD, let it install 200MB of print drivers plus ads plus crap plus add something to your taskbar plus who knows what else. Probably reboot. Then the printer might actually print.

On Debian, you plug in the printer to the USB port. You type printconf. 5 seconds later, your printer works.

I have been unpleasantly surprised lately by just how difficult hardware support in Windows really is, especially since everyone keeps saying how good it is. It's not good. Debian's is better, in my opinion.

Side note: According to the Skype RSS data feed, right now 174086 user are online. That less than 10% of the usual numbers, isn't it? To me, that's more than a few users.Skype is harmful. Here's the explantaion why:Skype uses P2P technology where it is not needed. There are parts of VoIP where it does make sense (when a direct connection is possible, transfer the data directly between clients), but pretty much any VoIP software does that.And there are parts, where it doesn't make sense. This includes not accessing the login servers directly, but trying to use other Skype users as proxies.Now what I've seen happening in some networks is the following: Skype fails to login directly, so it starts connecting random peers it has seen before. It keeps a list of 200 peers for that in the registry. So when it fails to connect, it will try contacting these 200 peers instead. Now if you have multiple users sharing your connection, let's say 10 and a cheap router with limited memory, 2000 connections is a lot to keep track of. It might start losing other connections. Such as your mail an web connections.This is exactly what I've seen today: the internet connection becoming pretty much unuseable (single packets would work fine, but not TCP/IP transfers of more than a few kb). When we found out it might be due to Skype malfunctioning, and gave the order for everyone to shut down Skype (which didn't work anyway, so people quickly complied), Internet became useable again. Let me emphasize that: it wasn't a bandwidth issue, but it was just the sheer number of connections opened by Skype that caused more important connections to be dropped by the routers.So why is Skype not using proper servers, like MSN, AIM and ICQ do? Because they're cheapskates. It's cheaper for them to let others use your bandwidth instead of having to provide enough to handle all the data. The other reason is that it's harder for system administrators to deny access to Skype, if any other Skype user can proxy your access to the actual login server. But if you want/need a central user registry, you can't rely on a true P2P system. It's bound to get disconnected at some point.And yes, this especially means that Skype is not a good idea in a corporate environment with a large number of users. The skype approach works well for a single user at home, but the Skype networking behaviour can take down your routers if you have too many users running Skype at the same time. Other software choices, such as the standardized, vendor independant SIP protocol for VoIP, however would even allow you to setup a proper VoIP telephone system, with conference rooms, complex call routing, dictionaries, callback, routing to ATM or mobiles, complex voice mail systems, you name it. Skype is designed for single home users, not for company use.P.S. Am I the only one who thinks the Skype UI is crap? No groups (at least not in my version, which is the latest there is for Linux), and totally idiotic button placement (like this ugly 'add' button I'm pretty much never ever, and a floding-up button next to it with the options, quit and recent chats (do they have anything in common?) hidden in there. How about using a menu bar like everybody else? How about organizing functionality in a human-comprehendable way?P.P.S. What I guess has happened: somehow, most nodes (especially super nodes) got convined that the actual login servers are bad. As long, as some still had the connection, login still worked. At least sometimes (many probably experienced an unstable Skype connection before they could no longer login). But at a certain point, a large part of the network got separated from the login nodes and started routing requests around chaotically, trying to find the actual login server. The only way they have for fixing it, is slowly convincing and nodes on the perimeter of the P2P network about the proper location for the login servers, and that way spreading the correct information around again. Or shipping a new version which will re-inject this information in clients again. Just my guesses.P.P.P.S. It would be interesting to know what had caused this. If it was some kind of subtle attack on the network. Like someone trying to convince supernodes that the login servers are bad and untrustworthy or something. Maybe just by sending spoofed bad data packets from the login servers IP or so.P^4.S. Interesting graph on the number of Skype users logged in. Still going down.

Introducing debcheckout Cute little tiny teeny new addition to devscripts: debcheckout (not yet uploaded though, in the mean time you can get it from here. It checks out the versioning repository used to maintain a given package. Sample usage:

$ debcheckout devscripts
declared svn repository at svn://svn.debian.org/devscripts/trunk
svn co svn://svn.debian.org/devscripts/trunk ...
A    trunk/debian
A    trunk/debian/control
A    trunk/debian/links
A    trunk/debian/dirs
A    trunk/debian/compat
<snip>
 U   trunk
Checked out revision 749.
$

The information about where to find a repository is extracted parsing (in a rather dumb way actually, but I really can't stand libapt-pkg API!) Vcs-XXX fields. Intended usages:

1. NMU scenarios: when you're NMUing, please commit your patches (if possible of course: directly to the repository if it has already adhered to the open your VCS campaign I'm sponsoring, or somewhere else if you're using a distributed VCS); with debcheckout the first step it's easy
2. ease the creation of patches: isn't it better to checkout a repository, fiddle around, and then just invoke svn (or whatever) diff instead of remembering (I always forgot that!) to first create a .orig copy of the debianized source tree?
3. retrieving the bleeding edge version of a package which includes the patch for a pending bug you have been waiting for ages

RFC Let me know what you think of debcheckout, feature requests, whatever. In particular let me know if I did something wrong using some VCS, since I'm not proficient in all VCSs supported by debcheckout; for example: I'm quite sure the Arch part is not working ... help is appreciated! Vcs-Cvs proposed convention debcheckout can also give the ground for standardizing the VCS-specific meanings of the various Vcs-XXX fields. In writing it I've noticed that almost all VCSs have de facto standards about what to put in the field, with the notable exception of CVS. That's probably because all modern VCSs rely on some URL-like identifier for a repository location, while CVS does not. It needs a pair URL/module to be checkout. The format I'm currently supporting in debcheckout is a pair of space separated values "CVSROOT MODULENAME"; later on I'm using those values as in cvs -d CVSROOT checkout MODULENAME. Also please note that if you do not put the heading ":pserver:" string in the CVSROOT, users won't be able to checkout the repository without providing a password. Tip of the day: Pod::Usage Perl's Pod::Usage module is cool. Finally I can write the usage string only once instead of duplicating it in the manpage and in the string to be printed upon --help. Ruby had something similar, but the output on console was so horrible that I preferred duplicating stuff for Ruby scripts. Update: I've changed the link to debcheckout.pl so that it points to the "live" version in the devscripts repository, since some patches are already flowing in ...

I have just visited the eBox team room and I have discovered a really nice project they are working on. Jos Antonio Calvo (aka Josh) is creating a really cool suite of programs called ANSTE that allows a developer to define network scenarios and run tests easily on them. We are going to use it to perform continuous integration for eBox, running a test of suites nightly to ensure that every module works correctly, which includes stuff like setting up two separate networks, connect them with the OpenVPN eBox module and checking connectivity through the vpn interfaces or ensuring that the traffic shaping module really does its job. Josh’s work is not this far yet, but ANSTE is already able to read XML scenario definition files like this one and generate the Xen virtual machines as specified, with the appropiate software, virtual interfaces, routes, … as defined by the scenario file. It’s also able to run suites of tests which right now are simple scripts. The next steps are integrating Selenium to perform automatized eBox configuration through a browser (thus testing the GUI too), creating nice GUI tools to generate the scenario files and manage the test suites and writing powerful reporting tools. Of course, ANSTE is free software under the GPL license and you can follow (or contribute to) the development in ANSTE’s Trac.

Today, I wanted to edit a few MP3 files into suitable ring tones for my phone. The idea seemed simple at first: load up a song, chop off a few bars and loop that as a ring tone. Nice in theory, except that sound editing software on a free desktop is challenging, at best: most of it was designed back in the days of OSS and GTK 1.2, some wants to use JACK for playback, while the most recent crop indeed supports ALSA but insists upon having exclusive access to the ALSA device. Dammit! My understanding was that, with Gstreamer and PulseAudio, the free desktop had finally acquired a comprehensive, standardized sound framework, but not many applications seem to support it or is there something I have missed?

When I got myself a widescreen LCD monitor, it didn't quite work as expected. The image showed up on the screen, but squeezed down to the wrong format. Fixing it was not trivial. Apparently, X uses the video mode supplied by the monitor through EDID, but the video mode is wrong. Whether this is because the monitor sends the wrong information, or because of a limitation in the EDID format, I don't know. Anyway, here's how I got it working. Note that I have an Intel graphics card and I use the new "intel" video driver. It might not work for your particular configuration. First, I generated a modeline with the cvt command. CVT means "Coordinated Video Timing", and it's some kind of VESA standard. Run the cvt command with the horizontal and vertical resolution as arguments:

$ cvt 1680 1050
# 1680x1050 59.95 Hz (CVT 1.76MA) hsync: 65.29 kHz; pclk: 146.25 MHz
Modeline "1680x1050_60.00"  146.25  1680 1784 1960 2240  1050 1053 1059 1089 -hsync +vsync

Now, put the modeline in the Monitor section of /etc/X11/xorg.conf:

Section "Monitor"
    Identifier  "Generic Monitor"
    # 1680x1050 59.95 Hz (CVT 1.76MA) hsync: 65.29 kHz; pclk: 146.25 MHz
    Modeline "1680x1050_60.00"  146.25  1680 1784 1960 2240  1050 1053 1059 1089 -hsync +vsync
    #
    Option "Preferred Mode" "1680x1050_60.00"
EndSection

The Preferred Mode option is needed so that the mode is actually chosen. In my case, I have a laptop, so there's also a built-in display, but the mode will not be used on it (the card knows the supported resolutions somehow). In order to be able to switch to a higher resolution when initially using a lower one, the framebuffer needs to be initialized to be big enough. This can be done in the Display subsection:

Section "Screen"
    ...
    SubSection "Display"
        Virtual 1680 1050
    EndSubSection
EndSection

Here's an interesting thing: the old Modes directives are not needed anymore. They are ignored by the intel driver. You only need one generic Display subsection. That's about it. The display should work automatically if it is connected when X is started. If it is connected afterwards, you can enable it with:

xrandr --output VGA --auto

It can be disabled with:

xrandr --output VGA --off

At least when running Gnome, the window manager automatically fits everything to the right screen size. See the xrandr(1) manual page for more options.

Today was a strange day. We are preparing our holiday trip to Sicily but had a full day at home, meant for packing and the like. Indeed, packing was done in a few hours this morning and I ended up....hacking on D-I. For whatever reason, I came back on the plans we have, among the D-I team, to switch to console-setup for console handling. Something already done in Ubuntu and which brings, for instance and if I'm correct, the benefit of having the same keyboard maps for X and the console. So, first, I summarized the floating ideas in a wiki page. Then I worked on a few possible modifications to "localechooser", the D-I component that allows users to choose their language/country/locale and which needs modifications for console-setup. That lead me to produce an experimental image of D-I netboot for i386 and asked translators for some testing. MJ Ray noticed the call for testing and asked Esperanto translators for some testing, which Serge Leblanc did...and reported a nasty bug, later confirmed by Damyan Ivanov, then investigated by Anton Zinoviev, then digged a little more by me again, then advised by Rapha l Hertzog and Julien Cristau on #debian-devel-fr...and lead to an RC bug reported against....perl base. All this, of course, was completely unplanned as of this morning. This is probably one of the reasons that make me enjoy free software development.

My desktop seems to have melted something precious in this morning’s lightning storm, despite the recent and rather hefty UPS sitting behind it. I am, to say the least, annoyed. It was a nice self-built Shuttle system from 2005, but now all it is is a machine check generator:

HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 0 4 northbridge TSC 6eb92ec370b
  Northbridge Watchdog error
       bit57 = processor context corrupt
       bit61 = error uncorrected
  bus error 'generic participation, request timed out
      generic error mem transaction
      generic access, level generic'
STATUS b200000000070f0f MCGSTATUS 4

It’s not even bad RAM or a bad RAM socket, the way most memory errors seem to be; I had two DIMMs in two sockets so I was able to try for both possibilities. Unless it fried both somehow, of course. It seems to boot a little further when left cold for a while, and not as far just after crashing. I am inclined to suspect the motherboard. I think I’m going to have to write it off as a loss. Which means I need a replacement. I can get by with just my laptop for a little while, but not indefinitely. Are there any good options for pre-built Linux systems yet? I was somewhat underwhelmed by Dell’s. I usually build from parts, so looking at pre-built systems always has a certain amount of sticker shock. My main concerns are CPU / memory performance (a lot of compiling here), room for at least three hard drives, and reasonably quiet. Otherwise it’s off to newegg again…

Update: I have been informed that git and mercurial have support for the hunk commit feature, excellent! See the comments section if you want to know the details. IIRC, The last time someone talked about this feature was when John Gorzen analyzed his options for another VCS (still now I can't find any reference to this issue).


Lars, darcs allows this and, as a supplement that is missing from any other VCS I know, is the ability to select individually changes in a file. That means you can commit only some of the changes affecting a file and leave others for later commits.

So for this diff:

$ darcs diff -u
diff -rN -u old-asd/asdnbfnd new-asd/asdnbfnd
--- old-asd/asdnbfnd 2007-08-08 18:12:17.000000000 +0300
+++ new-asd/asdnbfnd 2007-08-08 18:12:17.000000000 +0300
@@ -1,3 +1,3 @@
-#!/bin/ash
+#!/bin/sh

-echo "intentionaol spelling error"
+echo "intentional spelling error"

you can do this:

$ darcs rec -m "spelling correction"
hunk ./asdnbfnd 1
-#!/bin/ash
+#!/bin/sh
Shall I record this change? (1/?) [ynWsfqadjkc], or ? for help: n
hunk ./asdnbfnd 3
-echo "intentionaol spelling error"
+echo "intentional spelling error"
Shall I record this change? (2/?) [ynWsfqadjkc], or ? for help: y
Finished recording patch 'spelling correction'

$ darcs rec -m "make the script run on posix shell"
hunk ./asdnbfnd 1
-#!/bin/ash
+#!/bin/sh
Shall I record this change? (1/?) [ynWsfqadjkc], or ? for help: y
Finished recording patch 'make the script run on posix shell'

Of course, being able to unrecord/uncommit or to rollback is a nice thing to have.

Every time I do a "Getting Things Done" weekly review, I am amazed at how easy it is, and also how much it energizes me. Yet every time it is time to start one, I fear it, and procrastinate, sometimes for several weeks. Obviously I have a deep-set fear of being productive.

A common pattern in blog communication is referred to as a Meme. Here is one example of the commonly used definition of the term as applied to blogs. One common factor that doesn’t seem to get directly mentioned much when people define the term (but which always seems to be mentioned in passing) is the idea of tagging people. So the definition of a meme as applied to blogs seems to be a silly question that you answer in a blog post and then request that some other bloggers (usually 5) answer as well. At the end of this post I have included the dictionary definition of the term (here is the Wikipedia definition). I believe that it is incorrect to call a question such as “which superhero do you most identify with” a meme. Instead I think that there is a Memeplex associated with such posts. One meme is that when someone “tags” you (requests that you answer a question) it should be considered an honour (someone in the blog-sphere likes you enough to ask you random questions in a public forum). Another meme is that such discussion is a good thing (although an increasing number of people in the more serious part of the blog-sphere oppose this). A final one that is apparent to me (I’m sure that there are others) is that so-called memes and lazyweb posts are the same thing (I believe this to be wrong). I believe that lazyweb posts if written about interesting topics can contribute significantly to the community knowledge base. I also believe that chain lazyweb posts (here is a link to the only such post I’ve made so far) can also contribute if created in a sensible manner. Chain posts that don’t require any thought or input from the person re-posting them (EG “please post this message to all your friends so that they can know of the terrible war/famine/earthquake/whatever in some foreign part of the world”) are of course quite useless (you can make a post of links once a month if you want to spread the news about such things). Now I agree that some amount of conversation among bloggers in a community that is personal and not directly related to the main topics of discussion is good for building the community. From Jargon File (4.4.4, 14 Aug 2003) [jargon]: meme
/meem/, n. [coined by analogy with gene’, by Richard Dawkins] An idea considered as a replicator , esp. with the connotation that memes parasitize people into propagating them much as viruses do. Used esp. in the phrase meme complex denoting a group of mutually supporting memes that form an organized belief system, such as a religion. This lexicon is an (epidemiological) vector of the hacker subculture’ meme complex; each entry might be considered a meme. However, meme is often misused to mean meme complex. Use of the term connotes acceptance of the idea that in humans (and presumably other tool- and language-using sophonts) cultural evolution by selection of adaptive ideas has superseded biological evolution by selection of hereditary traits. Hackers find this idea congenial for tolerably obvious reasons. PS This evening I had planned to go to a LUV meeting and see my friend Andy Fitzsimon (blog) give a talk about Inkscape (for which he is famous). I also had a day off work, so it was going to be a day of non-stop fun. But instead I got some sort of cold/flu, stayed in bed for much of the day, missed the meeting, and was late in my blog post. This sucks.

Share This

At some point in the past, Steve Yegge had his "drunken blog rants", now sanitized to "Random whining and stuff", so I guess after one beer I ought to be allowed to rant a bit about what is, according to him, the "Next Big Language". He comes right out and says it in this video: http://blip.tv/file/319044/ (skip to the end if you want to hear it from the horse's mouth). What he gets right in his talk is that Javascript does not have a good brand due to inconsistent and troublesome implementations in browsers. Actually he gets a lot of things right - more geeks should pay attention to having a basic knowledge of marketing, because it's a reality that isn't going to go away just because you don't like it. Leaving that discussion for another time, though, what brought this to mind was this cleverly worded error message in an application I'm trying to stich together:

Object cannot be created in this context" code: "9

To borrow a line, "Note the consistent user interface and error reportage. EdJavascript is generous enough to flag errors, yet prudent enough not to overwhelm the novice with verbosity." Indeed, I don't have a clue as to what it's complaining about. Naturally, I had a look on Google with the error message, and found a few other individuals suffering from the same symptoms, and checked, and rechecked that I was indeed including prototype.js prior to including the file causing the error, but all to no avail. After an hour or so of googling and fiddling with firebug, and reloading, the problem seems to have "just gone away" (I wouldn't believe me either, but there it is...). Let's hope that if Javascript does become 'the next big thing', that even more effort is put into making it a pleasant experience for developers.

After reading Mike Hommey's idea about buildd network for QA testing, I decided to try to implement the 10.1 policy recommendations check using rebuildd. Here is how I proceeded. Install and configure rebuildd This is done by apt-get install rebuildd. Then, init database (I use sqlite) with rebuildd init. Then I initialized build system with rebuildd-init-build-system. I had to add nostrip to DEB_BUILD_OPTIONS, so I decided to do quick and dirty and I edited /usr/bin/dpkg-build-package adding the good export info in the first lines. Configuration file and post build cmd My configuration file is simple and the one by default: I only changed email addresses and add a post_build_cmd script. Here's the code for the script:

#!/bin/sh
dist=$1
pkg=$2
version=$3
arch= dpkg --print-architecture 
RET=0
cd /var/cache/pbuilder/result   exit 1
for file in $(grep '.deb$' $ pkg _$ version _$ arch .changes  cut -d\  -f6)
do
	echo Checking $file for stripped binaries...
	echo ---------------------------------------
	mkdir /tmp/$file
	dpkg -x $file /tmp/$file
	find /tmp/$file -type f   (while read line
	do 
		L=$(file $line   egrep 'ELF.*, stripped')
		if test ! -z "$L"
		then
			echo $L
			RET=1
		fi
	done; exit $RET)
	RET=$?
	rm -rf /tmp/$file
done
echo
echo ...done
echo Removing result files...
for file in $(grep-dctrl -sFiles . $ pkg _$ version _$ arch .changes   sed '1d'   cut -d\  -f6)
do
	rm -f $file
	rm -f /var/cache/rebuildd/build/$file
done
rm -f $ pkg _$ version _$ arch .changes
echo ...done
exit $RET

Fille the db Then I fill the database with:

grep-dctrl -sPackage,Version .  /var/lib/apt/lists/ftp.fr.debian.org_debian_dists_sid_main_source_Sources   cut -d: -f2-   tr '\n' ' '   sed 's/   /\n/g'  sed 's/  / /'   xargs -L1 -ixxxxxx echo xxxxxx sid   rebuildd-job add

And finally I just have to start rebuildd and let it do the job. Argh, already 3 build logs received with stripped binaries Update: Bug list can be found here