I agree Matthew Palmer, and by extension Joel on Joel on Software that rewriting software projects from scratch is a very bad idea, which in case of FOSS can mean death of the project because people lost interest on it due to the great effort needed to reimplement things that were working before. At the same time I take what Joel says: programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. Being myself a Civil Engineer I cannot agree most. The other option is patchy and result will never be so good as a well planned project, with a good design, time... and money. And this is one of the greatest things of software. With a good planning you can refactorize it, change what you want and even demolish and rewrite some parts, whithout having to throw away all the old code and start over.

To rewrite, or not to rewrite, that is the question. If there's one thing programmers seem to love doing, it's rewriting a piece of software from scratch. Which is funny, because "the industry" spends a lot of time bemoaning duplication of effort and generally touting "reusability" and "component-based software development". Joel on Software has a fairly good treatise on why rewriting software, in general, is a Bad Thing. I may not agree with him all the time, Joel is always worth a read, and I certainly can't fault his reasoning on this issue. My primary experience with rewriting software, as with most things software-related, is in the Free Software world. Although I've been involved in (or led) rewrites of proprietary software in the past, I've seen a lot more "let's rewrite it all" death-marches in FOSS. While I wouldn't characterise them universally as "failures", they don't seem to have the beneficial effect the authors were probably hoping for. What does seem to happen, in these FOSS projects, is that the developers squirrel themselves away in their backyard labs, using the latest and greatest in software engineering processes and tools, to produce... the same end result. (If you produce something totally different, it's not a rewrite, it's a new piece of software that does much the same things as the old one). When I say "the same end result", I mean exactly that -- it does the same thing, the code is still subject to coderot over time, and so on. It doesn't seem like much useful output has been produced. So why do we do it? Joel gives some good reasons (and why they're not such good reasons). Rewrites always take a very, very long time to do. It's in their nature. They're also very demoralising -- you're not really creating anything new, you're just recreating existing functionality. In the beginning, though, it's amazingly exciting -- a whole new world of bugfree code, an engineering marvel, something for the annals of software history. But several months later, as your new code is becoming cruftier by the day, and you're just duplicating functionality that already exists, you get bored. And boredom is the death knell of FOSS projects. I presume that this excitement-fading-to-boredom progression is why everyone starts rewrites, but they're often abandoned (or, at least, take a lot longer to complete than estimated). I'd make a comparison to that favourite target of unfair comparisons, marriage, but I know my wife reads my blog, so I won't. <grin>. This boredom factor is what has affected the python-appserver backed rewrite of IRM (from php3-style PHP). I took on maintenance of the existing PHP codebase over a year ago, and the rewrite had been in progress for a while at that point. IRM now has a bunch of useful new features, and there still isn't any indication of a useable tarball emerging from the rewrite. A rewrite in progress is a terrible thing. It tends to make people wary of working on the existing codebase, if it's going to be a deadend. Without something to work on, they'll tend to wander off to other projects. However, sometimes (due to existing deployments), users will work on the deadend. If the developers have been more-or-less neglecting the existing codebase while they take on their glorious rewrite, then the rest of the world is going to have been playing with and tweaking the existing code, adding their own improvements and modifications. The result of this, when the new rewrite finally releases, is that there's great wodges of third-party code that's incompatible with the rewrite, and a large chunk of your userbase says "stuff it" and sticks with what they know. This is what looks like happening in the world of SysCP. A few days ago, I found out that they're planning on doing a rewrite from the functional-spaghetti-PHP of the current version to the cult-of-MVC-PHP. This is nice, but their current estimate is of a first RC release in February. I take this estimate with a boulder of salt -- for the full collection of usual reasons (boredom, real life, overoptimism, etc). The fact that there's not a huge amount of movement on the commit log doesn't inspire confidence, either, and despite early claims of a massive development team, only one person has been committing recently. In the meantime, I'm tasked with adding a number of small features to the current SysCP codebase, which we need for work. When the rewrite comes out, it's highly unlikely that there will be a business case for rewriting my features for the new code, and it's not likely that the features we need will make it into core. So, SysCP will probably fork -- the old, featureful, crufty codebase, vs the new, shiny codebase. If there's sufficient new features to warrant an upgrade, we'll take the plunge, but what SysCP does now is very close to what we want, so it'll take a fair amount of shiny! to get us to move. This is a general issue of software development in FOSS -- if you take your eye off the ball, for any reason, someone else will probably pick it up and run with it. Your rewrite will be a completely new project, and you'll get some of the users and the fork of your old codebase will continue to live on in another guise. Let's face it, though -- your code is going to accumulate cruft. Over time, it morphs into this bletcherous pile of huge functions, multi-purpose classes, and reams and reams of bug reports. How do you deal with this cruft? Surely at some point it's easier just to throw it out and start again? Reality says no. Joel gave the key advice in the article linked above (go read it if you haven't already -- seriously), and it's strongly hinted in my article title. Don't revolt; instead, evolve your codebase. Tweak it, evolve it, move it slowly (and compatibly) towards your utopian ideal of perfect code. You won't get there, but you'll almost certainly get closer by gradual and continuous improvement than you would be going back to the start line and walking in a different direction. There's one big question that's nagging at me, though -- what happens when you want to switch languages? I'm loving Ruby on Rails at the moment, and so are several other people at work, but we've got (both personally and professionally) huge swathes of code in PHP (and other second-rate languages) that needs to be dealt with. It's a lot harder to evolve code into a different language than apply a gradual degunking. I have a number of ideas on this point, but I'd love to hear your ideas too. In a few months, I'll have a lot better idea of what works and what doesn't in massive PHP to Rails conversions, and I'm sure everyone out there in the blogosphere will hear about them in due time.

Somehow John Ringo’s latest, Princess of Wands, reminds me of another book he’s written recently. Both have the overtones of some sort of literary self-congratulation. In Ghost’s case, I counted that as a fault. In Princess of Wands‘ case, I count it as a strength. Both books are also episodic rather than regular novels, but the similarities end here. One problem in reviewing this book is that one cannot really discuss much of the setup without writing major spoilers. The cover blurb takes one approach: describe what the reader learns within the first five pages, who cares that the picture that gives one of the book is completely wrong. Unfortunately, the cover blurb is so misleading that it can make people ignore this book even though they should have picked it up. The genre is urban fantasy. The main character is a devout Christian soccer mom who finds herself in an adventure. The magic system and the ontology are quite interesting. There are battles with evil. There is a con. The publishing house itself, Pier… erm… Baen Books, as well as some of its authors, make an appearance onstage. (In fact, if you aren’t a barfly, you’re going to miss some of the fun like I probably missed a lot of it, never having visited a con where Baen people gather.) I found it a satisfying read, especially the middle episode. The final episode was so short that it didn’t feel well balanced, but it’s a minor issue. However, I would probably hesitate to recommend this book to a certain kind of Christans: if you think Harry Potter is evil, don’t come near this book. Overall, I’ll rate this book 4/5.

One possible way to bridge the gap between Debian and Ubuntu: It’s funnier because it’s theoretically possible.

I’m flying Song airlines for the first time, en route to Orlando on business. The individual interactive multimedia consoles are pretty cool. You can check out your altitude, flight path, etc., in real time, which I think helps mitigate the lack of control and attendant anxiety most people feel while flying. There is also a decent selection of “MP3s” (I wonder if they are actually stored as MP3s?) that you can queue up for a personal playlist. I’ve found more than enough selections to last the rest of the flight. (Strangely, I seem to have to hit ‘pause’ at the end of each track to advance to the next). There are also a bunch of satellite TV stations, pay-per-view movies and games, although I haven’t checked those out. I’m curious about how the music licensing works. The airline would like to provide the attractive feature of a broad variety of music for its customers; at the same time, it probably serves as effective marketing for the labels, just like radio play. You might, for example, check out an album you were thinking of buying on the flight, and purchase it later if you liked it. The chance that inflight music will be copied by customers is very low and presumably they don’t need to worry about DRM. So which way does money flow—from the airlines to the labels or vice-versa? Or are the tracks “free?” Now playing: The Beatles, Revolver, “Got to Get You Into My Life.” P.S. After we landed, the interactive multimedia consoles rebooted. It turns out they are running Red Hat Linux, circa 2002. Smart.

I've already fielded enquiries from two people who have found my blog via Google, and given the 10,500 visas a year being issued, there's only going to be more, so in the interests of not spending too much time answering questions individually, I'm going to write a detailed account of things. Disclaimer: This article is not a replacement for doing your own homework. I expressly disclaim any responsibility for you getting your visa application denied, being cavity searched when you attempt to enter the US, or being sent to Guantanamo Bay. In particular, this information is subject to change, and as I'm not applying for a new visa every day, I'm not going to know about the changes. If you find something here that is blatantly wrong, or out of date, please let me know, but at the moment, I'm writing about my experiences. I'm undecided about maintaining a website devoted to capturing the state of the art in E-3 visa applications. If in doubt, spend the money (and I do mean spend, it's not a cheap phone call) and call the information line - 1902 941 641 - to speak to a human being, and ask all the questions. Resources

• The Australian Government's Department of Foreign Affairs and Trade's page on E-3 visas
• The US Consulate's E-3 visa page
• The US Consulate's page on applying for a non-immigrant visa

What's the process? Your potential employer (sponsor) needs to file a Labor Condition Application (see the form here). At present, this form hasn't been brought up to date with respect to E-3 visas, it's the same form used for H1-B visas. Your employer needs to have written at the top of each page "E-3 - Australia - to be processed". This bit of handwriting is apparently key. You need to have the signed LCA in your hot little hand before you roll up at the US Consulate to apply for your visa. As for how long these take to be processed by the Department of Labor, the FAQ says a week for postal applications. From my experience, and the experience of a couple of other people I've spoken to, it takes anything up to about four weeks. Generally speaking, you do not have to concern yourself with the LCA process, unless for some reason, you're sweating on the LCA so you can apply for your visa. My personal opinion is that the whole process is so long, bureaucratic and unpredictable, that you really don't want to be making any firm plans based on any assumptions on when you think anything is going to happen. Start your planning from when you have been issued your visa, not when you expect you will be issued your visa. Once the LCA has been issued, your sponsor will send it to you along with a wad of paperwork, most of which you just take with you to the consulate. You can make an appointment online. Paperwork-wise, you need to take a completed DS-156 and a DS-157 if you are male between 16 and 45. These forms are available here. You can make an appointment online, the links to make an appointment at each of the US Consulates are also here. I found that the lead time to an appointment was around a month, so if you know the LCA application has been made, you can probably make the booking at the Consulate, so as to try and parallelise things a little bit. What do you need to take to the interview? You need to take your completed DS-156 and DS-157 forms. You need one for each person. So if you are the principal (i.e. the person being hired), you need a DS-156 for yourself, and your spouse, and whichever of you is male needs a DS-157. Bring the sponsor's offer letter too. You need a receipt for a non-refundable visa application fee. You need one receipt per person applying for a visa. You get these receipts by paying $130 per person at any Australia Post outlet. As the visa is non-immigrant, you need to be able show some sort of ongoing connection with Australia. If you own property here, and are planning to continue owning it while in the US, bring a rates notice. I don't know what else is useful to demonstrate an ongoing connection. Also bring a bank statement, as you need to demonstrate an ability to support yourself while in the US. Bring birth certificates and marriage certificates. Also bring a copy of your University transcript, and any assessments that your employer may have had done that state that you hold the equivalent of a US degree. I'm not sure how hard and fast the whole degree thing is. I have an incomplete degree and about 10 years of work experience. This was sufficient. I don't know how you'd go with absolutely no degree and lots of work experience. Finally, you need to bring two US-sized passport photos. These are 5cm x 5cm, which are not what your average passport photo is like if you go somewhere and ask for a passport photo. Processing takes about 3 days, although you get an indication on the day as to whether you are successful or not. If you don't want to come back to collect your passport with the visa in it, you need to leave sufficient stamped, self-addressed envelopes so that the Consulate can mail the passports back to you. It's a bit hazy as to what constitutes 3 days, but assume 3 full working days, plus whatever time it takes for the postage, so up to 5 working days. Good luck with your visa application. Let me know how it goes, and if this has been of assistance.

Toshok: as you are obviously proud of your vCard parser in evolution-data-server, how about going back into e-d-s for a minute and reviewing a patch which makes it twice as fast. Thanks!

After my last Drupal article in the German + international Linux Magazin(e) from early 2005, my new Drupal-related article has been published in PHP Solutions's issue 1/2006 (13). After I was back from a short vacation without Internet access, I was surprised to see a post by Robert Douglass titled "Uwe Hermann's Drupal article in PHP Solutions is out" on the drupal.org main page. At that point I didn't even know that the article had been published ;-) Robert wrote a short review of the article which outlines the main content. In contrast to my last article which was a general introduction to Drupal, this one concentrated on some specific (probably more advanced) topics instead: multi-site install, l10n+i18n, search engine optimization, and AJAX(-like) user interfaces of the upcoming Drupal 4.7 release. The printed article should be available in multiple languages (German, French, Polish and Italian, at least). There might be a free online version as soon as the next issue of PHP Solutions is published, but I'm not entirely sure. Thanks a lot to Robert Douglass for the review and to Drupal's Benevolent Dictator for Life Dries Buytaert who reviewed an early draft of my article and provided helpful suggestions!

Bob Sutor: “Standards can allow innovation by providing a strong underlying foundation on which to build the new ideas. Do standards themselves need to be innovative? No, but they can lead to innovation downstream. Can waiting too long to create open standards hinder innovation? Absolutely, if de facto standards force or convince people not to try to innovate in certain areas. So please don’t simply say ’standards might hinder innovation.’ That’s overly simplistic and doesn’t take into account how things evolve over time.”

... and so on. Today's example: For each File. Yes, you know what it does already. But that would make for a very short blog post, and that is so not my thing, so I'll quote the short description of the project:

For each File lets you run an arbitrary set of GNU BASH commands upon an arbitrary set of objects. It can be used to perform repetitive tasks such as renaming, extracting, moving, and manipulating a large number of files.

Yes, dear friends, someone has sat down and written a program (and associated manual) to implement a bash for loop (and possibly a bit of find as well). Since it's all written in bash, it's not as though the author of the program was incapable of working out how to write for i in /path/*; do [...]; done, either. I'm not quite sure which part amused me the most. Was it the ever-optimistic -devel and -user mailing lists, completely devoid of archived material? How about the user manual -- 40% of which was the licence text? Perhaps it was the fact that this not just a bash for loop -- it's a fully internationalizationable bash for loop. I know one thing, though -- with all of the energy put into this program (and it's been going for several years now, apparently) the author could have created the ultimate, never to be surpassed manual for the bash for keyword, which would have been significantly more flexible, and certainly less interesting to poke fun at. Addendum: the whole site that FeF is hosted on is accessable only via SSL with a self-signed certificate -- except for the download page, which is HTTP only. So you "know" that the non-existent list archives are really what they say they are (at least as long as you trust "Gna!" as a certificate authority, as I'm sure we all do), but the actual source code is hosted who-knows-where, with no useful assurances as to provenance whatsoever.

The Hogmanay celebration in Edinburgh started on December 29. This was coincidentally on the same day as my meeting with Steve Kemp, a fellow Debian developer. We had a nice chat and a huge and tasteful hot chocolate in a coffee shop on Nicolson Street. After our meet we ran into the Torchlight procession, Edinburgh’s Hogmanay first event. There seemed to be no end to the procession. I was cold, my feet were hurting, so I caught a bus home and missed out on the fireworks. New Year’s Eve On New Year’s Eve I took the last bus to the city centre (around 21:00). It was still raining a bit, but fortunately it had completely cleared up once I got out of the bus. There, I witnessed the start of the Royal Bank Street Party (consisting of concerts, a carnival, attractions, etc.), which meant thousands, ten thousands of people gathering to enter the party. This party was held in a closed down perimeter covering Princes Street and other adjacent streets. Since I had no ticket I made a walk around the perimeter to get a glimpse of the party itself but also to see what else was going on in the centre. It turned out that there were lots of ticket-less people who were partying in pubs or just singing and dancing on the streets. At 23:30, lots of people (including me) gathered on the North Bridge to wait for the 0:00 moment. At midnight loud yelling rose from the centre evolving into a load roar as if legions of men were sent to war and the Seven Hills Fireworks were set of. Everyone was cheering, jumping in the arms of strangers and wishing everyone a good and happy new year. It was an amazing thing to experience and be part of :), I hope that I can come back for it once more. A happy and good new year everyone!

It has been a tumultuous decade of involvement with Debian for me. I had been on the mailing list since mid 1994, but I was reasonably happy with my SLS system (installed using 40 floppies, including about a dozen for just X11 alone), and while I found Debian intriguing, I was not about to go through the pain of a brand new install until I felt that the new project was viable in the long term :-) I actually jumped ship in the spring of ‘95 and installed 0.93R5. The next step came with Bug 1766, my very first bug report: Bug in script checksecurity in package cron, on 25 Oct 1995. Once in, I rapidly went to the next phase: Here is the sum toto of the NM process I went through: my Hello, World mail. Those were the days :-). There was nothing between my ITP and the upload. My first significant package was kernel-package, since I was always missing something in the series of steps needed to build a kernel, and I started getting into it in the summer of ‘96. This is where the second part of my apprenticeship started – even though I had 3-4 packages in the archive, my kernel images were not yet trusted; so I sent my images to my sponsor (Hi Simon), who then uploaded the images to master. Somewhat later, I also was involved in the early design stages of apt, and the dependency sorting algorithms. While I was fairly silent during the whole DFSG/SC debates (to the extent I was labelled a mindless follower of Bruce –heh), I took an active part in the constitution debates (possibly due to the fallout of the beach story. Anyway, I seemed to want to get us a constitution, after it had seemed stalled for months. It is interesting to note how the technical committee was initially setup, there was a proposal, and then Ian Jackson responded by saying he wanted to appoint the ctte members, since he had been around for a while and was also the DPL. The earliest data I can find is from June of ‘98, as seen in a mail later that year, when the initial list of committee members was created (I also seem to recall I was not on the list initially, but I was added in the early days, before the committee was actually formed). I was interested in the technical policy fairly early, taking a stance that policy was more than just a bunch of ignorable guidelines. Eventually, a [thread that tried to reach a compromise][other] gave us something like the views we (well, I) hold today – including what happens when packages do not follow policy, and about policy editors. And abuse of power. About this time, the policy Czar resigned, burned out, and hounded by accusations of delusion of power. So I first proposed the whole policy editor and consensus approach to letting policy evolve, which eventually led to a formal delegation recently. Brian Basset was our very first secretary. My first involvement with things that would lead on to the secretaries job started with trying to do voting for policy. Then, around 2000, our the secretary and treasurer Darren Benham went MIA, and Raul, as the chair of the technical committee, had to take over and run the DPL election for 2001 – and forgot that DPL elections are supposed to be secret ballots. Ben Collins tapped me for the secretaries position mid-2001, as I recall. It is hard, in a decade or so, to find anything I have not touched – but NM is one such area. Apart from an early pre-current-NM mail, I have not been very involved in NM. Or the Debian installer. Or Debconf. Hmm. I seem to have drifted away from things that Joey Hess is involved in, which is a pity, he is high on the list of people I respect in the project, and this lack of interaction with as time goes on irks me.

Aggregation the G way.
I've been using the magic of RSS (and Atom) to keep up with Planet Debian, blogs of my friends, tech news and posts to an anime fan forum for the past several months, however one little problem bothered me - the forum has a lot of posts (sometimes more then 5 per minute) and an RSS feed of only last five items. That means that I had to have my RSS aggregator (Liferea) open at all times, so that I do not miss any posts. However keeping my laptop on and online at all times is quite bothersome, so I started looking for solutions.
I tried Google Reader however I didn't like it much - I like to see more of my feed. There is too much screen space wasted for all the wrong reasons, one can only see 5 items at the time (which is quite a hassle if you have 500+ of them), a lot of info I would like to see is not there (which blog did this come from???), some strange html conversions, ...
Now I have found my solution. It is a combination of rss2email and GMail. I have a computer that I always keep on and online (it could be a server, but in my case it is a simple workstation). I installed rss2email there, added all my feeds to the database there (hint: export feedlist from Liferea and do a bit of grep/sed magic), moded config so that all mails come from a single address, all have a custom identifiable header and all are HTML mails without any transformations. After that I configured cron to launch "r2e run" every half an hour and "r2e run 22" every minute (where '22' is the id of the feed of that anime forum). At Gmail side I simply filtered all mails from rss2email into a separate label and archived them (so that they do not clutter my inbox).
Now when I want to reed my feeds, I go to the last message of the RSS label in my Gmail, open it, read it, star it if I need to look at it later and then press "k" to get to the newer message. After I am done reading I usually go and remove RSS label from read messages, so that they do not appear in this labels "folder" and are only saved in Archive. If I start running out of space, I will simply search and delete old RSS items then.
Most of the screen space now is devoted to the message, I can see all the fields I want to see (source, author, topic, content, pictures, URLs) and I can manage messages by hundreds. Also being able to search both your mail and your rss feeds at the same time is neat.
Yesterday I did a little improvement to this scheme. The forum that I read has the title of every message made of "author: topic", so every message is a separate conversation in Gmail. I decided to try to use the conversation feature of Gmail and did a little modification to rss2email so that if the message is from this forum, then everything up to the first semicolon is cut from the title and inserted as a part of the name of the sender. In the end the subject of every message contains only the topic of the corresponding thread in the forum and thus all messages from one thread are neatly grouped together as a single conversation. At the same time every messages From: looks like "$forum_name $author " so that I can still see who wrote what in the forum.

Bonne Ann e !
!
Feliz a o nuevo !
Feli any nou !

This is a transcript of what a friend of mine sent as a Xmas card. Of course she is a lawyer, so I hope that this text can transmit you exactly my feelings.

Please accept with no obligation, implied or implicit, our best wishes for an environmentally conscious, socially responsible, low-stress, non-addictive, gender-neutral celebration of the winter solstice holiday, practiced within the most enjoyable traditions of the religious persuasion of your choice, or secular practices of your choice, with respect for the religious/secular persuasion and/or traditions of others, or their choice not to practice religious or secular traditions at all. We also wish you a fiscally successful, personally fulfilling and medically uncomplicated recognition of the onset of the generally accepted calendar year 2006, but not without due respect for the calendars of choice of other cultures whose contributions to society have helped make America great. Not to imply that America is necessarily greater than any other country nor the only America in the Western Hemisphere. And without regard to the race, creed, color, age, physical ability, religious faith or sexual preference of the wishee. By accepting these greetings you are accepting these terms. This greeting is subject to clarification or withdrawal. It is freely transferable with no alteration to the original greeting. It implies no promise by the wisher to actually implement any of the wishes for herself or himself or others, and is void where prohibited by law and is revocable at the sole discretion of the wisher. This wish is warranted to perform as expected within the usual application of good tidings for a period of one year or until the issuance of a subsequent holiday greeting, whichever comes first, and warranty is limited to replacement of this wish or issuance of a new wish at the sole discretion of the wisher.

The script starts out small and does a single job. You realise that you can do so much more, so it gets bigger and more arguments get added to it. Passing more than four parameters, in the correct order, gets painful so you add some option parsing code. Suddenly the number of options grows, so you add a manpage to try to explain the parameters and what they are used for. Other people start using the script and wish to change the behaviour, so you add yet more command line switches and complexity. Gradually you realise your once-tidy script is now a complete mess, so you remove 50% of the code and move it into external “plugins”, or helper scripts. The small script is now much neater, but suddenly users cannot find the code they wish to change - they have to grep through all your helper scripts and plugins to find the code to change. Beacuse users cannot find the right code they submit patches making your script bigger and more complex. Repeat indefinitely. p.s. xen-tools 0.6 is out. It has plugins

I won t really and technically be 17 for approximately 2.5 hours, but it s the right day so close enough. Landmarks of the date: now when I go to see R-rated movies with friends, it s actually legal. (Like that matters.) In other news, band ended last Sunday: I m now scurrying to catch myself up in a dozen different things, which included taking SATs all this morning (8-12:40) as well as updating my packages. (I swear, the SATs have evolved into a test of who can not go insane sitting faced with bubbles for four hours with two 5-minute and one 1-minute break.)

The Web site of the French Parliament features the full transcription of the debates about the DADVSI bill. For those of you who can read French, I suggest you have a look at it. I suggest you look at the speech by M. Christian Paul who gives a very good argumentation about why this text is EVIL. In the text, "M. le Ministre", is Renaud Donnedieu de Vabre, the Ministry for Culture and Communication of the government headed by Dominique de Villepin (hey, we should tell these guys that the French revolution was over 200 years ago!).