To rewrite, or not to rewrite, that is the question. If there's one thing
programmers seem to love doing, it's rewriting a piece of software from
scratch. Which is funny, because "the industry" spends a lot of time
bemoaning duplication of effort and generally touting "reusability" and
"component-based software development".
Joel on Software has a fairly
good treatise on
why
rewriting software, in general, is a Bad Thing. I may not agree with
him all the time, Joel is always worth a read, and I certainly can't fault
his reasoning on this issue.
My primary experience with rewriting software, as with most things
software-related, is in the Free Software world. Although I've been
involved in (or led) rewrites of proprietary software in the past, I've
seen a lot more "let's rewrite it all" death-marches in FOSS. While I
wouldn't characterise them universally as "failures", they don't seem to
have the beneficial effect the authors were probably hoping for.
What does seem to happen, in these FOSS projects, is that the developers
squirrel themselves away in their backyard labs, using the latest and
greatest in software engineering processes and tools, to produce... the
same end result. (If you produce something totally different, it's not a
rewrite, it's a new piece of software that does much the same things as the
old one). When I say "the same end result", I mean exactly that -- it does
the same thing, the code is still subject to coderot over time, and so on.
It doesn't seem like much useful output has been produced. So why do we do
it? Joel gives some good reasons (and why they're not such good reasons).
Rewrites always take a very, very long time to do. It's in their nature.
They're also very demoralising -- you're not really creating anything new,
you're just recreating existing functionality. In the beginning, though,
it's amazingly exciting -- a whole new world of bugfree code, an
engineering marvel, something for the annals of software history.
But several months later, as your new code is becoming cruftier by the day,
and you're just duplicating functionality that already exists, you get
bored. And boredom is the death knell of FOSS projects. I presume that
this excitement-fading-to-boredom progression is why everyone starts
rewrites, but they're often abandoned (or, at least, take a
lot
longer to complete than estimated). I'd make a comparison to that
favourite target of unfair comparisons, marriage, but I know my wife reads
my blog, so I won't. <grin>.
This boredom factor is what has affected the python-appserver backed
rewrite of
IRM (from php3-style
PHP). I took on maintenance of the existing PHP codebase over a year ago,
and the rewrite had been in progress for a while at that point. IRM now
has a bunch of useful new features, and there still isn't any indication of
a useable tarball emerging from the rewrite.
A rewrite in progress is a terrible thing. It tends to make people wary of
working on the existing codebase, if it's going to be a deadend. Without
something to work on, they'll tend to wander off to other projects.
However, sometimes (due to existing deployments), users will work on the
deadend. If the developers have been more-or-less neglecting the existing
codebase while they take on their glorious rewrite, then the rest of the
world is going to have been playing with and tweaking the existing code,
adding their own improvements and modifications. The result of this, when
the new rewrite finally releases, is that there's great wodges of
third-party code that's incompatible with the rewrite, and a large chunk of
your userbase says "stuff it" and sticks with what they know.
This is what looks like happening in the world of
SysCP. A few days ago, I found out that
they're planning on doing a rewrite from the functional-spaghetti-PHP of
the current version to the cult-of-MVC-PHP. This is nice, but their
current estimate is of a first RC release in February. I take this
estimate with a boulder of salt -- for the full collection of usual reasons
(boredom, real life, overoptimism, etc). The fact that there's not a huge
amount of movement on the commit log doesn't inspire confidence, either,
and despite early claims of a massive development team, only one person has
been committing recently.
In the meantime, I'm tasked with adding a number of small features to the
current SysCP codebase, which we need for work. When the rewrite comes
out, it's highly unlikely that there will be a business case for rewriting
my features for the new code, and it's not likely that the features we need
will make it into core. So, SysCP will probably fork -- the old,
featureful, crufty codebase, vs the new, shiny codebase. If there's
sufficient new features to warrant an upgrade, we'll take the plunge, but
what SysCP does now is very close to what we want, so it'll take a fair
amount of shiny! to get us to move.
This is a general issue of software development in FOSS -- if you take your
eye off the ball, for any reason, someone else will probably pick it up and
run with it. Your rewrite will be a completely new project, and you'll get
some of the users and the fork of your old codebase will continue to live
on in another guise.
Let's face it, though -- your code is going to accumulate cruft. Over
time, it morphs into this bletcherous pile of huge functions, multi-purpose
classes, and reams and reams of bug reports. How do you deal with this
cruft? Surely at some point it's easier just to throw it out and start
again?
Reality says no. Joel gave the key advice in the article linked above (go
read it if you haven't already -- seriously), and it's strongly hinted in
my article title. Don't revolt; instead, evolve your codebase. Tweak it,
evolve it, move it slowly (and compatibly) towards your utopian ideal of
perfect code. You won't get there, but you'll almost certainly get closer
by gradual and continuous improvement than you would be going back to the
start line and walking in a different direction.
There's one big question that's nagging at me, though -- what happens when
you want to switch languages? I'm loving
Ruby on Rails at the moment, and so
are several other people at work, but we've got (both personally and
professionally) huge swathes of code in PHP (and other second-rate
languages) that needs to be dealt with. It's a lot harder to evolve code
into a different language than apply a gradual degunking.
I have a number of ideas on this point, but I'd love to
hear your ideas too. In a few
months, I'll have a lot better idea of what works and what doesn't in
massive PHP to Rails conversions, and I'm sure everyone out there in the
blogosphere will hear about them in due time.
The future for Ruby and Emacs is looking hopeful. I learned that a
proper Ruby grammar for the semantic package is being worked on,
which will enable class browsing and intelligent completion.
I am still disappointed that there's no version of Emacs
that uses Xft or Pango for anti-aliased font rendering. To get sharp fonts,
I still need to use a bitmap font like misc-fixed. If I were in the
business of offering bounties for open source development, dragging
Emacs into the GTK2/Pango era would be the biggest one.
Checking the mailing lists indicates that 
