Disclaimer: There is a huge discussion going on right now in the
debian-devel
mailing list, about different ways to rearrange how Debian development
happens, and maybe to provide a Debian users with an additional
release. This blog entry is not a comment on that discussion, which
I am ignoring. The discussion did, however, prompt a
dent about "test driven
development". That prompted a short exchange of thoughts with
Tom Marble, and this blog post is a cleaned up version of my part
in that discussion.
Currently Debian development happens roughly like this: all packages
are uploaded to a part of the Debian archive called unstable.
Once they've been in use for a while without serious problems,
they get automatically copied into another part called testing.
Every year or two, testing is frozen and any remaining problems are
fixed, after which all of testing gets copied into yet another
part of the archive called stable, and that's the actual release.
There is a little bit of automatic testing, but only a little.
Almost all testing is done by people, by them using
unstable or testing on their usual computers. If they have problems,
they report them.
Contrast this with development using Test Driven Development, or
TDD, and
other modern development methodologies. Here's a rough summary of
what I do when I write (much) of my software.
- first write one or more automatic tests (unit or functional ones)
- write the actual code, enough to make all tests pass
- add more tests
- write more code, or change existing code
- repeat this until tests describe all the behavior desired of the code
In addition, I measure coverage to make sure all parts of the
code gets tested. I usually aim for 100% coverage, except for those
parts that very hard or quite pointless to test. (That's easier to
achieve than you'd think, but that's the topic of another blog post.)
This all sounds like a lot of bureaucratic nonsense, but what I get
out of this is this: once all tests pass, I have a strong confidence
that the software works. As soon as I've added all the features I
want to have for a release, I can push a button to push it out.
(Well, actually there's a
little bit more to it.)
This does mean I never have bugs in my software. Of course I do.
However, there's a lot fewer of them. In fact, there's so few of them,
after tests pass, that I would almost be happy to make an automatic
release after every successful run of "make check".
Another aspect of the way I do development is distributed version
control. The relevant feature is powerful branching and merging.
Most things I develop happen in short-lived single-purpose
branches: whenever I start work on a feature or bugfix, I create
a branch for it. (Unless I'm feeling particularly lazy, which happens
more often than you'd think.)
When I've finished the feature, or fixed the bug, I merge the branch
back into the trunk branch.
This way, the trunk is always in releaseable state. It might not
have all the features I want for the next release, but the features
that are there, do work. The branch probably has bugs, but if I've
written tests that are good enough, I know the software works well
enough.
Or that's the idea. Sometimes things go wrong. Then I write more
tests and the next time goes better. (It doesn't have to be perfect,
as long as it gets better every time.)
See the contrast between automatic tests with good coverage, and
the Debian style of relying on user feedback? There's no need to wait
for user feedback with automatic tests. This speeds up development,
makes releasing easier, but most importantly takes away any reason
to fear making changes, even big changes.
Automatic tests and good test coverage are easy achieve in small
projects. For a system as huge as Debian, good test coverage is
quite hard to achieve.
The thing about automatic tests, though, is that even a little bit
of it is helpful, and after you have a few tests, it gets easier
to add more. The first test is the hardest to get done, since you need
to set up all the infrastructure for it.
Debian does do a bit of automatic testing already.
(See lintian, piuparts, autopkgtest, edos, etc.)
I don't want to belittle that, but I think we could do better.
Here's what I would love to see:
- we have a part of the archive that corresponds to a trunk
branch, i.e., it is always in a releasable state
(the "testing" area was originally meant to be that; we
could make it so now)
- releaseable state is determined by two things:
- an automatic test suite
- user feedback, particularly in the form of bug reports (as now)
- whenever changes are made, they happen in "branches"
- the branch is not affected by changes elsewhere in the archive,
except by manual synching ("merge from trunk")
- individual package uploads, as well as groups of packages such as
for transitions, are each in their own branch
- this is sort of similar to a PPA, but more powerful
- when a branch is to be merged into trunk, the automatic tests must first
pass (or at least no new failures in them can be introduced)
- tests can also be run for any other branch, of course, so that those
developing the branch know if they're ready to push their changes into
trunk
- there's a culture of writing tests for bugs (whenever that is
feasible), and for new features
- particularly release goals should be expressed as automatic test suites
- there's a culture of sharing tests with upstreams, and
with other distributions
Since full test coverage is going to be impossible for Debian, some
subset should be targeted. Perhaps something like this would suffice
to start with:
- a version of debian-installer is generated from a branch
- test installation of a new system from scratch
- with a large set of tasks selected
- test upgrade from previous stable release to the branch
- with a large set of packages installed (as many as possible, actually)
- test upgrade from trunk branch to branch to be merged
- ditto large set of packages
- test with lintian, reject on specific tests failing
- test with piuparts
- test the whole system for specific functionality
- ssh access from outside
- sudo access for a logged in user
- sending mail with SMTP
- web server
- possibly test specific web applications
- possibly test Samba and NFS services
- possibly test printing services (CUPS)
- possibly test essential desktop functionality (automatic login, at least)
- if package comes with package specific tests, run those too
These tests would not guarantee that a set of changes would not break
Debian, but they would give a high confidence that at least basic
stuff would still work after the changes.
Now, obviously implementing all of this I'm dreaming of
(it is just past midnight, after all) is going to be impossible.
There's way too much work to do, there's not enough tools for writing
tests, and it would require too much
computing power to run, and so on and so forth. But it's late, and
I've had a bad day, and I might as well dream of something nice.
Anyway, anyone interested in these things should perhaps help
drive
DEP8.
For some time, code documentation for phpMyAdmin had no official place. The reason for this was phpdoc eating way too much memory on daily runs, so I've disabled the cron job. Today I gave a try to DocBlox and it seems to work fine, so I've re-enabled daily builds.
The new (daily updated) documentation is available at same location as the original one was:
http://www.phpmyadmin.net/phpdoc/
You can also check build failures in the build.log:
http://www.phpmyadmin.net/phpdoc/build.log
Another month, another Games Team IRC Meeting happening. This time it was decided to have it again on Sunday, the time was set to 10 am UTC. To find out the time in your localtime, issue 
Haven't been in my new home that long and through a long chain of contacts, my neighbour was finally able to contact me. (Just goes to show that having Debian friends nearby who like board games turns out to make some things JustWork.)
The near-term goals are certainly:
Since my laptop features 
We downloaded a total of 8 
IIT Bombay, like most likely most Indian educational instituates, has some rules and regulations about the interaction between boys and girls that, at least to the uninitiated eye of a foreigner who was brought up in a liberal society, seem arbitrary and archaic. At IIT Bombay, for example, there may be no visits in hostels by students of the other gender (yes, there are separate boys and girls hostels) between 10pm and 7am, and during the day any such visit has to be registered at the 