Water supply, in the developed world, is one of those things you just
pretty much take for granted. You turn on the tap, and clean, cool,
refreshing water comes out. Similarly, when I go to a fire in my shiny
fire truck, no matter what time of the day or night, I expect to able to
hook up to a hydrant and have a strong, steady supply of water available.
This level of reliability is something that we in the IT industry can
typically only dream of. Practically 100% reliability over a period of
decades, without constant maintenance and tweaking? Not a hope. To even
get close to that, we need clusters of fully-redundant servers, fancy
database replication techniques, and probably something totally out of
left-field like Erlang's ability to reload code on the fly.
But how do the water utilities do it? Clusters of fully-redundant
high-capacity pumps, fancy pipe re-routing techniques? Or something
totally out of left-field like a big water tank and gravity?
Where I live, we've got the latter. Thinking about how it probably
operates and is managed, I'm blown away by the sheer simplicity and
robustness of the whole design, and how it can handle all manner of
failures.
First off, consider how few components have to be working in order for
the water supply to continue for a while after some sort of catastrophic
failure. We need:
-
Gravity. The water tank is at a high point, and gravity (and the
derived force of air pressure) keeps the pipes pressurised. While gravity
can, in theory, fail, if it actually does then we've got bigger problems.
-
The water tank must be present and with at least some capacity to hold
water. Yep, we can blow a hole in the top or side of the water tank
and still have some (diminished) capacity to supply water. Try blowing
a hole in the side of your data centre and see what happens.
-
Piping from the water tank to the water consumption point. I'd
say this is the weakest link, but it's a pretty robust link overall
(modulo rampaging backhoes) and any other centralised water supply
method (other than perhaps teleportation) is going to be at the mercy of
a piping failure too.
Note that this list of elements doesn't include any moving parts, or even
guaranteed continuous water supply from a dam or other huge supply store.
You can lose your feed pump(s), supply lines, or anything else that's on
the supply side of the water tank for some period of time and nobody on the
consumption side will know or care. Need more resilience to
supply-side failure? Just build a bigger tank.
This blows my mind, it really does. Need to do pump maintenance? No
problem, just make sure that the tank is large enough to service
demand over the period of the "outage", and go for it. Can't find a
pump supplier to give you a restoration SLA of less than a week? Just
make sure you've got a water tank that'll provide for a week's
consumption.
Basically, any supply-side reliability problem can be solved
with "build a bigger tank", and while there's a limit to how big we can
make tanks, I'll bet we know a lot more about building huge water tanks to
withstand some freaky failure conditions than we do about building pumps
that won't fail for 100 years. If you
do need more capacity than
a single tank can provide, just parallelise -- horizontal scaling of the
water supply. Woohoo!
Your big water tank also provides cost savings in your other equipment.
If you had to pressurise a water supply using pumps, not only would you
need your redundant array of very expensive pumps (Hmm, RAVEP-5 sounds
like a Doctor Who villain), but those pumps would need to be able to
provide your peak consumption flow (toilet breaks during the Super Bowl,
probably). I'd imagine that could get mighty expensive, and without
providing much benefit for 99% of the time.
I see this capacity problem at work all the time. Customers who have the
occasional massive traffic spikes need to massively over-provision their
average utilisation to successfully service that 1% of the time that
they're doing heavy traffic. Yes, I know, cloud computing, horizontal
scaling, capacity on demand, yadda yadda yadda. It's not a panacea, and
the number of apps that are designed to properly scale horizontally over a
large range of traffic volumes is miniscule. My general point, though, is
that it when you've got a variable load, it costs a whole lot more to
provision systems to supply peak demand than to provision systems that can
deal with average demand.
A suitably large water tank means you can easily deploy a much smaller
capacity feed pump. All you have to do is make sure that when demand
exceeds supply, you've got enough water in your tank to cover the
difference between the demand and supply over that period. When your peak
capacity increases, and is starting to strain your infrastructure, you've
got a choice, too: you can upgrade the pump
or increase your
storage capacity, whichever is cheaper / easier / quicker / provides
better kickbacks / whatever.
This is all well and good, you say, but computer systems aren't water
supplies. There's a lot more moving parts, inputs, and outputs, and those
all have to be handled. This is quite true. Water-powered computers are
not in high demand. However, I think we could produce some much more
reliable systems if we looked for ways we could simplify capacity and
redundancy issues, water supply style, instead of layering more and more
cruft into our solutions.
Ironically, just after I started writing this post (and this gives you an
idea of how long it's been sitting in my drafts folder), I saw an article
in
the RISKS
digest about a water supply problem in Santa Cruz, caused by a power
outage killing the refill pump for an extended period and resulting in the
storage tank running dry. Hey, I never said that it was a foolproof
system -- especially when you've got failures of imagination that result
in the system that tells people that there's no power relying on the power
source it's monitoring being up in order to be able to tell people that
there's no power...
I maintain several xen machines. Most servers that i maintain use serial consoles (i should probably write another post in more detail about why serial consoles are the One True Way to manage a server). Trouble is, the way that xen works with the serial console has changed between etch and lenny. So what's changed? From what i can tell: 
A while ago, I thought: I seem to be less productive than I used to be. Why might that be? Do I spend more and more time with e-Mails? Or is it something else? I couldn t tell, so I needed a time tracking tool. There are a few of those for Linux that allow you, while working, specify what you are doing and then generate statistics about it. This approach has a few disadvantages:
With the vrms meme raging on Planet Ubuntu, I noticed some people still have sun-java6 installed. I ve been using openjdk-6 since Hardy, and everything I use works fine with it (e.g. 
Dear Lazyweb,
we re using 
Over the last months, I have been asking people in 
This isn t a problem which many companies have to deal with. It s simple for lots of them: ignore things like climate change except when laws or public relations (or employee relations?) require otherwise. As far as the company is concerned, the aim is to make money.
Not so for the third sector. One of the cooperative principles is to work for the sustainable development of our communities through policies approved by our members (taken from 
On 
The idea of breaking programs into functions was taught to some of
us with the metaphor of "divide and conquer." That is, make several
smaller problems and solve those instead.
This approach is now so deeply ingrained that I can just think
"partition and solve" which is of course a much more positive and
accurate description.
One might think of the "partition your program into functions"
principal as "the divide and conquer" principal. But of course, they
are both instance of the more general "partition and solve". I dare
say that "partition your program into functions" is even a more pure
and accurate instance of the principal.
Evolution may also be an instance of the principal. Different
species evolved to solve different problems, or to solve the same
problem (survival) in different ways.
Of course, when you partition a program into functions, you also
define those functions' relationships to each-other. Just because
you're partitioning doesn't mean that the problems are not related.
When people see several competing projects in the "free software
community" trying to solve the same problem, some might say, "Good.
We have divided them, now we will conquer them!" In their marketing
material (or more likely press-release ranting) they will say that a
house divided against itself will not stand [1].
Such people are probably purposefully ignoring the "partition and
solve" principal. They ignore the fact that different projects have
chosen different paths to solve different problems. They ignore the
relationships between the projects, and that we feed each-other much
more often than we actually compete. But no good programmer will
ignore the relationships between her functions.
Even within the 
On recent years I've only been in one or two free software related conferences
per year, usually because I've been invited to go for a day or two and give a
talk.
The truth is that my day to day activities don't leave me enough time to
contribute or participate a little bit more in free software projects, but
this year I decided (and negotiated) that I had to go to