A couple of times I have written code which basically does this:
- accept() an incoming connection
- do some magic to get another socket
- start passing bit between the two sockets
- perform a correct close down sequence
An example use case of this is a proxy implementing the HTTP CONNECT method, but in for some known hostnames it will log a message and mangle the hostname before proceeding. This has been used as a legacy fall back solution while changing a network setup. But my uses are not restricted to HTTP proxies I have done the same for a few legacy protocols where the magic has been of different complexities.
The two final steps are quite general and it would be nice to have a module doing just that. Take two sockets and make it easy (or even automatic) to pass bytes between them.
The na ve non-blocking solution would use a scalar string buffer for each direction and perform a select loop while maintaining the write vector depending on which buffers contain data. I have written this code multiple times. In development this is usually quite successful, in production less so. While Perl might be quite suited for the magic in step 2, the na ve way of passing bytes have quite an overhead for the buffer management.
A less na ve way would use a array of strings for buffers, but I m not quite sure if this would be a win in all cases. You might be able to get away with some string operations on the read side of the buffer, but it might be more expensive on the write side. I have not benchmarked this.
Most of the time I don t care about Perl level IO handles. I know that there is a real C level file descriptor beneath. So an even better POSIX compliant solution might be to use XS to have plain C strings and use readv()/writev() and a iovec structure as buffer.
Can we do even better? At least on Linux we can. With the Linux splice() specific system call it is possible to us a pipe as buffer and never to have to copy data from and to user space.
I have not been able to find any off the shelf solution on CPAN. So I think I need to write it myself, but what would the nice and general API be? I guess the basic interface would be something like:
my $chain = IO::Splice->new($fh1, $fh2);
$chain->pump(); # read and write from both handles if possible and needed
$chain->read($fh1); # read to buffer from one specific handle
$chain->write($fh2); # write from buffer to one specific handle
$chain->can_write(); # returns the handles it needs to write to
but it might be simpler to have two callbacks for setting a file handle in write or no-write state:
my $readset = IO::Select->new( $fh1, $fh2);
my $writeset = IO::Select->new();
my $chain = IO::Splice->new( $fh1, $fh2,
writable => sub $writeset->add( shift ) ,
unwritable => sub $writeset->remove( shift )
);
while ( ... select ... )
$chain->pump();
As said, I think I have plenty of implementations of the na ve way but before releasing some code it would be nice to get some input on the API. But the best feedback would be a module that already have a usable API but might not implement the Linux specific way. That would allow me to steal the interface
It has been a while since I last
ranted about the Deutsche Bahn, our national train service
monopoly. Out of necessity, I ve since become one of their
frequent
travellers. Together with the spiffy, silver card, I received a
pamphlet, in which the advantages of frequent travellers are
listed.
When I compare those advantages to
what was promised, I cannot help but notice quite a few
differences to my
disadvantage. Since I refused to believe
that the Deutsche Bahn could be this stupid, I double-checked with
the service hotline, and I now have the information to report:
-
While the website promises free access to DB lounges for two ,
the pamphlet clarifies this: free access to DB lounges for you and
your partner, provided each of you owns a first-class,
long-distance ticket. What they left out: Oh, you don t need the
frequent traveller status for that, the ticket suffices. Or, put
differently: the frequent traveller status does not give
you any access to the lounges. False advertising, anyone?
-
The pamphlet explains that there are special seat areas reserved
for frequent travellers. However, one must not be a frequent
traveller to use them. So should I expect people to prompt me to
show my card or clear my seat. And should I be expected to prompt
people to flash the card or leave? Not feasible, anyone?
-
I am told that I get priority treatment at the counter, except
there is only one counter (in Munich), usually with a line of
people for 1st-class and frequent travellers. All other travellers
get delegated to 12 counters by an efficient number system, which
means one is better off picking a number and standing in
line. The other day, a lady came and flashed her frequent traveller
card, expecting people to make way, but obviously noone did. Did
they actually think about this, anyone?
I don t need to go into detail on the other benefits : they
claim that there is special, reserved parking, but that s probably
only on paper. They claim reductions in hotels and rental cars, but
probably limited to availability, they claim exclusive events, but
those are likely the ones noone goes to anyway. And they claim a
service hotline, but it s a premium-rate number.
So all in all, Deutsche Bahn have once again managed to
disappoint. The frequent traveller card does not give any benefits.
It rather makes me regret having spent so much money on this
company. Scratch frequent traveller , make it repeat idiot .
NP:
Steven Wilson:
Grace for Drowning
The issue
This morning (while I was running late for an appointement) I
had a very weird stuff happening on my Thinkpad T61 laptop. Since I
recently offered myself a shiny Thinkpad x201s, I have to admit I
don't use much my T61 anymore. But this morning I had to print a
page (for this appointement) and, as I didn't yet configured my
printer on the x201s, I went to the T61. But I noticed that the
network was down. I've tried quickly on wireless but, bad luck, my
current wifi setup selects the channel automatically and it prefers
choosing channels which aren't available in the US. Guess what, my
T61 comes from the US and has those channels completely disabled,
so no wireless available either.
The investigation
I first tried to
modprobe -r e1000e
modprobe e1000e
to see if it fixed the problem, but it didn't. Worse, the
interface disappeared and never reappeared. I tried to reboot but
it didn't fix the problem, the link was still down. Running really
late, I put the file on a usb key and printed it from the powerbook
and postponed the fix for later.
Now, this evening, I tried to investigate a bit more. Symptoms
weren't only that the nic wasn't working, but there was a high load
on the system (1-2 at idle), unresponsiveness every second or so,
and watching top I could see spikes of high cpu usage for the
kworker kernel thread. Typing that on google you can find a lot of
people running on this issue, usually starting around kernel 2.6.36
or 2.6.37. Now, I might have upgraded the kernel recently to
3.0.0-4, but that didn't look related since the problem first
appeared when the laptop was up and running. And I tried to reboot
under 2.6.39, 2.6.38 and even 2.6.32 and the problem was still
present. Each time, unloading the module would fix the problem, but
loading it again wouldn't make the interface reappear. People
advised to boot with pcie_ports=compat but that didn't do anything.
I tried to boot without intel_iommu=force (disable Intel Vt-d) and
pcie_aspm (Active State Power Management) but nothing either.
Considering a userland issue, I've tried to boot a grml live distro (always keep a grml.iso in
your /boot, extlinux-update will even put it in your menu
automatically), and the problem was still present. So not a Debian
kernel issue, not a userland issue, only thing left was the laptop.
I didn't update the Bios recently, so I wondered exactly what could
be the problem. I started to feel a little bad, since I still
really like that laptop, and that I already decided to lend it to
my sister since her own T61 is sitting with a dead system board in
my shelf. I know she might have some negative waves, but she was
not even landed when the problem first appear.
The fix
Then I had a flash. It's not mystery that I'm used to break network
cards, and I had the bright idea to shutdown the laptop,
disconnect AC and battery, then let it idle a bit. I even tried the
secret Thinkpad power button code but I think it's unrelated.
Then I re-plugged the battery, booted to grml and the issue was
gone. I rebooted on the standard Debian and the link was up,
network was working.
So what happenned?
The (tentative) explanation
My guess is that, somehow, the network card firmware has an
issue and choked on something (a network frame or an attack exactly
like the one we demonstrated on ASF firmware). In fact, no, I don't
think it's the e1000e firmware. My T61 comes with Intel vPro, which
includes AMT (Active Management Technology), a remote management
solution a bit like ASF but more advanced. As far as I know, AMT
firmware always runs, even when it's disabled, it's just completely
idle. Idle, but in this case I think it choked on something, and a
reboot isn't enough to restart the AMT firmware. But a real hard
reset without any power seems to do the trick.
What next?
Well, a part of me is pretty scared, but another is just bored.
I mean, we know about that, that's exactly the kind of issue we are
warning people of. I have no idea what exactly happened, and
there's no way I'll be able to reproduce that, but I'm pretty sure
it's something lying at a pretty low level in the platform, and
which can severely disable your workstation. Now if it happens
again I won't lose too much time on this.
TL;DR: helping other people
In case you came here because you searched on google terms like
kworker cpu usage , e1000e, interrupts, it might be a good idea to
first reboot on a live CD to eliminate installation issues, then
shutdown the laptop, remove the battery and let it few seconds
idle. This might be enough to reset something inside and fix the
situation.
Poor Sarah.
Cat dramas always
seem to befall us when she's in Australia.
It looks as if Smudge has managed to escape the house. Being at work all
day, it took me 24 hours to notice.
Our friend Helen came over on Sunday night. Smudge was definitely around
then, because she was climbing all over Liam. Helen used the downstairs
bathroom before she went home, and I remember her having to shoo Smudge
out because she was in there. I'm pretty sure she went home after that. If
my memory serves, she saw herself out and I was in the kitchen cleaning up,
so my current theory is that she escaped when Helen left.
On Monday night, I had a lengthy video call with Sarah, who was at my
parent's place setting up their new Mac Mini. I didn't realise until after
that, when I was about to go to bed, that I hadn't been harassed by Smudge
all evening. So I searched the house that night, and couldn't find her.
Liam had already gone to bed for the night, so I hoped she was stuck in his
room, and went to bed too.
This morning, Liam and I both searched the house, and the immediate
surrounds, without any luck. I called the Palo Alto Animal Services (I got
the dispatch because the shelter hadn't opened yet), emailed
the
local neighbourhood
association, and went to work.
I called Palo Alto Animal Services again later in the day to report her
missing to the shelter itself, and Betty-Ann there recognised my accent and
name and asked me if I was Sarah's husband (Sarah used to volunteer there a
lot, and we adopted Smudge from there), so they're now keeping a look out.
Smudge is micro chipped, but wasn't wearing a collar. She's allowed in the
back yard, because it's totally enclosed and can't escape, so she shouldn't
be totally freaked out by being outdoors. Whenever she does escape out the
front door, she usually makes a beeline to the right, which is mostly
enclosed, but she can certainly get out of the complex. Unfortunately she
often doesn't come when called, so for all I know she's in some obscure
location 10 metres from the front door and I can't see her.
My main concerns are that we're only two blocks from Highway 101, so if she
does stray too far, she could get into serious trouble, and I'm also a bit
worried about raccoons giving her grief.
I spent some time tonight putting up posters in the immediate area and
poking around bushes. I've left the two humane cat traps that we have out by
the front door with some cat food in them, in the hope that she's roaming
around and will walk into one of them.
Oh and the
other big concern I have is that Halloween is coming up.
Shelters tend not to adopt out black cats around Halloween because bad
things can happen to them.
So now we wait. I'm really hoping she turns up safe and sound, soon.
I have several backup servers that run the excellent
rsnapshot software, which uses
Secure Shell (SSH) for remote access. The SSH private key of the backup server can be a weak link in the overall security. To see how it can be a problem, consider if someone breaks into your backup server and manages to copy your SSH private key, they will now have the ability to login to all machines that you take backups off (and that should be all of your machines, right?).
The traditional way to mitigate SSH private key theft is by password protecting the private key. This works poorly in an unattended server environment because either the decryption password needs to be stored in disk (where the attacker can read it) or the decrypted private key has to be available in decrypted form in memory (where attacker can read it).
A better way to deal with the problem is to move the SSH private key to a smartcard. The idea is that the private key cannot be copied by an attacker who roots your backup server. (Careful readers may have spotted a flaw here, and I need to explain one weakness with my solution: an attacker will still be able to login to all your systems by going through your backup server, however it will require an open inbound network connection to your backup server and the attacker will never know what your private key is. What this does is to allow you to more easily do damage control by removing the smartcard from the backup server.)
In this writeup, I ll explain how to accomplish all this on a
Debian/
Ubuntu-system using a
OpenPGP smartcard, a
Gemalto USB Shell Token v2 with gpg-agent/scdaemon from
GnuPG together with
OpenSSH.
First we need to install some packages. The goal is to configure
OpenSSH to talk to the
gpg-agent which will start and talk to
scdaemon which in turn talks to
pcscd which talks to the smart card reader. For some strange reason, the scdaemon binary is shipped with GnuPG s S/MIME interface in the
gpgsm package.
# apt-get install pcscd gnupg-agent gpgsm
The above command should install and start
pcscd and if all works well, you should be able to check the status of the smartcard using GnuPG.
# gpg --card-status
You need to initialize the smartcard and generate a private key on it, again using GnuPG. If you trust GnuPG more than the smartcard to generate a good private key, you may generate the private key using GnuPG and then move it onto the smartcard (hint: use the
keytocard command). Make sure you don t leave a copy of the private key on the same machine!
# gpg --card-edit
gpg: detected reader Gemalto GemPC Key 00 00'
...
gpg/card> admin
Admin commands are allowed
gpg/card> name
Cardholder's surname:
Cardholder's given name: host.example.org
gpg: 3 Admin PIN attempts remaining before card is permanently locked
Please enter the Admin PIN
gpg: gpg-agent is not available in this session
gpg/card> lang
Language preferences: en
gpg/card> generate
Make off-card backup of encryption key? (Y/n) n
Please enter the PIN
What keysize do you want for the Signature key? (2048)
What keysize do you want for the Encryption key? (2048)
What keysize do you want for the Authentication key? (2048)
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
"Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: host.example.org
Email address:
Comment:
You selected this USER-ID:
"host.example.org"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (33 seconds)
gpg: signatures created so far: 0
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (18 seconds)
gpg: signatures created so far: 1
gpg: signatures created so far: 2
gpg: existing key will be replaced
gpg: please wait while key is being generated ...
gpg: key generation completed (23 seconds)
gpg: signatures created so far: 3
gpg: signatures created so far: 4
gpg: key 12345678 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 2048R/12345678 2011-09-19
Key fingerprint = 1234 5678 1234 5678 1234 5678 1234 5678 1234 5678
uid host.example.org
sub 2048R/23456789 2011-09-19
sub 2048R/34567890 2011-09-19
gpg/card> quit
#
Now for the interesting part. OpenSSH talks to an agent for private key handling, and GnuPG s gpg-agent supports this protocol when the
--enable-ssh-support parameter is given. During startup, gpg-agent will print some environment variables that needs to be set when
ssh is run. Normally gpg-agent is invoked by the Xsession.d login scripts, so that the environment variables are inherited by all your processes. However, for an unattended machine without any normal login process, we need to write a script to start gpg-agent. First do these manual steps, to confirm that everything works.
# gpg-agent --daemon --enable-ssh-support > /var/run/gpg-agent-info.env
# . /var/run/gpg-agent-info.env
# ssh-add -L
ssh-rsa AAAAB3N... cardno:000500000BD8
#
If the final step printed a SSH public id, the (sometimes) tricky part in getting the hardware to work is (hopefully) complete. What remains is to script things so that
gpg-agent is started on boot and to make sure that your backup scripts has the proper environment variables before launching whatever processes will launch
ssh. Further, since we will be running unattended, we need a mechanism to unlock the smartcard using a PIN interactively once on each boot of the machine. I prefer manually entering the PIN on every boot over having the PIN stored in a file on the disk.
I will use the
/etc/rc.local mechanism to start gpg-agent, like this:
# cat> /etc/rc.local
#!/bin/sh -e
exec gpg-agent --daemon --enable-ssh-support \
--pinentry-program /usr/local/sbin/pinentry-unattended \
--write-env-file /var/run/gpg-agent-info.env
^D
The astute reader will now ask what
/usr/local/sbin/pinentry-unattended is and why it is needed. Now here is the situation. scdaemon will normally query the user for a PIN using a tool called
pinentry which reads and write to the user s TTY directly. This won t work in unattended mode, so we want the scdaemon to signal failure here unless we are actually unlocking the smartcard manually. Here is the entire script:
#!/bin/sh
# /usr/local/sbin/pinentry-unattended -- by Simon Josefsson
if test x"$PINENTRY_USER_DATA" = xinteractive; then
exec pinentry "$@"
fi
exit 1
What remains is a script to unlock the smartcard by providing the PIN. This is typically invoked manually if the server has restarted for some reason. Don t worry, any ssh sessions invoked by cron until you have managed to unlock the smartcard will fail with an authentication error it won t hang waiting for a PIN to be entered.
#!/bin/sh
# /usr/local/sbin/unlock-smartcard -- by Simon Josefsson.
. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID
gpg-connect-agent 'scd killscd' /bye > /dev/null
while ! gpg-connect-agent 'scd serialno' /bye grep -q SERIALNO; do
sleep 1
done
PINENTRY_USER_DATA=interactive
export PINENTRY_USER_DATA
checkpin
And the script
checkpin is as follows:
#!/bin/sh
# /usr/local/sbin/checkpin -- by Simon Josefsson.
id= gpg-connect-agent 'scd serialno' /bye head -1 cut -d\ -f3
gpg-connect-agent "scd checkpin $id" /bye grep -q OK
At this point, you should have everything configured and installed. Don t forget to
chmod +x the scripts. The typical use-pattern is as follows. After the machine has been started, gpg-agent is running but the smartcard is not unlocked with the PIN. You need to manually login to the machine and run unlock-smartcard and enter the PIN. In the script that runs the backup jobs, invoked via cron, make sure that the first line of the scripts reads (assuming Bourne shell script syntax):
. /var/run/gpg-agent-info.env; export GPG_AGENT_INFO SSH_AUTH_SOCK SSH_AGENT_PID
To avoid needlessly attempting ssh connections if the smartcard is not unlocked, your backup script can also call the
checkpin code and abort if it doesn t return true.
checkpin exit 1
Some final words about debugging. A basic command to run to check that the GnuPG side is working is
gpg --card-status, it should print some information about the smartcard if successful. To check that the SSH agent part is working, use
ssh-add -L. If you get error messages, try killing the
scdaemon process by running
killall -9 scdaemon and let
gpg-agent respawn a new
scdaemon process.
That s it! If you like my writeup, please flattr it.
I was expecting to write more about the 7th Rugby World Cup, but real
life, Debian work and running activities prevented me to do so..
Still, I watched several games and I can share my feelings now.
First of all, this is a tremendous organization from New Zealand. It
seems that about the entire country is working on making this even a
great success and I really appreciate to see a place I certainly have
to visit some day receiving such wordlwide attention (OK, admitedly,
mostly in the part of the world that understands rugby).
First round already gave several surprises even if the 1/4 finals were
after all kinda expected (at least, the list of countries).
- New Zealand All Blacks is by far the most shiny team. They crushed
all their opponents, including our French team, and were indeed
brilliant. So, really, the challenge is: who can beat them?
- Australia and South Africa were about the same. Australia has been
surprised by Ireland (one of the most exciting games of first round)
and is apparently no longer the incredible team that scored over 60
points in France back in November 2010. South Africa has been strong
and powerful during 1st round... but, indeed, not that impressive (I'm
not fond of south-african game, anyway).
- Argentina was again great. They had to qualify in a quite hard
pool and made it well. Funny to see that, at many moments, the 15
players on the field were players....that usually play in French club
teams. They were even sometimes talking in French with some referees
(either the excellent Romain Poite, from France, and the also
excellent M. Rolland, the most french Irish referee)
- Wales has been serious and strong enough to qualify in a not so
easy pool. Their rear lines are the most dangerous among qualified
teams, along with NZ
- Ireland made an excellent pool game. Their win against Australia
was really a surprise and a nice illustration of what "Irish fighting
spirit is". I'm not always fond of Irish rugby organisation (provinces
only playing4 to 5 really serious games in a season), but it gives
results
- Scotland is really reviving. OK, they didn't qualify, but this
team is now far from the very bad team they had during the 2000
years. Rear players as well as their 2m-tall front player (whose name
I can't remember now) are excellent
- England. Ah, England..:-). Seeing our "best enemy" in rugby being
not so good was kinda a pleasure for me..:-). I just feel sad to see
Mister Johnny Wilkinson in not so good shape, because that player is
really someone I have respect for. Others...I have less respect,
sometimes, I admit (particularly this Ashton guy who I expect to crash
in the grass, some day, when arrogantly diving when scoring a
try). OK, that's England. Always running love and hate story..:-)
- Fiji, Tonga and Samoa weren't as decisive and challenging as they
used to be. Out of these three "Pacific Islands" countries, the Tonga
team was IMHO the best and their match against France was a monument
of courage. They would even have deserved to qualify and we have to
thank Canada for beating them..:-)
- Italy was a deception for me. Their key players are aging and they
have hard times renewing the team. And now, definitely, having a good
scrum is not enough.
- Among the "have no chance to qualify" teams, I have a good
impression of Canada. With so few players in the country, it is
amazing to see that they can build a team able to beat Tonga and to
play a great game against France. Japan also made at least one
excellent game against France. I haven't seen many others, sorry for
that.
- And how about France? I have to say they have been very
French..:-). The pool round was nearly a disaster with very few good
spirit in this team, not even a good scrum and way too many
imprecisions and approximations. Still, they qualified.
Which lead us to the following 1/4 finals (in parenthesis are my
bets):
-
Wales-Ireland (Ireland)
-
England-France (England)
-
Australia-South Africa (South Africa)
-
New Zealand-Argentina (New Zealand)
If you know about the results, you know I screwed it nearly
completely..:-)
The Welsh team played a great game against an uninspired Irish
team. It has certainly been the best 1/4 final and they certainly
deserve their win. Even though I usually tend to be supprotive of
Ireland, I was very balanced here, and finally turned out to be in
favor of the Welsh. We really have to fear them in semi-finals.
Australia-South Africa was theoretically the most exciting 1/4 final
but finally turned out to be quite boring. Both teams insisted on
playing mostly to occupy their opponents part of the field, more than
trying to score, then relying on penalties to score. That was
apparently the good tactics for Australia ad they also deserve their
win against a South-African team where forward players were not as
decisive as they sometimes are.
Argentina was again there and really there. It has been the only team
up to now who lead score against New Zealand. And that was
deserved. What a wonderful 1st halftime! Obviously, it was impossible
for them to resist during second halftime and it slowly became obvious
that the Blacks (saved during 1st half by the kicks of a very inspired
Kiri Weepu) would finally manage to score tries. But, still, our
argentinian friends, for instance the inoxydable "Super Mario"
Ledesma, or the tireless Felipe Contepomi, were not here as a
sacrificial victim.
What to say about England-France? First half was astonishing for us,
of course. This is what we love (and hate) with our beloved French
team. Definitely the team that can make surprises and, imho, the only
one that can beat New Zealand if that has to happen (but also the only
one that can be entirely crushed by them). The defeat against Tonga
and the week that followed completely transformed them. A stunning 3rd
row, defending each and every single bit of England trying to invade
"la patrie en danger". Rear lanes with the magicians of Toulouse
(Clerc and M dard) as the ideal finishers of magic play by Parra,
Trinh Duc, Mermoz, Palisson (the good surprise of this world cup,
Alexis). And, during second half, a trilling resistance to assaults of
the British White Knights, concluded by this delivering drop-goal by
Trinh Duc. For sure, with games like this, they can beat everybody and
by everybody, I mean everybody. Remember Millenium Stadium in
2007..:-)
So, well. Australia-New Zealand and France-Wales. I know where my
heart is balancing for both games. The Blacks and Les Bleus in final,
thi is what we hope (and fear...), but both teams, particularly
France, will have to first climb a quite big wall before reaching
this.
Our friends Andrew and Janice finally tied the knot last Sunday, in
Langkawi, Malaysia.
They're Aussies currently living in New York, and Janice is of Malaysian
descent, so I think part of the reason was to accommodate some Malaysian
relatives that wouldn't be able to travel, and the other reason was that it
was going to mean a bunch of people had to travel anyway, so why not travel
somewhere a bit exotic?
As my sister is getting married later this month in Brisbane, we decided to
spare Zoe two massive international flights in close succession, and Sarah
and Zoe flew out a few days ahead of me, and went to Brisbane, and then Zoe,
Sarah and her mum flew to Langkawi from there, and I flew there from San
Francisco, with the plan being for Sarah and Zoe to hang out in Brisbane for
the 4 weeks in between weddings.
It was the longest amount of travel I think I've ever done. I hadn't sat
down and calculated what the travel time was beforehand, and when I got on
the flight from San Francisco to Hong Kong, and they said it was going to be
nearly 14 hours
just to Hong Kong, I nearly wept. The only good
thing about it was I didn't have to wrangle a toddler as well.
From there, I had to get to Singapore, with a several hour layover, and then
another flight from Singapore to Langkawi. I left SFO at 1:20am on Monday
morning, and finally arrived at about 7am (San Francisco time) the following
day.
I didn't even bother getting off the plane in Hong Kong, it was only an hour
and a half layover. I think the layover in Singapore was 5 hours, and it was
a separate flight. Singapore airport at least was pretty good to kill time
in.
It was funny, I was reading Wired while I was killing time in Singapore, and
they had an article, which included a side bar on
10 Best
Airports to Get Stuck In, and I observed that getting to Langkawi and
back involved using 4 of them.
The flight into Langkawi was more eventful than usual. I was dozing a bit as
we were coming in to land, and I woke up to the sensation of taking off
again. I wondered if we'd had an aborted landing? Sure enough, soon after,
the captain got on the public address and said that they'd had to abort the
landing due to bad weather in Langkawi. He had pretty bad Engrish, thanking
us for our frustration and saying something about fuel, so I didn't get a
lot of the details. We proceeded to circle for a while before finally
landing. I think the flight ended up landing something like an hour or more
later than scheduled. So that was mildly exciting. Langkawi airport was one
of those little "walk across the tarmac" places. I think I stood the closest
to an operating jet engine that I ever have, as another aeroplane taxied in
to a stop between my plane and the terminal while I was walking to it. Loud.
Anyway, that was all just me getting there. Sarah had a bit of fun getting
to Brisbane on her own with Zoe. She's now too long to sleep in an in-flight
bassinet, and doesn't seem to like sleeping vertically in the Ergo Baby
Carrier (possibly she was too hot in it). I think she got a bit of sleep
lying across Sarah's lap, but Sarah didn't get much sleep. I think there was
similar fun for the Brisbane to Kuala Lumpur flight, but at least she had
two laps to lie across for that one. Sarah, Zoe and Sarah's mum arrived in
Langkawi earlier in the day on Tuesday, and I made it to the resort at about
10pm (local time) and pretty much immediately crashed for the night.
The
Westin
Langkawi was just fabulous. I don't think we've stayed at a resort since
our honeymoon, and this resort was a whole other level compared to
Heron Island.
The rooms were very comfortable, and the resort was very accommodating of a
toddler. Their Kids Club menu was excellent. The breakfast buffet (included
in our room rate) was the most comprehensive breakfast buffet I have ever
seen in a hotel. The only thing missing was bacon, being a Muslim country,
but I can live without that. It was pretty easy to just fill up on
breakfast, skip lunch and just have a light dinner. The on-resort dining
prices were also very reasonable. The off-resort prices were even cheaper.
The resort facilities in general were very comprehensive. Three pools. A
children's play room. A well equipped gym and recreational area including
squash courts. An extremely nice day spa.
The thing I liked best about the whole destination wedding thing was all of
the activities leading up to the actual wedding. The wedding was on the
Sunday, and we had the Hen's and Buck's parties on Friday. Apparently
Australia is the only place that calls it a Buck's party. The rest of the
world calls it a Stag party, how about that? There was some last-minute
changes to the Buck's party, and we ended up chartering this huge $2 million
yacht for 4 hours and going for a cruise and doing some jetskiing and water
sports. The captain was Aussie, so we all had a good chat with him.
Apparently it's a corporate vessel, with an exclusive chartering arrangement
with the Westin.
On Friday night, we all went out to a seafood buffet dinner at the Yellow
Cafe over at Cenang Beach. Most of the guys and girls already knew each
other courtesy of the day's activities, so dinner involved a lot of meeting
the other halves.
What I really liked about all of the pre-wedding stuff was that by the time
the wedding rolled around, you really felt like you knew a bunch of Andrew
and Janice's family and friends, which I think made the whole thing feel a
lot more intimate.
Saturday was a recovery day. Sarah's aunties Glenda (who was in Singapore on
business) and Vicki (who lives in Thailand) had joined us at the resort by
this stage, so we were all knocking around together checking out the island.
We took the
cable car to
the top, but unfortunately the sky bridge was closed due to the inclement
weather.
There was a lot of touristy stuff at the bottom of where the cable car was,
and Zoe got to have an encounter with a very large python. She even tried to
give it a kiss.
One of the things that was surprising was how popular Zoe was with all of
the locals and other Asian tourists. We could probably have charged for
photographs. There was a little girl, who I thought was possibly a similar
age to Zoe (turned out she was two and a half) and I asked her parents if I
could take a photo of the two girls together, and no sooner had they sat
down together and a virtual paparazzi materialised out of nowhere taking
photographs. Similarly, all of the staff at the Westin were totally taken by
Zoe, and wanted to touch her. To her credit, Zoe handled all of the extra
attention pretty well.
The wedding wasn't until 6pm on Sunday, so we did a 4 hour island hopping
boat tour in the morning. That was a bit of an experience. It was nothing
short of sheer bedlam trying to board the right boat at the wharf (I use the
term lightly) and then the boat's captain had to arrange the passengers
appropriately so as to keep the boat on an even keel. It was a narrow motor
boat with a canopy cover over the back two thirds of it.
First stop was Dayang Bunting Island, home of Pregnant Maiden Lake, where we
stopped for 45 minutes. More bedlam ensued getting off the boat at the wharf
there (there was veritable flotilla of tour boats all trying to load and
unload at the same time). Monkeys abounded. They seemed incredibly tame. We
were practically tripping over them. Zoe didn't seem to be as taken by the
monkeys as I thought she would. It was a short walk up a reasonable flight
of stairs to get to the lake, though some incredibly humid jungle, and once
we got there, there wasn't a huge amount to do in the time we had available.
So we dunked Zoe's feet in the lake a few times and then made our way back
to the chaos at the wharf and got back on our boat.
We stopped for a similar amount of time at another island with a pretty nice
beach, and Zoe had enough time to a quick swim in the ocean with Sarah, and
then we made our way back to the boat again to another spot where we watched
sea eagle feeding. Heaps of eagles circling overhead would take turns to
swoop down and grab fish and fly off again. It was impressive. I think I
managed to snap some good action shots with the long lens.
Then we had the wedding itself. It was on the beach out the front of the
Heavenly Spa on the resort. Let me say, the Westin knows how to put a
wedding together. The ceremony was as lovely as the location.
For the reception, the one 5 bedroom villa on the resort was booked for the
night. Dinner was a buffet on the lawn out the front (briefly delayed thanks
to a rain shower) and then speeches and general partying happened around the
terrace out the back of the villa. There were some beautiful speeches.
Janice's brought a tear to my eye. Part of the whole package included
something like a 10 person video crew that were running around capturing
everything. It felt a bit surreal. It was definitely the most high-end
wedding I've ever been to (sorry Scott and Julie, your "Wedding of the
Century" has been eclipsed by the "Wedding of the Millennium").
Monday was a wind down day. Zoe mercifully gave us a sleep in until 8am.
Our flights didn't leave until the evening on Tuesday, so after we checked
out we went for a drive around the island to try and get Zoe to take a nap
in the car. We ended up at the
fruit
farm, which to the best of my understanding is run by the government.
Zoe had fallen asleep by this stage, and Sarah said she'd stay behind in the
car and have a nap too. So her mum and I went on the tour. It was a fun
little tour. There were only four of us on it. They drive you around the
farm on the back of a truck and show you the various crops they have. They
stop at a few places and give you a bunch of fruit samples, attempt to
answer your questions, and take funny pictures of you. The favourite seemed
to be to dangle a fruit on a knife up close in front of the camera while
having the subject(s) in the background look like they're carrying an
enormous fruit.
We got back to the car to an upset Sarah and small group of locals milling
around. Sarah had managed to lock Zoe in the car for about 30 minutes when
she got out to make her a sandwich after Zoe had woken up from her nap.
Fortunately the engine was running and the air conditioning was on, so there
was no risk of Zoe overheating (on the contrary, she was a bit on the cool
side). Some kind locals had somehow figured out which car rental company to
call (it's possible there's only one) and someone came out with a spare key,
so the crisis was averted just before we got back to the car. It's the first
time Zoe's been locked in the car (it's pretty much impossible in the Prius)
and it was just our luck that it'd happen in a foreign country!
One of the saddest things about Langkawi was the amount of garbage floating
around in the ocean, and washing up on the beaches. I was swimming in the
ocean off the yacht while we were out on the Buck's party, and it was
disgusting. The jetski was temporarily disabled due to sucking in something.
We visited Black Sand Beach, which was supposed to be something of a tourist
attraction. Covered in trash.
Ian Kiernan would be
sad. The captain of the yacht we were on for the Buck's party said it was
all from Thailand and got flushed out of all the rivers and streams by the
recent rain. Just goes to show how connected everything is.
The house is very quiet without Sarah and Zoe, but my friend Liam is
visiting from Zurich for a couple of weeks, so that's helping break the
silence. I'm using the time to get a heap of things done that I don't
otherwise get time to get done.
Unfortunately the photos from the trip are only available on Facebook at the
moment, but they'll eventually get uploaded
here.
For
Ada Lovelace Day this year, I want to share my appreciation for Dr. Marian C. Diamond.
In
years past, I ve saluted women in the field of computing, which is my field as well. Dr. Diamond, however, is a biologist. Her research includes neuroanatomy, environment, immune functions, and hormones. In particular, she is interested in studying the effects of the external environment, aging, and immune responses on the cerebral neocortex. She has, in her words, had a
love affair with the brain for about 70 years.
I know very little about biology. The content and methods of her research are, frankly, beyond me, though some of her results have garnered
popular attention. She has inspired me by demonstrating that rare combination of gifts: a deep understanding of a technical subject, and the ability to explain it to other people in an accessible way.
In her interviews, articles and lectures,
many of
which are available online, Dr. Diamond displays these gifts in abundance. Her skill and enthusiasm for both
learning and teaching is unmistakable. After applying her gifts in the classroom for many years, digital distribution has now enabled many more people to see and hear her, through millions of YouTube views.
In 1960, she became the
first female graduate student in UC Berkeley s anatomy department, and was apparently given the job of
sewing a cover for a magnifying machine. I can only imagine the persistence required to continue from there to become a recognized leader in her field. She has gone on to help many other students along their way, and was named an
unsung, everyday hero for the support she provided to students outside of the classroom or lab.
As if that weren t enough, she has also traveled to Cambodia to apply her expertise in
helping children injured by land mines. She still teaches today, just across the bay from where I write this, and will turn 85 next month.
It's become kind of a clich for me to claim that the reason I'm happy working on ACPI and UEFI and similarly arcane pieces of convoluted functionality is that no matter how bad things are there's at least some form of documentation and there's a well-understood language at the heart of them. My PhD was in biology, working on fruitflies. They're a poorly documented set of layering violations which only work because of side-effects at the quantum level, and they tend to die at inconvenient times. They're made up of 165 million bases of a byte code language that's almost impossible to bootstrap[1] and which passes through an intermediate representations before it does anything useful[2]. It's an awful field to try to do rigorous work in because your attempts to impose any kind of meaningful order on what you're looking at are pretty much guaranteed to be sufficiently naive that your results bear a resemblance to reality more by accident than design.
The field of bioinformatics is a fairly young one, and because of that it's very easy to be ignorant of its history. Crick and Watson (and those other people) determined the structure of DNA. Sanger worked out how to sequence proteins and nucleic acids. Some other people made all of these things faster and better and now we have huge sequence databases that mean we can get hold of an intractable quantity of data faster than we could ever plausibly need to, and what else is there to know?
Margaret Dayhoff graduated with a PhD in quantum chemistry from Columbia, where she'd performed computational analysis of various molecules to calculate their resonance energies[3]. The next few years involved plenty of worthwhile research that aren't relevant to the story, so we'll (entirely unfairly) skip forward to the early 60s and the problem of turning a set of sequence fragments into a single sequence. Dayhoff worked on a suite of applications called "Comprotein". The original paper can be downloaded
here, and it's a charming look back at a rigorous analysis of a problem that anyone in the field would take for granted these days. Modern fragment assembly involves taking millions of DNA sequence reads and assembling them into an entire genome. In 1960, we were still at the point where it was only just getting impractical to do everything by hand.
This single piece of software was arguably the birth of modern bioinformatics, the creation of a computational method for taking sequence data and turning it into something more useful. But Dayhoff didn't stop there. The 60s brought a growing realisation that small sequence differences between the same protein in related species could give insight into their evolutionary past. In 1965 Dayhoff released the first edition of the Atlas of Protein Sequence and Structure, containing all 65 protein sequences that had been determined by then. Around the same time she developed computational methods for analysing the evolutionary relationship of these sequences,
helping produce the first computationally generated phylogenetic tree. Her single-letter representation of amino acids was born of necessity[4] but remains the standard for protein sequences. And the atlas of 65 protein sequences developed into the Protein Information Resource, a dial-up database that allowed researchers to download the sequences they were interested in. It's now part of
UniProt, the world's largest protein database.
Her contributions to the field were immense. Every aspect of her work on bioinformatics is present in the modern day larger, faster and more capable, but still very much tied to the techniques and concepts she pioneered. And so it still puzzles me that I only heard of her for the first time when I went back to write the introduction to my thesis. She's remembered today in the form of the
Margaret Oakley Dayhoff award for women showing high promise in biophysics, having died of a heart attack at only 57.
I don't work on fruitflies any more, and to be honest I'm not terribly upset by that. But it's still somewhat disconcerting that I spent almost 10 years working in a field so defined by one person that I knew so little about. So my contribution to Ada Lovelace day is to highlight a pivotal woman in science who heavily influenced my life without me even knowing.
[1] You think it's difficult bringing up a compiler on a new architecture? Try bringing up a fruitfly from scratch.
[2] Except for the cases where the low-level language itself is functionally significant, and the cases where the intermediate representation is functionally significant.
[3] Something that seems to have involved a lot of putting punch cards through a set of machines, getting new cards out, and repeating. I'm glad I live in the future.
[4] The three-letter representation took up too much space on punch cards
comments
The Ubuntu Technical Board conducts a regular review of the most popular
Ubuntu Brainstorm ideas (previous reviews conducted by
Matt Zimmerman and
Martin Pitt). This time it was my turn. Apologies for the late arrival of this review.
Contact lens in the Unity Dash (
#27584)
Unity supports
Lenses, which provide a consistent way for users to quickly search for information via the Dash. Current lenses include Applications, Files, and Music, but a number of people have asked for contacts to be accessible using the same interface.
While Canonical's DX team isn't currently working on this for Ubuntu 11.10 or 12.04, we'd love somebody who's interested in this to get involved. Allison Randal
explains how to get started, including some skeleton example code and several useful links.
Displaying Ubuntu version information (
#27460)
Several people have asked for it to be more obvious what Ubuntu version they're running, as well as other general information about their system.
John Lea, user experience architect on the Unity team, responds that in Ubuntu 11.10 the new LightDM greeter shows the Ubuntu version number, making that basic information very easily visible. For more detail, System Settings -> System Info provides a simple summary.
Volume adjustments for headphone use (
#27275)
People often find that they need to adjust their sound volume when plugging in or removing headphones. It seems as though the computer ought to be able to remember this kind of thing and do it automatically; after all, a major goal of Ubuntu is to make the desktop Just Work.
David Henningson, a member of Canonical's OEM Services group and an Ubuntu audio developer,
responds on his blog with a summary of how PulseAudio jack detection has improved matters in Ubuntu 11.10, and what's left to do:
The good news: in the upcoming Ubuntu Oneiric (11.10), this is actually working. The bad news: it isn't working for everyone.
Making it easier to find software to handle a file (
#28148)
Ubuntu is not always as helpful as it could be when you don't have the right software installed to handle a particular file.
Michael Vogt, one of the developers of the Ubuntu Software Center, responded to this. It seems that most of the pieces to make this work nicely are in place, but there are a few more bits of glue required:
Thanks a lot for this suggestion. I like the idea and it's something that software-center itself supports now. In the coming version 5.0 we will offer to "sort by top-rated" (based on the ratings&reviews data). It's also possible to search for an application based on its mime data. To search for a mime-type, you can enter "mime:text/html" or "mime:audio/ogg" into the search field. What is needed however is better integration into the file manager nautilus. I will make sure this gets attention at the next developer meeting and filed bug #860536 about it.
In nautilus, there is now a button called "Find applications online" available as an option when opening an unknown file or when the user selects "open with...other application" in the context menu. But that will not use the data from software-center.
Show pop-up alert on low battery (
#28037)
Some users have reported on Brainstorm that they are not alerted frequently enough when their laptop's battery is low, as they clearly ought to be.
This is an odd one, because there are already several power alert levels and this has been working well for us for some time. Nevertheless, enough people have voted for this idea that there must be something behind it, perhaps a bug that only affects certain systems. Martin Pitt, technical lead of the Ubuntu desktop team, has
responded directly to the Brainstorm idea with a description of the current system and how to file a bug when it does not work as intended.
We use SugarCRM at work and I've complained about its not-very-RESTy
REST interface. John Mertic a (the?) SugarCRM Community Manager asked
me about what problems I'd had (apart from its lack of RESTfulness)
and I said I'd write a blog post about it.
In our case, the REST interface is used to integrate Sugar and RT so
we get a link in both interfaces to jump from opportunities to the
corresponding RT ticket (and back again). This should be a fairly
trivial exercise or so you would think.
The problems, as I see it are:
- Not REST-y.
- Exposes the database tables all the way through the REST interface
- Lack of useful documentation forcing the developer to cargo cult and
guess
- Annoying data structures
- Forced pagination
My first gripe is the complete lack of REST in the URLs. Everything
is just sent to
https://sugar/service/v2/rest.php. Usually a POST,
but sometimes a GET. It's not documented what to use where.
The POST parameters we send when logging in are:
method=>"login"
input_type=>"JSON"
response_type=>"JSON"
rest_data=>json($params)
$params is a hash as follows:
user_auth =>
user_name => $USERNAME,
password => $PW,
version => "1.2",
,
application => "foo",
Nothing seems to actually care about the value of
application, nor
about the
user_auth.version value. The password is the md5 of the
actual password, hex encoded. I'm not sure why it is, as this adds
absolutely no security, but it is. This is also not properly
documented.
This gives us a JSON object back with a somewhat haphazard selection
of attributes (reformatted here for readability):
"id":"<hex session id>,
"module_name":"Users",
"name_value_list":
"user_id":
"name":"user_id",
"value":"1"
,
"user_name":
"name":"user_name",
"value":"<username>"
,
"user_language":
"name":"user_language",
"value":"en_us"
,
"user_currency_id":
"name":"user_currency_id",
"value":"-99"
,
"user_currency_name":
"name":"user_currency_name",
"value":"Euro"
What is the
module_name? No real idea. In general, when you get
back an
id and a
module_name field, it tells you that the id
exists is an object that exists in the context of the given module.
Not here, since the session id is not a user.
The worst here is the
name_value_list concept which is used all over
the REST interface. First, it's not a list, it's a hash. Secondly, I
have no idea what would be wrong by just using keys directly in the
top level object, so the object would have looked somewhat like:
"id":"<hex session id>,
"user_id": 1,
"user_name": "<username>,
"user_language":"en_us",
"user_currency_id": "-99",
"user_currency_name": "Euro"
Some people might argue that since you can have custom field names
this can cause clashes. Except, it can't, since they're all suffixed
with
_c.
So we're now logged in and can fetch all opportunities. This we do by
posting:
method=>"get_entry_list",
input_type=>"JSON",
response_type=>"JSON",
rest_data=>to_json([
$sid,
$module,
$where,
"",
$next,
$fields,
$links,
1000
])
$sid is our session id from the login$module is "Opportunities"$where is opportunities_cstm.rt_id_c IS NOT NULL. Yes, that's
right. An SQL fragment right there and you have to know that you'll
join the opportunities_cstm and opportunities tables because we
are using a custom field. I find this completely crazy.$next starts out at 0 and we're limited to 1000 entries at a time.
There is, apparently, no way to say "just give me all you have".$fields is an array, in our case consisting of id, name,
description, rt_id_c and rt_status_c. To find out the field
names, look at the database schema or poke around in the SugarCRM
studio.$links is to link records together. I still haven't been able to
make this work properly and just do multiple queries.- 1000 is the maximum number of records. No, you can't say -1 and get
everything.
Why this is a list rather than a hash? Again, I don't know. A hash
would make more sense to me.
The resulting JSON looks like:
"result_count" : 16,
"relationship_list" : [],
"entry_list" : [
"name_value_list" :
"rt_status_c" :
"value" : "resolved",
"name" : "rt_status_c"
,
[ ]
,
"module_name" : "Opportunities",
"id" : "<entry_uuid>"
,
[ ]
],
"next_offset" : 16
Now,
entry_list actually is a list here, which is good and all, but
there's still the annoying
name_value_list concept.
Last, we want to update the record in Sugar, to do this we do:
method=>"set_entry",
input_type=>"JSON",
response_type=>"JSON",
rest_data=>to_json([
$sid,
"Opportunities",
$fields
])
$fields is not a
name_value_list, but instead is:
"rt_status_c" : "resolved",
"id" : "<status text>"
Why this works and my attempts at using a proper
name_value_list
didn't work? I have no idea.
I think that pretty much sums it up. I'm sure there are other
problems in there (such as the over 100 lines of support code for the
about 20 lines of actual code that does useful work), though.
Some years ago I came across a really peculiar newsgroup post. It was not about technicalities of any sort. It was about history. A beautifully written history of computers. From the Turing machine to the Free Software world, the original author managed to capture all the important events of the computer revolution with a great deal of humor. Re-posting it here, it is just brilliant.
The Gospel of Tux (v1.0)
In the beginning Turing created the Machine.
And the Machine was crufty and bodacious, existing in theory only. And von Neumann looked upon the Machine, and saw that it was crufty. He divided the Machine into two Abstractions, the Data and the Code, and yet the two were one Architecture. This is a great Mystery, and the beginning of wisdom.
And von Neumann spoke unto the Architecture, and blessed it, saying, Go forth and replicate, freely exchanging data and code, and bring forth all manner of devices unto the earth. And it was so, and it was cool. The Architecture prospered and was implemented in hardware and software. And it brought forth many Systems unto the earth.
The first Systems were mighty giants; many great works of renown did they accomplish. Among them were Colossus, the codebreaker; ENIAC, the targeter; EDSAC and MULTIVAC and all manner of froody creatures ending in AC, the experimenters; and SAGE, the defender of the sky and father of all networks. These were the mighty giants of old, the first children of Turing, and their works are written in the Books of the Ancients. This was the First Age, the age of Lore.
Now the sons of Marketing looked upon the children of Turing, and saw that they were swift of mind and terse of name and had many great and baleful attributes. And they said unto themselves, Let us go now and make us Corporations, to bind the Systems to our own use that they may bring us great fortune. With sweet words did they lure their customers, and with many chains did they bind the Systems, to fashion them after their own image. And the sons of Marketing fashioned themselves Suits to wear, the better to lure their customers, and wrote grave and perilous Licenses, the better to bind the Systems. And the sons of Marketing thus became known as Suits, despising and being despised by the true Engineers, the children of von Neumann.
And the Systems and their Corporations replicated and grew numerous upon the earth. In those days there were IBM and Digital, Burroughs and Honeywell, Unisys and Rand, and many others. And they each kept to their own System, hardware and software, and did not interchange, for their Licences forbade it. This was the Second Age, the age of Mainframes.
Now it came to pass that the spirits of Turing and von Neumann looked upon the earth and were displeased. The Systems and their Corporations had grown large and bulky, and Suits ruled over true Engineers. And the Customers groaned and cried loudly unto heaven, saying, Oh that there would be created a System mighty in power, yet small in size, able to reach into the very home! And the Engineers groaned and cried likewise, saying, Oh, that a deliverer would arise to grant us freedom from these oppressing Suits and their grave and perilous Licences, and send us a System of our own, that we may hack therein! And the spirits of Turing and von Neumann heard the cries and were moved, and said unto each other, Let us go down and fabricate a Breakthrough, that these cries may be stilled.
And that day the spirits of Turing and von Neumann spake unto Moore of Intel, granting him insight and wisdom to understand the future. And Moore was with chip, and he brought forth the chip and named it 4004. And Moore did bless the Chip, saying, Thou art a Breakthrough; with my own Corporation have I fabricated thee. Thou thou art yet as small as a dust mote, yet shall thou grow and replicate unto the size of a mountain, and conquer all before thee. This blessing I give unto thee: every eighteen months shall thou double in capacity, until the end of the age. This is Moore s Law, which endures unto this day.
And the birth of 4004 was the beginning of the Third Age, the age of Microchips. And as the Mainframes and their Systems and Corporations had flourished, so did the Microchips and their Systems and Corporations. And their lineage was on this wise:
Moore begat Intel. Intel begat Mostech, Zilog and Atari. Mostech begat 6502, and Zilog begat Z80. Intel also begat 8800, who begat Altair; and 8086, mother of all PCs. 6502 begat Commodore, who begat PET and 64; and Apple, who begat 2. (Apple is the great Mystery, the Fruit that was devoured, yet bloomed again.) Atari begat 800 and 1200, masters of the game, who were destroyed by Sega and Nintendo. Xerox begat PARC. Commodore and PARC begat Amiga, creator of fine arts; Apple and PARC begat Lisa, who begat Macintosh, who begat iMac. Atari and PARC begat ST, the music maker, who died and was no more. Z80 begat Sinclair the dwarf, TRS-80 and CP/M, who begat many machines, but soon passed from this world. Altair, Apple and Commodore together begat Microsoft, the Great Darkness which is called Abomination, Destroyer of the Earth, the Gates of Hell.
Now it came to pass in the Age of Microchips that IBM, the greatest of the Mainframe Corporations, looked upon the young Microchip Systems and was greatly vexed. And in their vexation and wrath they smote the earth and created the IBM PC. The PC was without sound and colour, crufty and bodacious in great measure, and its likeness was a tramp, yet the Customers were greatly moved and did purchase the PC in great numbers. And IBM sought about for an Operating System Provider, for in their haste they had not created one, nor had they forged a suitably grave and perilous License, saying, First we will build the market, then we will create a new System, one in our own image, and bound by our Licence. But they reasoned thus out of pride and not wisdom, not forseeing the wrath which was to come.
And IBM came unto Microsoft, who licensed unto them QDOS, the child of CP/M and 8086. (8086 was the daughter of Intel, the child of Moore). And QDOS grew, and was named MS-DOS. And MS-DOS and the PC together waxed mighty, and conquered all markets, replicating and taking possession thereof, in accordance with Moore s Law. And Intel grew terrible and devoured all her children, such that no chip could stand before her. And Microsoft grew proud and devoured IBM, and this was a great marvel in the land. All these things are written in the Books of the Deeds of Microsoft.
In the fullness of time MS-DOS begat Windows. And this is the lineage of Windows: CP/M begat QDOS. QDOS begat DOS 1.0. DOS 1.0 begat DOS 2.0 by way of Unix. DOS 2.0 begat Windows 3.11 by way of PARC and Macintosh. IBM and Microsoft begat OS/2, who begat Windows NT and Warp, the lost OS of lore. Windows 3.11 begat Windows 95 after triumphing over Macintosh in a mighty Battle of Licences. Windows NT begat NT 4.0 by way of Windows 95. NT 4.0 begat NT 5.0, the OS also called Windows 2000, The Millenium Bug, Doomsday, Armageddon, The End Of All Things.
Now it came to pass that Microsoft had waxed great and mighty among the Microchip Corporations; mighter than any of the Mainframe Corporations before it had it waxed. And Gates heart was hardened, and he swore unto his Customers and their Engineers the words of this curse:
Children of von Neumann, hear me. IBM and the Mainframe Corporations bound thy forefathers with grave and perilous Licences, such that ye cried unto the spirits of Turing and von Neumann for deliverance. Now I say unto ye: I am greater than any Corporation before me. Will I loosen your Licences? Nay, I will bind thee with Licences twice as grave and ten times more perilous than my forefathers. I will engrave my Licence on thy heart and write my Serial Number upon thy frontal lobes. I will bind thee to the Windows Platform with cunning artifices and with devious schemes. I will bind thee to the Intel Chipset with crufty code and with gnarly APIs. I will capture and enslave thee as no generation has been enslaved before. And wherefore will ye cry then unto the spirits of Turing, and von Neumann, and Moore? They cannot hear ye. I am become a greater Power than they. Ye shall cry only unto me, and shall live by my mercy and my wrath. I am the Gates of Hell; I hold the portal to MSNBC and the keys to the Blue Screen of Death. Be ye afraid; be ye greatly afraid; serve only me, and live.
And the people were cowed in terror and gave homage to Microsoft, and endured the many grave and perilous trials which the Windows platform and its greatly bodacious Licence forced upon them. And once again did they cry to Turing and von Neumann and Moore for a deliverer, but none was found equal to the task until the birth of Linux.
These are the generations of Linux:
SAGE begat ARPA, which begat TCP/IP, and Aloha, which begat Ethernet. Bell begat Multics, which begat C, which begat Unix. Unix and TCP/IP begat Internet, which begat the World Wide Web. Unix begat RMS, father of the great GNU, which begat the Libraries and Emacs, chief of the Utilities. In the days of the Web, Internet and Ethernet begat the Intranet LAN, which rose to renown among all Corporations and prepared the way for the Penguin.
And Linus and the Web begat the Kernel through Unix. The Kernel, the Libraries and the Utilities together are the Distribution, the one Penguin in many forms, forever and ever praised.
Now in those days there was in the land of Helsinki a young scholar named Linus the Torvald. Linus was a devout man, a disciple of RMS and mighty in the spirit of Turing, von Neumann and Moore. One day as he was meditating on the Architecture, Linus fell into a trance and was granted a vision. And in the vision he saw a great Penguin, serene and well-favoured, sitting upon an ice floe eating fish. And at the sight of the Penguin Linus was deeply afraid, and he cried unto the spirits of Turing, von Neumann and Moore for an interpretation of the dream.
And in the dream the spirits of Turing, von Neumann and Moore answered and spoke unto him, saying, Fear not, Linus, most beloved hacker. You are exceedingly cool and froody. The great Penguin which you see is an Operating System which you shall create and deploy unto the earth. The ice-floe is the earth and all the systems thereof, upon which the Penguin shall rest and rejoice at the completion of its task. And the fish on which the Penguin feeds are the crufty Licensed codebases which swim beneath all the earth s systems. The Penguin shall hunt and devour all that is crufty, gnarly and bodacious; all code which wriggles like spaghetti, or is infested with blighting creatures, or is bound by grave and perilous Licences shall it capture. And in capturing shall it replicate, and in replicating shall it document, and in documentation shall it bring freedom, serenity and most cool froodiness to the earth and all who code therein.
Linus rose from meditation and created a tiny Operating System Kernel as the dream had foreshewn him; in the manner of RMS, he released the Kernel unto the World Wide Web for all to take and behold. And in the fulness of Internet Time the Kernel grew and replicated, becoming most cool and exceedingly froody, until at last it was recognised as indeed a great and mighty Penguin, whose name was Tux. And the followers of Linus took refuge in the Kernel, the Libraries and the Utilities; they installed Distribution after Distribution, and made sacrifice unto the GNU and the Penguin, and gave thanks to the spirits of Turing, von Neumann and Moore, for their deliverance from the hand of Microsoft. And this was the beginning of the Fourth Age, the age of Open Source.
Now there is much more to be said about the exceeding strange and wonderful events of those days; how some Suits of Microsoft plotted war upon the Penguin, but were discovered on a Halloween Eve; how Gates fell among lawyers and was betrayed and crucified by his former friends, the apostles of Media; how the mercenary Knights of the Red Hat brought the gospel of the Penguin into the halls of the Corporations; and even of the dispute between the brethren of Gnome and KDE over a trollish Licence. But all these things are recorded elsewhere, in the Books of the Deeds of the Penguin and the Chronicles of the Fourth Age, and I suppose if they were all narrated they would fill a stack of DVDs as deep and perilous as a Usenet Newsgroup.
Now may you code in the power of the Source; may the Kernel, the Libraries and the Utilities be with you, throughout all Distributions, until the end of the Epoch. Amen.
Woodchuck and FrOSCon
At the end of August, I attended FrOSCon in Bonn again, after skipping it last year. The evening before FrOSCon however, I visited
Neal Walfield, his wife Isabel and their little son Noam in D sseldorf. Besides having a great time and a lovely dinner, I was most impressed by their collection of Maemo devices (they had at least two N770s, an N900 and, to my jealousy, an N950) which Neal is doing research on these days. He works on
woodchuck, which is a project investigating how to improve data availability on mobile devices and our conversation prompted him to implement
ATP Woodchuck, which makes smarter decisions when to run APT upgrade on your Maemo device then the standard updater. As part of the research, they also run a
user behaviour study which I joined, where one installs a client which records various data off your N900 and sends them anonymized (he seems to be doing a
good job at that) to figure out how people use their mobile devices and hopefully enhance the experience. So if you have a N900, you should consider
joining the study so they get better data.
The next day, I picked up Martin Michlmayr nearby and we headed for FrOSCon. I was quite impressed by the Makerbot at the Tarent booth, but I still don't know what they are really doing and why they had it on display... In the afternoon, I attended a couple of talks in the PostgreSQL developer room and a talk about a big OpenVPN deployment, before ending the day with the excellent as always social event barbeque. On Sunday, I went to quite a few talks, but I thought that two of them were particularly interesting:
Michael "Monty" Widenius of MySQL gave a talk titled
"Why going open source will improve your product" about starting businesses on an open source project, or how business can/should open-source their product. Besides a detailed discussion about the various forms of Open Source licenses and the Open-Core model, he proposed the idea of "Business Source" (see slide 20 of his
presentation), where a startup would distribute the source code under a non-commercial (but otherwise open-source) license with the explicit guarantee that the license would be changed to a true FLOSS license at some defined point in the future, giving the company a head start to develop and nurture their project. I asked whether this has been already implemented in practise and how the community could be sure that e.g. lawyers after a hostile takeover would not just remove that part of the copyright notice, as long as a true distribution under a FLOSS license has not happened yet. Monty wasn't aware of any real-word cases, and he did not seem to be concerned about this and said the original intent would be clear in a possible court case. This was the first time I heard about this approach, I wonder how other people think about it, whether it would work in practise and be a useful thing to have?
Second, I attended a talk by
Gregor Geiermann, a Ph.D. student in linguistics on
"Perceptions of rudeness in Free Software communities". He conducted an
online survey about the perceived rudeness of several forum thread posts on Ubuntu Forums. Survey participants were first asked a couple of generic questions about their gender, nationality etc. and were then presented with a series of posts. For each post, they were asked to rate how rude they thought it was on a scale of 1 to 5 and they also had the possibility to highlight the parts of the post they considered rude as well as add comments. He presented a neat web application for analyzing the results, which makes it possible to select different groups (he did male vs. female and Americans vs. Germans in the talk) and have their overall rudeness ratings as well as the highlighted texts visualized as different shades of blue. Comments can be easily accessed. There were quite a few interesting differences e.g. in how Germans perceived rudeness compared to Americans (RTFM comments were considered less rude by Germans for example, IIRC). In response to my question, he said he intended to release the web application as open source and this might be an interesting tool for FLOSS projects to analyze how their public communication channels are perceived by various groups. Unfortunately, I cannot find any other resources about this on the web as of today, so I should try to contact him about it at some point.
No, my blog isn't dead, and neither is me. It's just that way too many things happened since 
eysler.debian.org is now DSAed, online and building packages (including
autosigning). I will shutdown monteverdi.ayous.org which is still at my
place in the next days, reinstall the system and send it to Darmstadt as
another buildd for data center hosting there. After that happens all mipsel
buildds are DSAed as it should be, and are running in a data center and not
via some DSL line.
(In case you're looking for hardware at your place, there are a couple of
loongson 2f-systems available to buy. 2f is the successor cpu of 2e. Some
2f systems get delivered with Debian installed on it, see e.g.