Romain Francoise: Rectangles are the new squares
I'm thrilled to find out that Facebook has a group named The CTRL-X R K appreciation society; I'm member #4. C-x r k is one of my favorite Emacs commands.
Groups that campaign for free technology, like the Free Software
Foundation and the Electronic Frontier Foundation, tend to be
supported primarily by technologists. Both groups have struggled to
communicate their messages to non-geeks. I have written an article
and helped create a new weblog, both called Revealing Errors, that
attempt to address a root cause of this issue in what I hope is both an
insightful and entertainingly manner.
Geeks support groups like the FSF and EFF because, as people who
understand technology, they understand just how powerful technology is.
Geeks know that control of our communication technologies is control
over what we can say, who we can say it to, and how and when we can say
it. In an increasingly technologically mediated age, control over
technology is not only the power to control our actions; it is the power
to limit our possible actions. Our freedom to our technology is our
freedom, full stop.
This message fails to resonate with non-geeks but it does not fail
because non-geeks are happy to hand over their freedom. It fails to
resonate simply because the vast majority of people do not understand
that technology, and control over it, is powerful enough to impact their
freedom. Most people fail to see the power because, quite simply, most
people fail to see technology. While we all see the effects of
technologies, the technologies themselves are frequently hidden. We see
emails but not mail transport agents. We see text messages but not the
mobile phone network. Before one can argue that such systems must be
free, one must reveal their existence. Technologists are keenly aware
of the existence of these systems. To everyone else, they are completely
invisible.
Marc Weisner of Xerox PARC cited eyeglasses as an ideal technology
because, with spectacles, "you look at the world, not the eyeglasses."
When technology works smoothly, its nature and effects are invisible.
But technologies do not always work smoothly. A tiny fracture or a
smudge on a lens renders glasses quite visible to the wearer indeed.
Similarly, people see their MTAs when messages bounce. They see Windows
on their ATM or phone when the system crashes. Technological errors are
moments when usually invisible technology becomes visible. They are, in
this sense, also an educational opportunity.
I have recently published an article in Media/Culture Journal
from the University of Melbourne within a special issue called Error. If
you are interested in learning more about what I'm trying to do or
looking at some examples, please read the article.
With support and ontributions from Aaron Swartz, I have also created
a new weblog, Revealing Errors, that reveals errors that reveal
technology by posting descriptions of errors with commentary on what the
error reveals. I've posted a few examples there already and I will be
updating it regularly. The goal is to help explain the power and
influence of technology in the service of broadening the base of people
who can get excited about freedom to technology.
Eventually, I hope to be able to communicate this message to a less
technical audience. With that said, I hope that even seasoned
technologists will learn things about their technological environment
through the analysis and interaction. I hope readers of this blog will
subscribe to it and, if possible, comment on and contribute to the
project as it moves forward.
Top posters in a few Debian-related Planets:
Sometimes I think this should be renamed Planet Buxy.$ planet-scores.sh Planet Debian-FR : 19 Rapha l Hertzog 4 Roland Mas 3 Jean-Christophe Dubacq 2 Gr gory Colpart 2 Alexis Sukrieh
Newly added contributors to that planet have all their recent articles aggregated, not only the ones they wrote since they were added.Planet Debian-FR (utilisateurs) : 10 Julien Candelier 8 Emilien Macchi 4 Guilhem Bonnefille 3 Shams Fantar 1 Rapha l Hertzog 1 Olivier Berger (perso) 1 Jean-Christophe Dubacq 1 Jean-Baptiste H tier (djib) 1 Eric Veiras Galisson
Planet "Christian loves rugby".Planet Debian : 40 Christian Perrier 2 Russell Coker 2 Raphael Geissert 1 Wouter Verhelst 1 Steve Kemp 1 Romain Francoise 1 NOKUBI Takatsugu 1 Michal iha 1 John Goerzen 1 Joey Schulze 1 Gerfried Fuchs 1 Fathi Boudra 1 Enrico Zini 1 Emanuele Rocca 1 Dirk Eddelbuettel 1 David Welton 1 Christine Spang 1 Antti-Juhani Kaijanaho 1 Adam Rosi-Kessel
I know I have an encoding problem on some planets, but that script is a very basic curl+shell+sed+grep+recode+sort+uniq pipeline, and I only use it for the amusement value. Maybe I'll recode it with a proper RSS parser some day if I feel utterly bored.debian-community.org : 4 Holger Levsen 3 Andrew Donnellan 2 Evgeni Golov 1 Wolfgang Lonien 1 Rapha l Hertzog 1 Martin Albisetti 1 Marcos Marado 1 Jean-Christophe Dubacq 1 Cord Beermann 1 Benjamin A'Lee 1 Andreas Putzo $
I was in New York last weekend to run the 37th
New York Marathon which
reportedly currently stands as the largest marathon ever run in
terms of starters and finishers.
And it couldn't have happened on a nicer day: sunny at the start with
temperatures in the 50s, no wind, and some clouds in the second half prevented it
from getting too hot.
Large crowds at most parts of the course, a decent number of
bands, and a generally very excited atmosphere. And of course a nice course
across the five buroughs finishing in Central Park.
For once, I managed to run the race steadily and yet fairly fast, ending up
with a time of 3:18:47 (and thus a 7:35 pace). This is about a minute slower
than my PR from Sunburst
2006, and just two seconds faster than my best Chicago
Marathon result from 2006, yet much better than this year's times from Boston
2007 where it was too cold, and Chicago
2007 where it was way too hot. Given that the NY course is somewhat
hillier, and that was definitely busier and more crowded than the other races, I'm quite happy with the
time, and the way I ran, getting through without any walking breaks. Not quite negative
splits at around 1:38 and 1:40 for the two halfes. With enough energy left
at the end, I finished the remaining 2+ km after the 40 km with a sub-7:00
min/mile pace which felt great. And it is a nice feeling to have completed
the
Boston,
Chicago
and now New York marathons in the same year.
Last but not least, the weekend was a general blast as I was staying with a friend I hadn't seen
in a decade which made for lots of stories, and even more beers...
After sending 785 reports I received three notification messages about my message awaiting moderator approval.Your message to Debian-arabic-packages awaits moderator approval
Your message to Pkg-icecast-devel awaits moderator approval
Your message to Pkg-dia-team awaits moderator approval
...and, really, we nearly never thought that Argentina could win tonight.
South Africa was really too strong for them.
The only domain where Argentinians dominated were scrums but this was much
compensated by their great weakness on line-outs, losing something like 5 or
6 of them.
South Africa thus logical won very easily...and Bryan Habana is now the best
try scorer all along RWCs, along with Jonah Lomu. He is a really great
player, indeed.
Much much impressive South Africa: they'll be by far the favourites of the
final against England, which they beated already during that World Cup. I
said several times that I'm not a deep fan of RSA's play, but I could become
one. Their game improved all along the World Cup and today, they probably
played their best match.
Two remakes next week-end: France-Argentina for the 3rd place and South
Africa-England as final. That will be good matches, no doubt...and I hope
that Bernard Laporte will change the French team to play against Argentina.
After one day off, we watched Argentina-Georgia yesterday evening.
Interesting game because it directly affect France's chances to qualify
after Argentina's win on Friday.
Argentina had a big disadvantage by playing only 4 days after their historic
battle against France. As a logical consequence, they appeared quite tired
and very imprecise. On the other hand, Georgia fought very courageously with
a strong pack of forwards, encouraged by most of the crowd in Lyon. Finally,
Argentina won *and* scored the bonus point by scoring a try in the very last
minute (which might become very important to qualify in this pool. Good game
even if Argentina was not that impressive, but I still think this is mostly
because of the 4-days delay.
Today, 3 games were scheduled. We didn't watch USA-Tonga (15-25). USA was
slightly favourite for that one but they also were quite tired by playing 4
days only after their battle against England on Saturday. This was the first
match for Tonga and obviously the island players were less tired and played
a good rugby to win that one. Japan-Fidji (31-35) seems to have been the
most uncertain match as of now. I somewhat regret not watching it as these
two are among the teams I like watching and 7 tries in the match probably
made it very interesting.
Tonight, we watched Italy-Romania (24-18). The italians compete with
Scotland for the remaining qualifying spot in that pool, if one assumes that
New Zealand will obviously win all its matches. So, Italy *had* to win that
match. The quality of romanian rugby has decreased in the last 10 years,
but they still have a good set of great players, most of them playing...in
France (just like one third of the italian team, indeed). The match was
quite indecise with italy keeping the ball all first half long, but only
leading 8-0 on half time....and being surprised by two romanian tries at the
beginning of second half (8-12). After a strong domination of Italy, Romania
was logically sanctioned by one expulsion and a penalty try (for successive
voluntary faults), but, until the end, Italy was still in danger. They
finally made it, which was the most important for them, 4 days after they
were sweeped out by the All Blacks.
We're now eagerly waiting for *THE* game of this week, England-South Africa
in the Stade de France on Friday. Elizabeth found out that a few tickets are still
available, but for 253 euros each, we decided to watch it on TV. So, between
21:00 and 23:00UTC that day, don't even think about pinging me on IRC. I
still haven't decided who I'll support the most as I'm not a huge fan of
either team....but they're still among the best in the world, of course.
Today I've stumbled upon this old page about the GNU project.Linux is a copylefted kernel, which uses GNU, BSD and X software to make a compleat free OS. This or FreeBSD or NetBSD are great OS's to use until the Hurd is stable.
One of the things I have been tasked to do in my current
assignment is to create a dashboard of the status of various
software components created by different contractors (participating
companies) in the program. These software components are built by
different development groups, utilizing unlike toolsets, languages
and tools though I was able to get an agreement on the VCS
(subversion yuck). Specifically, one should be able to tell which
components pass pre-build checks, compile, can be installed, and
pass unit and functional tests. There should be nightly builds, as
well as builds whenever someone checks in code on the release
branches. And, of course, the dashboard should be HTTP accessible,
and be bright and, of course, shiny.
My requirements were that since the whole project is not Java,
there should be no dependencies on maven or ant or eclipse projects
(or make, for that matter); that it should be able to do builds on
multiple machines (license constraints restrict some software to
Solaris or Windows), not suck up too much time from my real job
(this is a service, if it is working well, you get no credit, if it
fails, you are on the hot seat). And it should be something I can
easily debug, so no esoteric languages (APL, haskell and Python
:P)
So, using continuous integration as a google search
term, I found the comparison matrix at
Damage Control
I looked at anthill, and cruisecontrol, and the major drawback
people seemed to think it had was that configuration was done by
editing an XML file, as opposed to a (by some accounts, buggy) UI
is not much of a factor for me. (See this). I also like the fact
that it seems easier to plug in other components. I am
uncomfortable with free software that has a commercial sibling;
we have been burned once by UML software with those
characteristics.
Cruisecontrol, Damagecontrol, Tinderbox1 & tinderbox2,
Continuum, and Sin match. I tried to see the demo versions; Sin s
link led me to a site selling Myrtle Beach condo s, never a good
sign. Continuum and Damagecontrol were currently down, so I could
not do an evaluation. So, here are the ones I could get to with
working demo pages: http://cclive.thoughtworks.com/ and
http://tinderbox.mozilla.org/showbuilds.cgi?tree=SeaMonkey
Cruisecontrol takes full control, checking things out of source
control; and running the tests; which implies that all the software
does build and run on the same machine this is not the case for
me. Also, CC needs to publish the results/logs in XML; which seems
to be a good fit for the java world; but might be a constraint for
my use case.
I like the tinderbox dashboard better, based on the information
presented; but that is not a major issue. It also might be better
suited for a distributed, open source development model;
cruisecontrol seems slightly more centralized more on this below.
cruisecontrol is certainly more mature; and we have some experience
with it. Tinderbox has a client/server model, and communicates via
EMAIL to a number of machines where the actual build/testing is
done. This seems good.
Then there is flamebox nice dashboard,
derivative of tinderbox2; and seems pretty simple (perhaps too
simple); and easily modifiable.
However, none of these seemed right. There was too much
of an assumption of a build and test model and few of them seemed
to be a good fit for a distributed, Grid-based software
development; so I continued looking.
.
I finally decided CABIE
Continuous Automated Build and Integration Environment. Cabie is a multi-platform, multi-cm client/server based application providing both command line and web-based access to real time build monitoring and execution information. Cabie builds jobs based upon configuration information stored in MySQL and will support virtually any build that can be called from the command line. Cabie provides a centralized collection point for all builds providing web based dynamic access, the collector is SQL based and provides information for all projects under Cabie s control. Cabie can be integrated with bug tracking systems and test systems with some effort depending on the complexity of those systems. With the idea in mind that most companies create build systems from the ground up, Cabie was designed to not have to re-write scripted builds but instead to integrate existing build scripts into a smart collector. Cabie provides rapid email notification and RSS integration to quickly handle build issues. Cabie provides the ability to run builds in parallel, series, to poll jobs or to allow the use of scripted nightly builds. Cabie is perfect for agile development in an environment that requires multiple languages and tools. Cabie supports Perforce, Subversion and CVS. The use of a backend broker allows anyone with perl skills to write support for additional CM systems.The nice people at Yo Linux have provided a Tutorial for the process. I did have to make some changes to get things working (mostly in line with the changes recommended in the tutorial, but not exactly the same. I have sent the patches upstream, but upstream is not sure how much of it they can use, since there has been major progress since the last release. The upstream is nice and responsive, and have added support in unreleased versions for using virtual machines to run the builds in (they use that to do the solaris/windows builds), improved the web interface using (shudder) PHP, and and all kinds of neat stuff.
Keeping /etc in git and other version control systems has been talked
about and attempted by lots of people. I tried it with CVS, but that was a
joke, with svn, but it sucked, now with git I actually have something I
think I'm happy enough about to use.
There's already a tool to do this, but I was not really impressed with
isisetup. After considerable pain to find the
source code (download links that resolve to 192.168. addresses are niice),
I found that it was pretty old and had lots of weird hardcoded things like
server names, and it wants to be a git porcelin with its own set of
commands for manipulating the /etc repository. I don't want that, I just
want a simple setup procedure that lets me use regular git commands to
manage the repo.
So instead I followed the lead of
these
blogs
combined with David H rdeman's metastore for solving the metadata storage
problem. I also hacked in a solution to the empty directory storage
problem, and a few other issues with trying to use git for this thing
that it was not really designed for.
I'm very happy with the way it works, you just run "etckeeper init" and
then can commit etc as usual with the file permissions etc included in the
commits. Clone the same repo to elsewhere and again "etckeeper init" is all
you need to restore those permissions (be sure to not clone the repo to a
world readable directory, as there is a window when files would be readable).
Automatically committing changes at the end of apt runs is also a handy
feature. :-)
etckeeper is currently only available from git. ftp-master is down
so I can't upload packages, but both it and metastore can be built from git
easily enough:
git clone git://git.hardeman.nu/metastore.git; cd metastore; dpkg-buildpackage
git clone git://git.kitenet.net/etckeeper; cd etckeeper; dpkg-buildpackage
If you should discover, while sitting in a airport departure lounge,
that your TomTom has lost its SD card, don't
despair. Provided you have a spare SD card (I stole the one from my
camera) and have the laptop you've used TomTom Home with to hand, all
might not be lost. I was able to do something along the lines of:
cd /media/disk cabextract ~noodles/TomTom/HOME/Downloads/navcore_7.131.8483.one.cab mkdir "US - Western Region" cd US\ -\ Western\ Region/ cabextract ~noodles/TomTom/HOME/Downloads/US_-_Western_Region.cab cd umount /media/diskwhich got me up and running again. Of course it doesn't have the UK maps that came with the device, but as I was on my way to the US that's not so important (and Katherine has found the SD card anyway, so I can just replace it when I return). I was quite pleased with myself. And then I ended up with a rental car that had GPS of its own anyway... (Oh, yes, I'm in Fremont, CA, flying back on the morning of Saturday 18th November. I am currently jetlagged and trying hard not to go to bed. If you are in the area, know me and would like to meet up for food/beer/whatever feel free to drop me an email.)
Some people say, write it right ahead or you won't write it at all. I fear the latter, thus I'm trying to catch up with things that happend in the last months. Not that I didn't mention some aspects, but I haven't written about some others and want to keep them at least for myself in good rememberance.
There were some movies I went to, and some of them I totally enjoyed. First thing was "1408"—a horror shocker with John Cusack, and I really enjoyed it. I'm not a big fan of splatter horror, and this wasn't, it's a pretty nice done psycho shocker. If you still have the chance and like the genre, go and watch it. :)
Then there was "I Now Pronounce You Chuck and Larry" which I watched even twice, with both my best friends. One would expect it being more cliche, more shallow—but it isn't. It really astonished me a fair bit how well it tackles the topic, and as an additional benefit the soundtrack of the movie is yet another must-have.
Which brings me to "Lissi und der wilde Kaiser". Don't. Just... don't. This one is cliche, this one is shallow. Some small jokes, weakly connected to some story... Bully, you did way better before.
Alright, first round of catching up, more to come. After all I'm syndicated on some Free Software related planets, and people might be interested in what's getting done in those parts, too. Not that I would do things in secret...
El general en jefe del ejercito libertador del sur Emiliano Zapata Manifiesto Zapatista en Nagua
I have often posted on the accuracy of my mail filtering
mechanisms on the mailing lists (I have not had a false positive in
years, and I stash all discards/rejects locally, and do spot checks
frequently; and I went through 6 months of exhaustive checks when I
put this system in place). False negatives are down to about 3-4 a
month (0.019%). Yes, that is right: I am claiming that my
classification correctness record is 99.92 (99.98% accuracy for
messages my classifiers are sure about). Incorrectly classified
unsure ham is about 3-4(0.019%) a month; incorrectly classified
unsure Spam is roughly the same, perhaps a little higher. Adding
these to the incorrect classification, my best estimate of not
confidently classified mail is 0.076%, based on the last 60 days of
data (which is what gets you the 99.92%).
I get unsure/retrain messages at the rate of about 20 a day
(about 3.2% of non-spam email) about 2/3 rds of which are
classified correctly; but either SA and crm114 disagree, or crm114
is unsure. So I have to look at about 20 messages a day to see if a
ham message slipped in there; and train my filters based on these;
and the process is highly automated (just uses my brain as a
classifier). The mail statistics can be seen on my mail server.
Oh, my filtering front end also switches between reject/discard
and turns grey listing on and off based on whether or not the mail
is coming from mailing lists/newsletters I have authorized;
mimedefang-filter
However, all these numbers are manually gathered, and I still
have not gotten around to automating my setup s overall accuracy,
but now I have some figures on one of the two classifies
in my system. Here is the data from CRM114. I ll update the numbers below
via cron.
First, some context: when training CRM114 using the
mailtrainer command, one can specify to leave out a
certain percentage of the training set in the learn phase, and run
a second pass over the mails so skipped to test the accuracy of the
training. The way you do this is by specifying a regular expression
to match the file names. Since my training set has message numbers,
it was simple to use the least significant two digits as a regexp;
but I did not like the idea of always leaving out the same
messages. So I now generate two sets of numbers for every training
run, and leave out messages with those two trailing digits, in
effect reserving 2% of all mails for the accuracy run.
An interesting thing to note is the assymetry in the accuracy:
CRM114 has never identified a Spam message incorrectly. This is
because the training mechanism is skewed towards letting a few spam
messages slip through, rather than let a good message slip into the
spam folder. I like that. So, here are the accuracy numbers for
CRM114; adding in Spamassassin into the mix only improves the
numbers. Also, I have always felt that a freshly learned css file
is somewhat brittle in the sense that if one trains an
unsuremessage, and then tried to TUNE (Train Until No Errors) the css file, a large number of runs through the training set are needed until the thing stabilizes. So it is as if the learning done initially was minimalistic, and adding the information for the new unsure message required all kinds of tweaking. After a while TOEing (Training on Errors) and TUNEing, this brittleness seems to get hammered out of the CSS files. I also expect to see accuracy rise as the css files get less brittle The table below starts with data from a newly minted .css file.
| Date | Corpus | Ham | Spam | Overall | Validation | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Size | Count | Correct | Accuracy | Count | Correct | Accuracy | Count | Correct | Accuracy | Regexp | |
| Wed Oct 31 10:22:23 UTC 2007 | 43319 | 492 | 482 | 97.967480 | 374 | 374 | 100.000000 | 866 | 856 | 98.845270 | [1][6][_][_] [0][3][_][_] |
| Wed Oct 31 17:32:44 UTC 2007 | 43330 | 490 | 482 | 98.367350 | 378 | 378 | 100.000000 | 868 | 860 | 99.078340 | [3][7][_][_] [2][3][_][_] |
| Thu Nov 1 03:01:35 UTC 2007 | 43334 | 491 | 483 | 98.370670 | 375 | 375 | 100.000000 | 866 | 858 | 99.076210 | [2][0][_][_] [7][9][_][_] |
| Thu Nov 1 13:47:55 UTC 2007 | 43345 | 492 | 482 | 97.967480 | 376 | 376 | 100.000000 | 868 | 858 | 98.847930 | [1][2][_][_] [0][2][_][_] |
| Sat Nov 3 18:27:00 UTC 2007 | 43390 | 490 | 480 | 97.959180 | 379 | 379 | 100.000000 | 869 | 859 | 98.849250 | [4][1][_][_] [6][4][_][_] |
| Sat Nov 3 22:38:12 UTC 2007 | 43394 | 491 | 482 | 98.167010 | 375 | 375 | 100.000000 | 866 | 857 | 98.960740 | [3][1][_][_] [7][8][_][_] |
| Sun Nov 4 05:49:45 UTC 2007 | 43400 | 490 | 483 | 98.571430 | 377 | 377 | 100.000000 | 867 | 860 | 99.192620 | [4][6][_][_] [6][8][_][_] |
| Sun Nov 4 13:35:15 UTC 2007 | 43409 | 490 | 485 | 98.979590 | 377 | 377 | 100.000000 | 867 | 862 | 99.423300 | [3][7][_][_] [7][9][_][_] |
| Sun Nov 4 19:22:02 UTC 2007 | 43421 | 490 | 486 | 99.183670 | 379 | 379 | 100.000000 | 869 | 865 | 99.539700 | [7][2][_][_] [9][4][_][_] |
| Mon Nov 5 05:47:45 UTC 2007 | 43423 | 490 | 489 | 99.795920 | 378 | 378 | 100.000000 | 868 | 867 | 99.884790 | [4][0][_][_] [8][3][_][_] |

[19:10] thanks to emacs you learned to play piano
There’s an interesting article on LWN.net about Gentoo: Who made Gentoo Linux, and when? A commit analysis
I was wondering how Debian compared, so I made some stats using the debian-devel-changes archives.
First, the number of source uploads per quarter:

Next.