We have a somewhat esoteric heating arrangement at my parent’s house. We burn bales of bean straw inside a large combustion chamber (similar to these) which is located in an out-house to the main building. A number of water-filled pipes run through the chamber which is then pumped around the house. Environmentally, it’s really good. Most of the straw would have been burnt anyway (the rest cut into the soil) and the combustion is fan-assisted to minimise incomplete combustion. Very little ash is produced from burning such a bale. Anyway, the delivery and storage of approximately 700 straw bales has become a yearly tradition in our house. As a guide, the trailer in the picture below is holding 160. It’s pretty tough going: even individually these things weigh a considerable amount, and they require removing from the trailer and then stacking in a 3 metre pile inside another shed! You quickly adopt the rules of moving a bale as little as possible vertically (carry them on your back if you’re young enough!), and never, ever, moving a bale twice. There’s also a large amount of dust, and a geek’s delicate palms quickly tire of the string holding the bales together. Anyway, this happened today on a couple of hours notice. The other two males in the house decided to be elsewhere *cough*, leaving it up to me and my poor mum. If anyone was expecting email/code/chat/info today.. sorry.

Procmailrc, TRAP and LASTFOLDER I use procmail and lbdb. lbdb is invoked at the beginning of procmailrc harvesting From: addresses from all incoming mail, so that they can later be queried with mutt. It's really neat. However, if it's put at the beginning of procmailrc, it harvests From addresses from spam, too. This causes the ldbd databse to become very big, slow to query, and in some unlucky cases, to return lots of bad results. Problem is, I run the spam filter AFTER I delivered to the mailing lists. The rationale is that mailing lists already do their own spam filtering to some useable level. At least those I'm subscribed to. So, my current procmailrc goes like:

feed a copy through lbdb
filter list 1
filter list 2
...
filter list N for N very big
filter out spam
deliver to inbox

Now, is there a way I could invoke lbdb for every mail except what is caught by the spam filter? A way, I mean, that doesn't involve hooking into each and every of the N list rules an lbdb invocation? Here is the way. In .procmailrc, add:

TRAP=~/bin/scan-nonspam

Then write ~/bin/scan-nonspam:

#!/bin/bash
if [ $ LASTFOLDER:0:4  != 'spam' ]
then
    /usr/bin/lbdb-fetchaddr
fi
exit 0

For reference, this is the documentation of TRAP:

When procmail terminates of its own accord and not because it received a signal, it will execute the contents of this variable. A copy of the mail can be read from stdin. Any output produced by this command will be appended to $LOGFILE. Possible uses for TRAP are: removal of temporary files, logging customised abstracts, etc. See also EXITCODE and LOGABSTRACT.

And this is the documentation of LASTFOLDER:

This variable is assigned to by procmail whenever it is delivering to a folder or program. It always contains the name of the last file (or program) procmail delivered to. If the last delivery was to several directory folders together then $LASTFOLDER will contain the hardlinked filenames as a space separated list.

Recently, Josh Berkus blogged about The Myth of the Benevolent Dictator. In it he complained about people who try to posit that a benevolent dictator was "an unalloyed asset" to their project, and that there were many successful and valid forms project governance other than the singleton project leader. Furthermore, he argues that the best way is to form a community, and then ends by suggesting that people who say "dictatorship good" is really saying "democracy bad", and perhaps the motives of the person using that argument should be questioned.

Given that I was one of the first people (and probably the first person) to use that phrase to describe Linus's governance model, obviously I feet the need to respond to his thoughts on the matter.

First of all, the term "benevolent dictator" comes from an old saying, "the best form of government is the benevolent dictator --- there's only one problem, finding the benevolent dictator"[1]. And in my presentations where I explained the Linux project management model, I was always careful to say that there were other models that were used by other projects, and that it was interesting to study why certain management models worked for some projects but not others.

So by definition, if a project is successful with a single project leader ultimately responsible for making decisions to settle conflicts, then that leader is a "benevolent dictator"; if they aren't successful, then that would be an example of a dictator who wasn't able to be "benevolent", eh? :-)

In all seriousness, though, I agree with Josh when he says that the key part of an open source project being successful is gathering together a healthy community. But let's unpack that a little. In order for a community to be healthy, there needs to be general consensus on the values and cultural norms for that community. So for example, in the case of Debian, which Josh holds up as a successful community, not everyone has the ability to vote; you have to pass a fairly stringent process which tests your technical skills and your willingness to subscribe to Debian's core values as embedded to the Debian Social Contract. People who do not pass the New Membership process do not have the right to vote in Debian, even if they are the most rabid, fanatical users of Debian.

This brings up a key point about governance analogies; they all don't map cleanly into the real world for two closely related and extremely important reasons: (1) No one has a fundamental human right to be a member in some OSS project, and (2) No one is forced to stay in a particular OSS project. These are really two sides of the same coin, and what it means is that, as in Debian, "democracy" does not mean that everyone is entitled to political franchise, or the right to vote, and in a project which has a "benevolent dictator", the powers of the dictator is fundamentally limited by the simple fact that if he is too unreasonable, people are always free to leave or fork the project. These differences are extremely important when trying to reason about "democracy" or "dictatorship" as being good or bad as applied to open source projects.

Democracy is rarely the best way for an open source project to make decisions. For technical issues, you want the right answer, and majority vote isn't necessarily way to resolve a technical issue. As far as policy decisions, if the decision can not be made by consensus, then some percentage of the community will be unhappy with how the decision, and that can cause a huge amount of arguments and dissension. If the community has very clearly expressed values, then the number of times where such votes need to be made will be minimal.

For example, Debian has an explicit declaration of its values and mission in its Debian Social Contract; people who don't agree to the Social Contract aren't allowed to become Debian Developers and thus are not allowed to vote. But, one of the more controversial disputes that have threatened to tear apart Debian is the question of the non-free packages, and that's fundamentally because of an ambiguity in the Social Contract. On the one hand, Debian has declared that it is fundamentally 100% about Free Software, to the extent that it has adopted Richard Stallman's rather petulant request that the Linux operating system be called GNU/Linux (which admittedly is better than Lignux.) On the other hand, the Social Contract also explicitly specifies the existent of the non-free section in the distribution. Over time, this ambiguity have been the source of a lot of highly passionate debate and controversial votes that have satisfied neither the FSF fanatics nor the more Linux-like moderate position tolerating the use of proprietary as well as open source software. I submit that if Debian had either explicitly stated an FSF-centric position, and only accepted members that supported point of view, or explicitly subscribed to a position that users should be able to use whatever software they feel meets their needs, as Ubuntu has, it would avoided many arguments that have threatened to tear apart Debian.

More recently the fact that the Social Contract has been silent about whether or not Debian Developers should be paid or purely be 100% volunteer. This was never an issue in the Linux Kernel community, but for some reason it has been a highly divisive and disruptive issue inside Debian. Democratic votes have not really resolved the conflicts, and in fact some might claim that these votes have if anything hardened various Debian Developers' positions.

Going back to Josh's final point, "Next time you hear the words 'benevolent dictator' you probably want to think about the motives of the person saying it. Because the corollary to 'dictator good' is 'democracy bad', and is that a place you really want to go?", I am quite proud to stand by my convictions that for Open Source Projects, pure democracy is a bad idea. If a projects wants to be technically successful, it needs to be a meritocracy, and chose the best technical decisions; Linus does not decide by abiding with majority vote whether or not a patch should be accepted, and that's as it should be.

Even Debian's "raw democracy" doesn't let any random luser get to vote on who should be the Debian Project Leader. Decisions need to be made by people who are competent to make a valid choice. Even in real life, one could argue that the choice made by Americans, over of which 70% believed that Saddam Hussein was "very likely" or "somewhat likely" to be personally involved with the 9/11 attacks even though there has been absolutely no evidence to support this "gut feeling", made a rather disastrous choice in voting to re-elect George Bush to the presidency. Democracy without sufficient education, literacy, and intellectual curiosity to learn about what is happening in the world can result in truly disastrous choices being made. So certainly, democracy is not a panacea. In the real world, it may be the best choice of many bad choices, given how much people have suffered under military dictatorships. However, in the Open Source world, things are different because people are always free to leave one project and affiliate with another --- not an alternative most people living in dictatorships have historically had. Given these key differences, I think it is clear that for open source projects, democracy is indeed often a bad idea, and that if you if a project has a single person with a strong technical vision and good interpersonal skills, the benevolent dictator model can indeed be a very viable and successful.

Does that mean that other forms of governance can't work? Of course not! The rotating dictatorship used by Perl[2], the core-team-with-strongarm leader used by OpenBSD, the corporate dominated scheme of MySQL, the self-selecting core team model of Samba, all are successful models. I just have to strongly disagree with Josh's claim the single strong leader approach is somehow inferior to "democracy" as if democracy was somehow a magic bullet.


[1] For am early example of this quote, nine years ago, please see this message, where I'm pretty sure I authored the colon prefixed quote, and I know I was using the term "benevolent dictator" much earlier than 1998.

[2] In the early days of Perl, it also used the benevolent dictator model. Larry Wall is a great exemplar of how someone who has a strong vision of what he wanted to accomplish, and was humble and easy to work with, both on-line and in person, can make all the difference in the world for an up-and-coming open source project.

Good comrades, great chat, greater motivation.
Some good Scotch whisky, a good friend who was lumped along with me in .br by mere chance. A very good time.
A strong whisky aftertaste and numbness in the mouth. My first couple of pages of Terry Pratchet - Thanks a bunch!
My mind wanders, I go over all the nice and dear things and people
Maybe tomorrow I should restart my reading? I know Pratchett is not supposed to make much (rational) sense, but still... Although I've enjoyed this couple of pages, I'd rather let the whisky wear out before continuing with the gollem owner's life and tribulations, with his friendly treatment for his plausible but honorable assassin...
Today I saw quantum superpositions mixed with special relativity in a small (less than 200 lines) Perl module. Positronic variables go back in time, yay! Damian Conway rules.
Time to sleep, yes, if tomorrow you want to be in time for the lightning talks. Oh, BTW: great honor, great honor: One of the lightning talkers requested /me to sit on the front row "to have a friendly face to look at". Have to comply, have to do.
But... Last reflection before going away: What a better way to describe bliss in Vienna (oder auf Deutstch, ich glaube - gl cklich in sterreich;) than to go to sleep while listening to the neighbour's fucking-loud music? Of course, his fucking-loud-music is just the-Balkan-music-you-seem-to-love, so everything's berfine!
Thanks to everybody who made this possible. Specially to the Austrian friends.

A new version of PerlConsole is available. As you can see in the changelog, lots of goodness with this release comes to you (~/.perlconsolerc rules, thanks to S. Zachariah Sprackett for his patch). After reading the source code of Devel::REPL and learning from it (this is really a great library, congrats to mst) I decided not to orphan Perl Console. Yes, that was a possibility as Devel::REPL provides anything you need to build a console (and comes with “re.pl” which actually is a console itself). I decided not to give it up for one reason: Devel::REPL is great, and better written than Perl Console (it supports plugins and profiles and is Moose-based) but, it’s slow. When you launch it, you have to wait a couple of seconds before the console prompts you. This is because of Moose I suppose. I know that’s not blocker as a console is a development tool, but I feel like some of PerlConsole’s users would like to have the choice between a full featured, extensible and Moose-base console that’s slow, and a light one that is basic in design and implementation. I backported one of the most interesting plugin of Devel::REPL to PerlConsole, being the console’s lexical environement. Devel::REPL did a great thing there, using Lexical::Persistence to simulate a namespace in the console. Feel free to give your feedback about this, I wonder if there’s still an audience for PerlConsole when Devel::REPL is in the town. I think so, maybe I’m wrong, then tell me.

A few days ago, Joe Reagle was telling me about the rise of (sometimes very arcane) policies and bureaucratic imperatives that now characterize Wikipedia. A few days later I stumbled across a few interesting posts on the topic, posts (especially Nick Carr’s) seeping with almost celebratory gloom and doom:

‘But, given human nature, is it really so “incredible” that Wikipedia has evolved as it has? Although writers like Yochai Benkler have presented Wikipedia as an example of how widescale, volunteer-based “social production” on the Internet can exist outside hierarchical management structures, the reality is very different. As Wikipedia has grown, it has developed a bureaucracy that is remarkable not only for the intricacies of its hierarchy but for the breadth and complexity of its rules. The reason Deletionism has triumphed so decisively over Inclusionism is pretty simple: It’s because Deletionism provides a path toward ever more elaborate schemes of rule-making - with no end - and that’s the path that people prefer, at least when they become members of a large group. The development of Wikipedia’s organization provides a benign case study in the political malignancy of crowds.”

It is without question that a problem has arisen in Wikipedia, a problem composed of a thick web and net of rules that can be helpful as guidelines but often are often confusing and clearly work to ensnarl new users. But can we so quickly put blame on so-called human nature? Or is it not a problem of human organization, which as the anthropological and sociological record shows, can take many forms? And is it not just that: a problem begging for a solution instead of an opportunity to declare the fundamental nature of wikipedia (and that of human nature?). Not only may history prove him wrong, other large scale collaborative projects at least prove that solutions can be found to deal with problems of growth and scale. It is as if Carr just wants to see a project like Wikipedia fail, which somehow, this morning hit a raw nerve. What I find exciting about large scale projects of (at times unwieldy) collaboration are not just the explicit outputs of the projects (an encyclopedia or operating system) but the social worlds they create. And there is no inevitable path they *must* follow. These groups have a choice to react to and respond to these sort of problems and enact solutions that will hopefully solve them and allow these projects to change. Debian, a slightly older project than Wikipedia, has gone through many growing pains and there was even a period when the the whole process of integrating new Debian developers was shut down and if my memory serves me correctly, for 2 years! At the time, it could have been possible to say: “This signals the end of Debian” but eventually a solution was found, the New Maintainer process, which while not perfect (what is?) allowed the project to grow and produce a great operating system for years to come. Debian today faces new problems and is working to find solutions. I hope that Wikipedia can and will do the same. And instead of declaring its death, why not wait and see, and offer something a little more constructive and illuminating, than destructive?

I was recently talking to a client about the lack of guidelines for acceptable personal use of office resources in his company. He rejected the suggestion that he provide any real rules or guidelines (apart from some old rules that most employees were not aware of and of which there was no procedure to remind them to periodically read). He said that he only wanted to hire motivated people who wanted to work, and that people who need rules should work elsewhere. The results of his approach seem reasonable, everyone who works for him works really hard. In return his attitudes towards employees are more relaxed than most employers, among other things he is very supportive of employees who desire career opportunities greater than his company can offer. I have been wondering whether rules alone can make people lose interest in working, or whether it’s the type of person who doesn’t want to impose rules that inspires hard work. My experience is that when working for people who have a rule-free environment I tend tend to work really hard, and that when working for companies that have lots of rules I find it difficult enough to get out of bed in the morning - let alone become motivated to do any work. I am interested to see comments from other people, both workers and managers. Is my experience common in this regard?

Share This

Random notes on effective bugs cleanup on a bigger scale:

Ignore the subject. In 90% of the cases it is badly formulated and provides no clue.

Be on a good mood or don't even start.

Use different overviews (bts, mbox, web, reportbug-ng, ..) to not miss the easy stuff right on start.

Read the last message first. A please close message on the tail or an upstream nack keeps you from reading the previous ones. Obviously as curious person i often violate that rule.

Have prepared standard answers on FAQ: Debian kernel patch acceptance guidelines, report issue to Linux upstream, debug initramfs.

Be bold: A big wontfix list won't help anyone. Close right away.

In the ping mail directly write To the submitter. A Cc to the bug report makes it more personal.

Tag any bug that is against an old version with moreinfo in the ping mail. Gives the submitter a clear indication that this bug needs an update.

Sometimes a close acts better as ping, reopen quickly and use the info.

My itinerant brother, presently in Tucson, Arizona is trying to visit my less-itinerant brother in Tahoe, California. (The latter is an editor and photographer for the Tahoe Daily Tribune, now available online in full print-like layout). Later this week, the itinerant brother flies from Tucson to Sacramento, the least expensive airport closest to Tahoe. Amtrak has a route from Sacramento to Tahoe (actually nearby Truckee), but it is actually a bus. Having made his plane reservations, my brother waited until today to order his Amtrak tickets online, only to discover the following: Yes, it’s true, you can’t actually board that bus (the only way to get to Tahoe after my brother’s plane arrives), unless you’ve at least set foot on a train first. Some research suggests that California State Law prohibits Amtrak from offering bus-only services within the state, apparently so it won’t compete with bus companies. I did figure out what I hope is a clever workaround: the next closest train stop on one of the lines to Sacramento is Davis. Amtrak allows you to buy a ticket from Davis to Truckee via Sacramento for basically the same price as the bus from Truckee to Sacramento. Hopefully no one will check to make sure he actually traveled from Davis to Sacramento before getting on the bus to Truckee. There must be some profound lesson here. I would love to have been a fly on the wall during the floor debate in the state legislature when this law was enacted.

Share This

(I had to omit some of the smaller changes - it should all be in the ChangeLog but ask if you spot something major that I haven't mentioned here.)

IMPORTANT! dpkg-cross_1.99+2.0.0pre1 will BREAK things!
(hence it is going into experimental)

1. BREAKAGES!
   The changes in dpkg-cross are so extensive that apt-cross and emdebian-tools are broken. Current SVN for emdebian-tools builds 0.3.9 which works with pre1. There is a branch of apt-cross (apt-nc-cross) that builds 0.2.9 which works with dpkg-cross pre1. HOWEVER apt-cross 0.2.9 has VERY limited functionality (even --get is incomplete) so dpkg-cross pre1 should only be installed in a chroot or on a system that does not need apt-cross or emdebian-tools. There's little point adding a NEWS.Debian comment on this because it would be too late but Conflicts: are in place:
   Conflicts: apt-cross ( 0.2.9), emdebian-tools ( 0.3.9)
   (The watchful will note that the previous Conflict on pkg-config has been dropped because the conflicting version is no longer in stable or oldstable.)
   The dpkg-cross pre-releases will all go into experimental and no upload will be made to unstable until apt-cross 0.2.9 is available as a package in Emdebian and is also ready to go into Debian as v0.3.0. emdebian-tools needs less changes, some of which can be done AFTER dpkg-cross 2.0.0 reaches unstable.
   I anticipate at least three pre-releases, maybe more.
   


2. REMOVED PERL FILE:
   /usr/share/perl5/dpkg-cross.pl
   Replaced by:
   use Debian::DpkgCross;
   There are some functions not universally exported but which can be brought in manually:
   gccross : use Debian::DpkgCross qw(simplify_path);
   dpkg-shlibdeps : use Debian::DpkgCross qw(dpkgcross_application);
   dpkg-buildpackage : use Debian::DpkgCross qw(setup_cross_env);
   The universally exported functions are:
   read_config setup get_architecture create_tmpdir convert_path get_keepdeps get_removedeps get_config get_version detect_arch convert_ld_library_path get_tool rewrite_pkg_name dump_debug_data check_arch
   


3. Perl changes:
   use strict;
   use warnings;
   All scripts and the new module are now fully strict and -w compliant. Some perl modules have also been added to the dependency list.
   


4. MAKEFLAGS are no longer exported!
   This is a big change and could affect a lot of packages but the honest
   reason is this:
   We should not be exporting MAKEFLAGS ourselves. (in the context of debian-embedded) Packages that were buggy with MAKEFILES are now non-buggy and packages that relied on MAKEFLAGS are now buggy.
   CDBS packages have a single line fix in debian/rules:
   DEB_CONFIGURE_SCRIPT_ENV =
   I'll be filing a bug in due course.
   This allows packages like libx11-6 to build internal tools using CC_FOR_BUILD (set to gcc) and run them to make certain generated source files during the cross-build. I expect it to also work for fontconfig. This method is far preferable to either creating patches for the generated content or trying to replicate the generation process in an interpreted language.
   


5. New function:
   &read_config();
   &setup();
   
   
   $config = &get_config;
   get_config is new because the configuration is now an internal variable within the module and is not in scope for the scripts. It is a hash table:
   $config 'crossbase' = $crossbase;
$config 'crossprefix' = $crossprefix;
$config 'crossdir' = $crossdir;
$config 'crossbin' = $crossbin;
$config 'crosslib' = $crosslib;
$config 'crosslib64' = $crosslib64;
$config 'crossinc' = $crossinc;
$config 'crossroot' = $crossroot;

   The values from the two config files:
   $conffile = "/etc/dpkg-cross/cross-compile";
my $home = File::HomeDir->my_home;
$private_conffile = "$ home /.dpkg-cross/cross-compile";

   are read in using Config::Auto which removes all the syntax problems of the previous bespoke parser. Comments are ignored as before.
   Everything you specify in either of the config files is now available within the module but only "recognised" options are exported. Config file values are merged - data from /etc/ is read first and data from /home/ is overlaid so that any data not in /home/ comes from /etc/ and any data from /etc/ is overwritten if it also exists in /home/
   @removedeps and @keepdeps need to be sorted out still.
   


6. REMOVED CONFFILES:
   /etc/dpkg-cross/cross-config.common obsolete
/etc/dpkg-cross/cross-config.gnu obsolete
/etc/dpkg-cross/cross-config.linux obsolete
/etc/dpkg-cross/cross-config.w32 obsolete
/etc/dpkg-cross/crosstools obsolete



7. TESTING
   Ah, yes, testing. So far, only two packages have been tested. I plan to test more tomorrow. Please check for regressions and other brokenness and report via the BTS. (I may forget if it's only done via the debian-embedded list.) Please tag all pre1 issues 'experimental'. Ta.
   


8. Speaking of bugs - 6 closed so far, a few more possible. I would like
   to close all but one of the current dpkg-cross bugs. (I'm not sure how
   to fix #285569 )
   


9. Debian::DpkgCross.3 manpage - new but horribly incomplete.
   


10. %pkgvars - currently disabled. I will merge scope: into the existing %config which then leaves no place for %pkgvars or crosstools but this is pending for pre2.
    


11. debug output.
    You now get two debug files in /tmp/:
    gccross2.log : Details the environment, %config and check process of the last call (the last failed call if the build breaks).
    dpkg-cross2.log : Actually misnamed, this is a dpkg-buildpackage log file that contains %config, PATH changes, $ENV changes and the contents of the gccross temporary directory for the last build, including variables set and modified via the module. I'll change the name in pre2.
    The debug output is meant for debian-embedded, not other users. To this end, I'm experimenting with debian/buscript which could go into /usr/share/bugs/ to prompt users reporting bugs for the content of these logs and other configuration details, exactly as with apt. If this works, I'll ditch the idea of turning --debug off and leave it enabled for all builds.
    debian/bugscript is not enabled in pre1. There again, because the module is a new package, pre1 will spend some time in the NEW queue. All changes for pre1 are now in CVS so it should be easy for everyone to build their own packages until pre1 clears NEW and enters experimental. Just be careful where you install dpkg-cross_1.99+2.0.0pre1_all.deb !!!
    (And remember: install libdebian-dpkgcross-perl_1.99+2.0.0pre1_all.deb)
    

I just had a hard time finding this excellent procmail resource on the web. I am thus blogging it for posterity, in case anyone is looking for procmail documentation, tips, tricks, a how-to, or anything else related to procmail. And if you procmail and have not read the document, I suggest you do. It's truly outstanding. NP: A Silver Mt. Zion: He Has Left us Alone, but Shafts of Light Sometimes Grace the Corner of our Rooms

Training the Discriminators It has been a while since I posted on this category actually, it has been a long while since my last blog. When I last left you, I had mail (mbox format) folders called ham and/or junk, which were ready to be used for training either CRM114 or Spamassassin or both. Setting up Spamassassin This post lays the groundwork for the training, and details how things are set up. The first part is setting up Spamassassin. One of the things that bothered me about the default settings for Spamassassin was how swiftly Bayes information was expired; indeed, it seems really eager to dumb the Bayes information (don t they trust their engine?). I have spent some effort building a large corpus, and keeping ti clean, but Spamassassin would discard most of the information from the DB after training over my corpus, and the decrease in accuracy was palpable. To prevent this information from leeching away, I firstly increased the size of the database, and turned off automatic expiration, by putting the following lines into ~/.spamassassin/user_prefs:

bayes_expiry_max_db_size  4000000
bayes_auto_expire         0

I also have regularly updated spam rules from the spamassassin rules emporium to improve the efficiency of the rules; my current user_prefs is available as an example. Initial training I keep my Spam/Ham corpus under the directory /backup/classify/Done, in the subdirectories Ham and Spam. At the time of writing, I have approximately 20,000 mails in each of these subdirectories, for a total of 41,000+ emails. I have created a couple of scripts to train the discriminators from scratch using the extant Spam corpus; and these scripts are also used for re-learning, for instance, when I moved from a 32-bit machine to a 64-bit one, or when I change CRM114 discrimators. I generally run them from ~/.spamassassin/ and ~/var/lib/crm114 (which contains my CRM114 setup) directories. I have found that training Spamassassin works best if you alternate Spam and Ham message chunks; and this Spamassassin learning script delivers chunks of 50 messages for learning. With CRM114, I have discovered that it is not a good idea to stop learning based on the number of times the corpus has been gone over; since stopping before all messages i the Corpus are correctly handled is also disastrous. So I set the repeat count to a ridiculously high number, and tell mailtrainer to continue training until a streak larger than the sum of Spam and Ham messages has occurred. This CRM114 trainer script does the hob nicely; running it under screen is highly recommend. Routine updates Coming back to where we left off, we had mail (mbox format) folders called ham and/or junk sitting in the local mail delivery directory, which were ready to be used for training either CRM114 or Spamassassin or both. There are two scripts that help me automate the training. The first script, called mail-process, does most of the heavy listing. This processes a bunch of mail folders, which are supposed to contain mail which is either all ham or all spam, indicated by the command line arguments. We go looking though every mail, and any mail where either the CRM114 or the Spamassassin judgement was not what we expected, we strip out mail gathering headers, and then we save the mail, one to a file, and we train the approprite filter. This ensures that we only train on error, and it does not matter if we accidentally try to train on correctly classified mail, since that would be a no-op (apart from increasing the size of the corpus). The second script, called mproc is a convenience front-end; it just calls mail-process with the proper command line arguments, and feeds them the ham and junk in sequence; and takes no arguments. So, after human classification, just calling mproc does the classification. This pretty much finishes the series of posts I had in mind about spam filtering, I hope it has been useful.

Growing up in Eugene, Oregon, which like Berkeley or Boulder could have the label "People's Republic of" applied to it, I always thought of "business friendly", as something along the lines of helping huge corporations avoid laws against pollution, or other antisocial behavior. Only after moving to Europe did I begin to get an idea of what the very positive side of "business friendly" is in the US. Truth be told, all countries tend to protect their 'big players' to some degree, be it the US propping up creaking airlines after September 11th, Italy finding various clever ways to get around rules about funding Fiat and Alitalia, or France finding it in their 'national interests' to discourage a potential bid for Danone (yogurt!) by PepsiCo. Some are better or worse (the UK has been pretty good about not interfering), but there is a tendency to want to intervene. Leaving be the discussion over whether those sorts of policies are good, bad, or ugly, the biggest difference between continental Europe and the US is the ease with which new companies - 'startups' can be created and enter a market. As a first hand example, I decided this summer to create DedaSys as a real company in order to better separate my business and personal financial dealings. Were I to do that in Italy or Austria, we would be talking about fees upwards of 3000 Euros (about 4000 dollars at market rates), which is a great deal of money for something that is not making a lot of it at this point in time. Contrast this with what it took to create a Limited Liability Company (LLC) in Oregon, my home state. To have things done up professionally, it's certainly possible to lay down a bit of cash there too, but by trading my time for money, and with the assistance of some nolo.com books about the creation and maintenance of an LLC, I was able to register DedaSys with the state of Oregon for the grand total of 55 dollars, and was actually able to complete the process remotely from Austria prior to going home on vacation, where I did the only thing I needed to do in person: open a bank account. So it costs 1% of what it costs in Italy or Austria to open a company that provides limited liability... a very impressive difference, especially to a small, new company that does not have the connections a Ford or a Fiat will likely have to enable it to deal with all the other paperwork, rules, and regulations to deal with. Add to that a culture of greater risk taking (meaning also more acceptance of failure), better funding opportunities, and... one comes to the conclusion that Paul Graham is right. It's a pity, because the people in Europe are top notch. In Italy alone, I know a bunch of really bright hackers. Granted, some of them aren't interested, and are probably better off not starting a company or otherwise dealing with the business side of the equation, but it's always nice to have that opportunity. In closing, here is another example of bureaucracy in action, from my personal on line journal about life in Italy, and now Austria, which I recently revamped by moving it to the Typo platform: Confronting the bureaucratic beast - registering an Italian domain Two months to accomplish what can be done in ten minutes with a .com!

Random testing of my local Debian mirror shows that 644 binary packages out of 20774 (3.1%) are missing the DEBIAN/md5sums control file. This file (used by debsums) is very useful to check for disk corruption, and even though debsums can generate the data on the fly when a package is installed, it's better to have this information computed at build time.

So if your packages use debhelper, make sure that you have the proper dh_md5sums call in debian/rules. If you don't use debhelper, you'll have to generate the DEBIAN/md5sums control file manually (see dh_md5sums's source for inspiration). If you use a high-level build tool like CDBS, you probably don't have to do anything.

Should I file bugs against the 446 affected source packages? A few maintainers apparently exclude some binary packages on purpose; for example the zsh source package generates md5sums for zsh, zsh-static and zsh-doc, but not for zsh-dev and zsh-dbg...

Update: updated statistics with a run on a full amd64+all mirror.

Short update:

• I had no time to do hacking lately
• there are too many mails on -boot; I wish they changed their minds and made a -boot-dev list, the BTS mails are incredibly annoying (yes, I made a filter, but is not that efficient); I am thinking seriousely about unsubscribing from that list
  
• I would like to retreat from supporting the ppp-udeb package of ppp since I have no more interest in it; I already unsubcribed from part of the mails related to it
• I have a new HDD from my router and is really silent :-) (some good news, at last)
• glest-data contains some unclearly licensed stuff; imagine how nice is that when upstream is unresponsive (although not dead - adds new messages on the upstream forum, but never answers other's posts)
  
• sorry for not having time to work on svn-buildpackage, glest, naughtysvn, wormux and oolite; I would really like NMU work on the debian packages (but I don't know what to say about svn-buildpackage since Eduard Bloch is the lead developer)
• real life is taking is share back
• I am thinking of upgrading to lenny in an attempt to motivate myself back into coding
• aspell-ro with support for the correct diacritics should be uploaded soon in sid; cedilla support is dropped; minor obstructions in the way of the uplaod
  
• LVM rules
• RPM still sucks big time not because of itself, but because of the pure lack of tools for it
• scons can't cross build
• the freetype cross building bug (reported and found in gentoo) has a working patch; they claim they sent the patch
• I got 2 cool presents for my birthday (27 now): an ASUS wl-500g Premium wireless router and a Fujitsu USB HDD
  
• chroot-ing in armel from arm isn't possible (for me?):

ritter:/home/eddy# chroot /data/chroots/armel-root-fs/
Illegal instruction
ritter:/home/eddy# uname -a
Linux ritter 2.6.18-4-ixp4xx #1 Sun Aug 5 19:53:40 EEST 2007 armv5tel GNU/Linux

• I no longer have this on my belly (it washed away):
  

• the vacation was really great
• I have too many interests in and outside of Debian, I'll have to cut back on some of them
• Ross Burton rules because he wrote tasks and he maintains it properly
  

Firefox is constantly gaining marked share (especially here in Europe) and it’s little not-yet-so-popular brother Thunderbird is evolving too. While I’m happy that they’re contributing their part to bring Free Software to the masses, I’m concerned that the quality of their software (especially Firefox under Linux) has decreased in the last months. Here are my top 5 annoyances for Firefox and Thunderbird. Some of them only apply to Debian’s version others are more visible on my fairly old and “slow” (1.5GHz/512MiB) Laptop than on current and therefor faster machines: Firefox

• Firefox seems to have a massive problem with font kerning. Sometimes text is hardly readable since several letters seem to share the same one-letter space. And marking messed up text like this gives funny results:
  
• When visiting pages with a fixed background image like this, scrolling the page down causes massive CPU load and very slow scrolling even on fairly fast computers. Konqueror seems to handle pages like this much better.
• When reloading all tabs at once in a session and you have many (read: more than say, 6) tabs open, Firefox takes all the available CPU resources and pretty much becomes unresponsive until all tabs are loaded.
• Firefox literary freezes while loading very large pages (combine this with the previous annoyance for funny results)
• I still haven’t figured out why Firefox on my laptop (correctly) wants to launch kpdf when I open a .pdf while Firefox on my desktop seems to prefer xpdf. I’ve checked all the available options on both systems and wasn’t able to find the difference in the configuration.

Thunderbird

• The most annoying bug in Thunderbird is Debian specific: it doesn’t open links in a browser anymore for months. Since you have to change some non-obvious configuration to fix this, I assume this is the default behavior for many users out there which are now forced to copy-paste links to their browser.
• Like most other mail clients Thunderbird is also able to manage contacts. But it lacks two important features:
  1. No im/export of vcard: Hmm? Even outlook can do that and vcard is pretty much standard for storing contact information platform independent.
  2. No birthdays associated to contacts: Superb! When combining the Mozilla calendar application with Thunderbird you cannot just import all the birthdays from your contacts (which are stored as vcards of course), no you have to separately create those events which then of course are totally unrelated to the persons in your roster.
• The “search entire message” feature in the search box does absolutely nothing when searching in newsgroups. Why is it enabled then? Or: why does it pretend to do something by hiding all threads when entering something, suggesting no match was found?
• Wasn’t Thunderbird 1.x able to work with RSS feeds? What happened to this feature in Thunderbird 2.x?
• What happened to Thunderbird’s icon?

While I’m still happy with Thunderbird (or Iceddove as we call it here) I’m really searching for an alternative to Firefox (Iceweasel). Unfortunately there is currently no alternative available which fits all my needs. Firefox has two extensions no other browser seems to provide and without which I wouldn’t want to work anymore (I wonder if that was a proper English sentence…):

• Foxmarks Bookmark Synchronizer: which allows me to store my bookmarks on my (that’s right, not just foxmark’s) server to keep it in sync with all my Firefoxes on various places and machines.
• Adblock + Filterset.G Updater: which keeps me mostly ad-free and automatically supplies me with the newest filter set rules.

While other browsers have some ad-blocking functionality too, they all seem to lack the filter set updater. The next best alternatives are Konqueror and Epiphany but besides the lack of the above features every one of them has it’s own little quirks which doesn’t really motivate me to make the switch yet. But Konqueror is making progress and I’m really looking forward to KDE4 which will hopefully bring a usable web browser so I can finally get rid of Firefox. Update: Ooops, I nearly forgot my favorite pet annoyance of Firefox under Linux: Why does it have to insult the user’s eyes with this ugly toolkit when drawing check boxes, line edits, buttons etc? Firefox under Windows paints stuff like that according to Windows’ current theme, Konqueror under Linux as well, but what does Firefox for years?

Yes, it seems that SCO doesn't own the Unix copyrights. This means that SCO has lost the legal war against Linux that they started back in 2003, arguing that as Linux was a Unix derivative, it was using UNIX System V code in it. At the time that meant tons of articles and information to be written about this issue, as well as next year, in 2004, in which some lawsuits were filled. They even announced a lawsuit agains IBM for $1 billion. Years later, when a court rules that SCO does not own any rights over the IP, thus saying that it belongs to Novell, I have only seen this appearing in Slahsdot and some Slahsdot and some Slahsdot and some Slahsdot and some other blogs but without all the hype that should follow such a desition. For sure people reaction is quite logical, as they yet knew that this was going to happen. They knew that since the very moment that SCO was asked to point which pieces of code were under violation of their rights and they refused with vague anwers.i

Once again, someone suggests banning feeds with low debian-specific content from Planet Debian. (lists.d.o) This time, it's led by Steve McIntyre, whose blog comments about his work, friends, family trips and pub challenges, among other things. This purge saw Evan Prodromou unlink his journal (which comments about his work, friends, family trips and wikipeda, among other things) in reaction to hate, commenting:

"I've heard from people who really like reading my blog posts on Planet Debian, and people who really hate them. The people who like my blog can add it to their feed readers directly. I believe I've followed the rules for PD, but there's no point antagonizing people who don't want to read it"

I'm sad to see one of the more interesting "off-topic" authors disappear from the site like this. Hatred wins again? Updated: Evan's back. Yippee! (2007-08-09: 1 comment)

1. We always underestimate the number of stupid people


2. The probability of a person being stupid is independent of any other characteristic of that person


3. A stupid person is someone who causes damage to another person, or a group of people, without any advantage accruing to himself (or herself) or even with some resultant self-damage


4. Non-stupid people always underestimate the damaging power of stupid people. They constantly forget that at any moment, and in any circumstance, associating with stupid people invariably constitutes an expensive mistake.


5. A stupid person is the most dangerous person in existence


6. Stupid people don t know they are stupid

The above has been floating around the net for a while. While it makes an interesting read I think that having a boolean criteria of labelling someone as stupid is not necessarily accurate. While there are a small number of people who are stupid, there are many more people who are good at some things and stupid when they act outside their area of expertise. There is a saying “never take investment advice from your dentist” which is apparently due to the incidence of con-artists targetting wealthy people (such as dentists) who sometimes then pass on the bad advice to their customers. One stupid thing that I did a few years ago was to spend more effort in choosing a mobile phone than on choosing a car. I was totally happy with my mobile phone but not totally happy with my car - and the car cost a lot more… Of course the difference between pure stupidity and tactical stupidity (for want of a better term) is that a smart person who is about to do something stupid can generally be persuaded not to do it with a logical argument. If someone had pointed out to me the fact that the amount of time spent on background research for a decision should be in some way proportional to the amount of money involved (maybe proportional to the log of the value) then I would have been convinced. So for rule 4 maybe it would be best to say taking advice from someone in an area where they are stupid is a mistake. For rules 5 and 6, if someone is known to be smart in other areas then tactical stupidity may be overlooked (until it’s too late). Update: Don Marti commented with this link http://www.math.wisc.edu/~miller/old/incomp.txt about incompetent people being unable to judge their own level of incompetence as further evidence for point 6.

Share This