Just as I was blogging about my trip to Prozac Land and how I got my health back, after working for a week and a half I started feeling sick again. Very sick.

I was having these ups and downs, one day I felt I could fly and the next I felt I was down in a hole. I was throwing up every morning and I had stopped eating anything but fruit already quite some time ago. I was smoking like a truck driver (Spanish saying) and drinking coffee like a mad hatter would drink tea in every Unbirthday Party.





My psychiatrist then withdrew all my medication at once, which was sweet and sour. This crap fucks up your brain chemistry, and on one hand, you feel like yourself again, but on the other hand, the down is scary. I have been dizzy, clumsy and sleepy at all times since I am off the happiness pill.

My endless doctor trips have not finished yet, but I made an unexpected stop in ER last Thursday. I was on the verge of a nervous breakdown, delirious that I am going to loose my job because of my terrible health and, subsequently, lack of accountability, stressed as hell about it, and physically feeling like crap, I landed in ER Land, and stayed there for 6 hours. I felt frivolous taking this picture, my arm feeling numb and foreign from the Peripheral IV line, which is what I was first given upon arrival, it felt too tight, slightly painful, and I was terrified to move or bend my arm. It just lay there, beside me, as if it were no longer part of me, out of my control.

All kind of tests were performed on me, blood was taken to feed the lab vampires, I don't even want to think about the urine, x-rays documented my piercings locations, physical exploration, endless questions, talking to two different, unfriendly, tired and overworked, doctors, and nothing was found, all looked normal. All but me, I was despaired. If all is fine, and I am not dying from bladder cancer, why do I feel like this? I felt stupid, I was looked at as I was there wasting their scarce time. Nobody could tell me what is wrong with me.

It looks like Prozac played an important role in killing my health completely. My psychiatrist has been on the phone with me all weekend, calling me every couple of hours. Why is he so worried about me? Why doesn't he want me to sleep alone at my place and wants me to sleep at my parents? Is it because of people committing suicide closely related to Prozac? Why, if he is aware of this, which I do not know, did he give it to me? I am not suicidal, dear doctor. I am just feeling very sleepy, very clumsy, I can barely manage to press the right keys in Mutt, my email client of choice since 2000, which I know how to use by heart, my fingers just do their job, but not anymore. But how can I explain that to you?

My friends, again, are here, close to me, keeping an eye on me, calling me on the phone, feeding me, letting me take a nap in their couch.

Dear doctor, you have mentioned the latest groovy drug to replace Prozac with, I am sorry, it is not going to happen. I am going to deal with whatever is wrong with my brain, mind and body in a natural way, with therapy, and healthy living habits. I trust you and respect you, but I do not trust these drug makers, and they are already making enough money. If it is going to be painful, I want to feel it. Please do not numb me and then make feel it all at once like this. Let me deal with it the way I choose to.

I choose the hard way. Can we agree that I am strong enough?

Uphold, maintain, sustain: life in the trenches Now that I have a baseline filter, how do I continue to train it, without putting too much of an effort? There are two separate activities here, firstly selecting the mails to be used in training, and secondly, automating the training and saving to the mail corpus. On going training is essential; Spam mutates, and even ham changes over time, and well trained filters drift. However, if training disrupts normal work-flow, it won't happen; so a minimally intrusive set of tools is critical Selecting mail to train filters There are three broad categories of mails that fit the criteria: 1. Misclassified mail This is where human judgement comes in, to separate the wheat from the chaff.

• misclassified Spam. I do nothing special for this category -- I assume that I would notice these mails, and when I do, I just save them in a junk folder, for latter processing. The volume of such messages has fallen to about one or two a month, and having them slip though is not a major problem in the first place
• misclassified ham is far more critical, and, unfortunately, somewhat harder to get right, since you do want to reject the worst of the Spam at the SMTP level. A mistake here is worse than a false negative: all that happens with a false negative is that you curse, save to the junk folder for later retraining, and mode on. With missed Ham, you never know what you might have missed -- and hope it is nothing important.
  • If one of the two filters did the right thing, then the mlg script can catch it -- more on it below. The only thing to remember to do is to look carefully at the grey.mbox folder that is produced. While this is not good to have misclassified ham, at least this sub-category is easy to detect.
  • If both filters misclassified it, this would then mean a human would have to catch the error. The good news is that I haven't had very many mails fall into this category (the last one I know about was in the late summer of 2005). How can one be sure that there are not ham messages falling through the cracks all the time? The idea is to accept mail with scores one would normally discard, and treat this as a canary in a mine: no ham should ever show up in these realspam messages. My schema for handling Spam is shown in the figure below.
  I try and keep all the mail that I have rejected in quarantine for about a month or so, and so can retrieve a mail if informed about a mistaken rejection. I also do spot checks once in a while, though as time has gone on with no known false positives, the frequency of my checks has dropped.

2. Partially misclassified mail This is mail correctly classified overall, but misclassified by either crm114, or spamassassin, but not both. This is an early warning sign, and is more common than mail that is misclassified, since usually the filter that is wrong is wrong weakly. But this is the point where training should occur, so that the filter does not drift to the point that mails are misclassified. Again, the mlg script catches this. 3. Mail that crm114 is unsure about This is Mail correctly classified, but something that mailreaver is unsure about -- and this category is why mailreaver learns faster than mailtrainer. Spam handling and disposition . At this point I should say something about how I generally handle mails scored as Spam by the filters. As you can see, the mail handling is simple; depending on the combined score given to the mail by the filters. The handling rules are:

• score <= 5.0: Accept unconditionally
• 5.0 < score <= 15: Grey list
• 15 < score: reject

So, any mail with score less than 15 is accepted, potentially after grey-listing. The disposition is done according to the following set of rules:

• score <= 0: Classify into folder based on origin
• 0 < score <= 10: file into Spam (some of this survived grey-listing)
• 10 < score: file into realspam (Must have survived grey-listing)

In the last 18+months, I have not seen a Ham mail in my realspam folder; chances of Ham being rejected are pretty low. My Spam folder gets a ham message every few months, but these are usually spamassassin misclassifying things; and mlg detects those. I have not seen one of these in the last 6 months. So my realspam canary has done wonders for my peace of mind. With ideally trained filters, spam and realspam folders would be empty. mail list grey I have created a script called mlg ("Mail List Grey") that I run periodically over my mail folder, that picks out mails that either (a) are classified differently by spamassassin and crm114, or (b) are marked as unsure by mailreaver. The script takes these mails and saves them into a grey.mbox folder. I tend to run them over Spam and non-Spam folders in different runs, so that the grey.mbox folder can be renamed to either ham or junk, in the vast majority of the cases. Only for misclassified mails do I have to individually pick the misplaced email and classify it separately from the rest of the emails in that batch. At this point, I should have mail mbox folders called ham and/or junk, which are now ready to be used for training either crm114 or spamassassin or both. Processing these folders is the subject of the next article in this series.

The other week, I ordered a Microsoft Optical Desktop Elite for Bluetooth keyboard from Amazon. It was actually sold by ComputerGeeks.com (I didn't realise until then that Amazon was acting as a front-man for other retailers for some items). I'd been eagerly awaiting its arrival, and thought that the FedEx package that arrived yesterday might have been it, but that turned out to be the iPod I'd ordered. So today I rummaged through my email and found the email that the Geeks.com people had sent me with a username and password for their website, so I could track the order, and lo and behold, according to the link to FedEx, it had been delivered on Wednesday and left on the front porch! I just popped down to the leasing office to check if it hadn't actually ended up there (they have a very nice package receiving service) but it hadn't. I'm totally miffed that they just left it on the front porch, when the tracking information shows they tried to deliver it at 7:58 PM the day before, and determined "customer not available or business closed" (the leasing office closes at 6:00 PM) and so retried the next day and went for leaving it on the front porch at 5:00 PM, when they could have left it at the leasing office. Anger. It sounds like this happens a lot, because Geeks.com's customer service page even has an FAQ item for it. You'd think that if it happens enough to warrant a FAQ item, and it reads like they have a good policy of reshipping items that go AWOL like this, that they'd just save themselves the time and money and use the shipping service that requires a signature. Pity it's a long weekend and Geeks.com don't operate (their call centre) on the weekends. Patience is a virtue.

The New York Times reports that the Pentagon has gone on the offensive against the pro bono attorneys representing Guantánamo detainees. Although there is a lot to criticize in U.S. detainee policy, these attacks represent a new low:

The senior Pentagon official in charge of military detainees suspected of terrorism said in an interview this week that he was dismayed that lawyers at many of the nation’s top firms were representing prisoners at Guantánamo Bay, Cuba, and that the firms’ corporate clients should consider ending their business ties. … The same point appeared Friday on the editorial page of The Wall Street Journal, where Robert L. Pollock, a member of the newspaper’s editorial board, cited the list of law firms and quoted an unnamed “senior U.S. official” as saying, “Corporate C.E.O.’s seeing this should ask firms to choose between lucrative retainers and representing terrorists.” In his radio interview, Mr. Stimson said: “I think the news story that you’re really going to start seeing in the next couple of weeks is this: As a result of a FOIA request through a major news organization, somebody asked, ‘Who are the lawyers around this country representing detainees down there?’ and you know what, it’s shocking.” The F.O.I.A. reference was to a Freedom of Information Act request submitted by Monica Crowley, a conservative syndicated talk show host, asking for the names of all the lawyers and law firms representing Guantánamo detainees in federal court cases. Mr. Stimson, who is himself a lawyer, then went on to name more than a dozen of the firms listed on the 14-page report provided to Ms. Crowley, describing them as “the major law firms in this country.” He said, “I think, quite honestly, when corporate C.E.O.’s see that those firms are representing the very terrorists who hit their bottom line back in 2001, those C.E.O.’s are going to make those law firms choose between representing terrorists or representing reputable firms, and I think that is going to have major play in the next few weeks. And we want to watch that play out.”

Perhaps the worst part is the following statement, which as best I can tell has no basis in truth:

When asked in the radio interview who was paying for the legal representation, Mr. Stimson replied: “It’s not clear, is it? Some will maintain that they are doing it out of the goodness of their heart, that they’re doing it pro bono, and I suspect they are; others are receiving moneys from who knows where, and I’d be curious to have them explain that.”

Deval Patrick, the recently-elected governor of Massachusetts, endured similar attacks for his work representing criminal defendants. Fortunately, that strategy backfired and Patrick was elected in a landslide. Although others in the administration have attempted to distance themselves from Stimson’s comments (including the Attorney General), the suggestion that large corporate clients should boycott these attorneys as punishment for representing detainees remains disgusting. The right to counsel is not a political issue. You could hold the unlikely belief that each and every Guantánamo detainee is a confirmed terrorist and still advocate for fair treatment. These attacks represent a new brand of McCarthyism. I hope that this strategy is as unsuccessful nationally as it was in Massachusetts. It strikes me that attacking the partners of some of the largest firms in the country is also bad political strategy. These attorneys are the type of people who may be major campaign donors and well-connected in both parties. I would think twice before attempting to vilify them. They are unlikely to drop their representation (and might be ethically barred from doing so) in light of these statements. If anything, I suspect their determination to ensure the detainees are accorded due process will intensify. Update: abovethelaw.com speculates that this may be an intentional good cop/bad cop strategy.

Gunnar doesn't like the idea of recommending XML for configuration files, why do you need to be able to edit a XML-file with a non-XML-aware editor if you don't like the raw syntax? If you don't like the raw syntax, use an editor that gives you a different representation. Or use some transformation. Write a tool that converts YAML to XML and back, if you like YAML better. (Btw, this is another reason to use a common library for configuration file handling - let people choose their configuration file formats!) Writing XML in the raw with a good schema-aware editor with syntax highlighting is actually quite nice. Have you ever edited an XML file with eclipse? You really should do that... I once opened my Openbox (a rather minimalistic window manager) configuration file in eclipse. Guess what, it was giving me useful syntax completion! It had loaded and used the referenced schema file. It's not as if I think XML is the ultimate thing; (nor is Eclipse an editor I'd use for configuration files; startup takes years and it frequently crashes for me. vim also has some XML support...) IMHO there is a lot in XML that should be stripped (such as attributes); I like JSON syntax better, except it's in turn lacking essential information such as character encoding, namespaces and schema information. I also don't thin JSON allows comments. But when handling information from multiple sources (and multiple schemes), XML is really useful. It removes most of the quessing needed for handling other formats. And that is what I'm precisely advocating: use standardized formats. Consider for example the apache configuration. Do you know of any tool that can parse the apache configuration files other than apache? Some parts look like SGML/XML, but they don't have much more in common than using < and >. When you are in need of automating something with apache, you'll be annoyed by this. If apache would be using something where you have a reliable parser ready for - that would be nice. Have a look at the xchat.conf configuration file. It uses "key = value", but they have these extra spaces there and don't use quoting, this means the file can't be loaded by many parsers, e.g. bash. Now lets use at buttons.conf - compeltely different syntax, "KEY value" blocks, separated by empty lines... Btw, note that configuration handling with XML to me means also keeping comments somehow... most applications will nuke any comments in their configuration files; which is funny since most configuration syntaxes do have a notion of comments, but did you ever come across an application using sh-style configuration (i.e. that you could source in bash/dash/zsh), that keeps comments? P.S. The YAML homepage is not YAML. It's valid XHTML. Only if you strip out all the tags and attributes and use only the text content within the /html/body/pre tag, then you have something which probably is YAML. This compatibility with HTML is probably why XML was at all successful.

I wrote SPI gives away opensource.org and opensource.net and Bruce Byfield wrote SPI to transfer domain names to OSI - I think it's interesting to see how differently we view the disagreement. One possible reason for the difference is that the email discussion around this resolution happened on private lists. I saw the December meeting notice a couple of days before the meeting, sent the board a request that they fix member communications and then started a discussion about giving away the opensource domains with:

"SPI still holds the opensource domains because SPI is a membership controlled organisation and OSI was a self-perpetuating board; and OSI had not done enough to balance their bad licence proliferation work. Why isn't that mentioned in the proposal? Why does 2006-11-18.dbg.mjs.1 not mention 2005-07-26.bp.1? Is it good to keep reposting rejected proposals until they pass? Why is the SPI board being asked again to surrender the domains while OSI is unreformed?"

The reason I started the discussion on spi-private was Josh Berkus writing

"If you think it's important that we should drag it onto spi-private, though, I suppose we should."

which rather suggested a private list was the appropriate venue. As Neil wrote:

"Some discussion has occured on the spi-private mailing list as to how the opensource.org resolution has been handled by the board."

That's putting it mildly. I wrote about the lack of public discussion (as mentioned in my previous report):

"I still don't see how some board members reconcile their votes to give away community assets with their promises. How the devil does board expect members to oversee it when board hides its operations from our view?"

There is a report on the discussion at the next meeting, which will hopefully put the reasons on the public record at last. Unsurprisingly, the first January 2007 meeting agenda omitted my member communications request, but board member Jimmy Kaplowitz accepted my reminder.

5 days ago I started my journey to Australia on monday afternoon by going to Hamburg airport. The security at London Heathrow was disgusting and silly as expected, but I won't go into much more detail here, just one thing: the memory of losing my beloved Free Software Foundation pullover last summer at a security checkpoint at LHR made me kept checking my stuff all the time, which somehow fitted very well in the paranoia... But I should also mention the one plus thing in Terminal 4: a nice smoking lounge with power sockets and comfy chairs. Anyway...

The flight entertainment system in the 747 was very cool, with lots of movies and documentaries on demand, so that I didnt bother to read any of the books I packed nor used my laptop batteries completly, though the latter was mostly due to the lack of space in the plane and having worked quite a bit prior to leaving. So I watched four movies to not fall asleep in the (european) night and get into australian timezone, which worked more or less quite well. Not sleeping is a good way to switch timezones, but the side effect is sleepiness, doh!

Out of those four movies the only one really worth mentioning is "Little miss sunshine", a really great movie, very funny and quite insightful as well. See it! On my way back I might even choose to see it again :) I didn't really understand much of the english in "the black dahlia" but it was quite interesting to see how smoking once was ubiquitous as the war on terror is today :)

The flight itself was not as bad and boring as expected, they even thought about the poor smokers and made the whole plane get out in Bankgok, only to enter it again 30min later. Needless to say that it took those 400 people a bit longer...

BTW, and in case the famous lazyweb is reading: Kolkata (formerly called Calcutta) from 10km above really looked like it was swamped, but according to wikipedia this only happens in summer. Any ideas what I saw / why it looks liked this?

When I arrived in Sydney, I was not only welcomed by the sun and summer (yaaaaaay!!), but also by Sylvia, the (main) FOMS organizer, who was so very kind to come to the airport at 7.30 AM to pick me up (yaaay!). More about FOMS in another blog entry very soon to come.

Today, Terah:

• Wrote about multimeters


• Linked to a website selling "the only combination impact and torque tool designed to meet your wire termination requirements"


• Linked to a site named vanguardpipe.com


• Mentioned that her favorite version-control system is CVS

And all of that in a single blog post (which is also very amusing). Oh, and on Tuesday, she wrote about debconf.

I think this is the surest evidence yet that both Terah and I are geeks. She claims that only one of us is. What does everyone else think?

I should also add that she has strong operating system preferences (OS X or Linux), likes OpenOffice, uses Firefox and IMP, used Pine as her first email system when she was a student at Purdue, remembers using telnet, uses ssh to this day, and is more bothered than I am when the Internet link goes down.

That's right, more bothered than a Debian hacker.

She's talking about forming a "wives of geeks" BoF at debconf. I think just a simple "geeks" BoF would be fine.

I wonder if we should do some RDF representation of packages - especially including dependency data. That way we can maybe use some RDF reasoners to query our data, and maybe extract some interesting information. On the other hand, people with interest in RDF to use our real-world data for their experiments, and maybe we get something back from them. There is a couple of package metadata we currently are not tracking inside the actual archive, but in different places. Including licensing information (debian/copyright), homepage location (on packages.qa.debian.org), download location (debian/watch) - it would be nice to aggregate these into some RDF store, and export them somehow. For most of the package information (especially dependency information), we'll have to write our own ontology (I wonder if we can map version numbers to some standard rule language, or if applications will need an external reasoner to process them?); for some things we can reuse the FOAF (Friend-of-a-friend) or DOAP (Description of a project) ontologies. The first is rather common for describing people, people-people and people-thing relationships; the latter was designed for describing opensource projects (but won't be directly applicable to packages of a project). I've blogged about my RDF export of Debtags data before; the canonical first step would actually have been to export the package data, and enrich it with the Debtags collected data... Note that RDF is designed in a way that you can have one site provide metadata for another site. For example, the Debtags RDF export contains "category" information for Debian packages, but does not contain e.g. the description of the packages it talks about via an URI. So there is nothing wrong from a RDF point of view of keeping e.g. the licensing, watch or homepage data separate. For the Google Summer of Code, there was a proposal including "collaborative repository of meta-informations about source packages (CRMI)"; but the first part of the proposal, the "distribution wide tracker tool (DWTT)" showed to be a bigger task than expected. But maybe we'll still see CRMI at some point, and maybe we can have it provide an RDF export of data (using a semantic wiki might be a good starting point for CRMI maybe?). [P.S. this blog posting maybe belongs more into the en/linux/debian category. But only the xml category is also syndicated on planet.XMLhack, and I want this post to go there to reach more RDF users. I really need to switch my blog to some software which supports tagging...]

Ruby's thread more "functional" than OCaml's I'm relaxing some of mine initial bad impressions on Ruby. This is mainly due to the fact that the Ruby book I ordered on Amazon finally arrived (after a lost shipment, the first of my whole customer story with Amazon, happily they refunded me completely). In the book I managed to find a rigorous (though not formal as I would have liked) description of Ruby's semantics and even some snippets of the grammar of the language (Rant: how the heck is possible that a modern language like Ruby lacks a complete description of its grammar????). Anyhow, now I'm feeling more comfortable with the language and I'm starting to put in the right setting some of what I believed to be its rough edges. Today, playing with Ruby's threads, I discovered a function in Ruby's Thread API that I can't help noticing is "more functional" than corresponding features in the Thread API of my beloved OCaml. Using Thread#value indeed the value returned by the last statement executed in a given thread can be retrieved (note that in Ruby every statement returns a value: yet another functional pearl). It's like a Thread#join on steroids. In OCaml the only way (that comes into my mind) to do the same is establishing a (typed) channel among the thread creator and the created thread, sending a return value from the thread, and having the creator block on the matching receive. Or (please don't) ... you can use a global variable ... Neat Ruby threads. To balance this, according to my first read of Ruby's threading capability, it's my impression that not only at most one thread can execute Ruby code at a time (limitation shared by OCaml, due to the non-distributed nature of mark and sweep garbage collectors), but also a thread blocked on a syscall will block all other threads to run. Dumb Ruby threads (but I still hope I'm wrong ...)

Aigars Mahinovs mentioned that it would be nice if applications would not clutter the home directory with their config files as much. I agree a lot. Dotfiles for configuration suck badly. If you start a new application (or if you are in an early stage and have a small userbase), please follow the Freedesktop.org XDG base directory specification. I.e. use .config/applicationname instead of .applicationname for configuration files (unless an overriding environment variable is set), and .local/share/applicationname for data files. And for cache files, use .cache/applicationname (anyone writing a patch for firefox?) What I'd also like to propose is to reduce the number of different configuration file syntaxes. It sucks having to find out which syntax an application uses again and again, because every application is different. We probably won't be able to convince everybody on one format, but we could maybe cut it down to, like 3 formats, please? There is no reason that one application needs to start comments with "#" and another with ";" while a third application uses "//" but doesn't allow "/* comment */"... A small set of syntaxes (preferrably with standardized parsers) should be good enough for just about every application:

• key=value pairs, quotations optional, comments match "^\s*#"
• ini-style configuration files with limited nesting
• JSON syntax (nesting, all strings quoted)
• XML files (arbitrary nesting, mixed contents, repeated elements)

If we had just these few syntaxes and an appropriate schema description language, we could write generic configuration file editors. That would be very nice. Also we avoid having config file parser bugs by using libraries, and if someone prefers using e.g. gconf or dconf, this could be handled by the library, transparently for the application. (Note that many application lack error handling when writing their configuration files, especially when the disc is full... - they really should be using a library. IIRC xchat also has some bugs of this kind, I remember having tried to patch some of them once when we were preparing for the sarge release, just to find out that there might be many more of them...)

... and some brainstorming to think/rethink/remember ideas discussed in Extremadura, and now there is another proposal for tdeb implemenation on the wiki.

Comments, ideas are welcome.

Now I am off to bed.

According to this blog, I am one of the top 10 living girl geeks! How flattering. In other news, Robert Anton Wilson has died. He's one of my favorite authors, and I found this news to be rather tragic and upsetting, but it seems he remained in good spirits until the end. May he rest in peace.

First to answer why we need TDebs:

localepurge: Disk space freed in /usr/share/locale: 256828K
localepurge: Disk space freed in /usr/share/man: 4496K

Total disk space freed by localepurge: 261324K

Second, I will write my tdebs implementation proposal after posting this (although Aigars made a really good sythesis of the discutions we had in Extremadura and there is almost no room for improvement :-). I have posponed long enough.

From The New York Times:

CARACAS, Venezuela, Jan. 10 President Hugo Ch vez was sworn in to a new six-year term at a ceremony here on Wednesday in which he described Jesus as “the greatest socialist in history” and pledged to speed Venezuela s metamorphosis into a Socialist country. “Fatherland! Socialism or death, I swear it!” Mr. Ch vez yelled as he was sworn in and given the presidential sash and a golden key to the tomb where the remains of Sim n Bol var, the South American liberator, are interred.

/me cheers

Yesterday I went to activate phone service out at the farm. It got me to thinking a bit about how things change, and how they stay the same, too.

Before I go on, I'll have to say that every one of my history books is in storage, so if I get some details wrong, it's because I'm not remembering correctly.

Anyhow, phone service came to our community via an unusual route about 100 years ago. It wasn't Bell/AT&T or some other company that brought it there, as it was most places. I'm sure they figured that a small, scattered rural community would cost too much to support. So the community organized, built, and supported the phone system themselves.

Even today, roads around here can be impassible after a good rain. I'm sure that, in the early 1900s, before heavy road-maintaining machinery, things were worse -- and, of course, transportation was a lot slower then anyway. There were real problems: getting the word out about funerals, being able to summon a doctor when necessary, or letting people know that church was cancelled because of too much snow.

People in the community saw a phone system as a real need. So did the churches, which have left a legacy that is still reflected in phone company territories today.

Once phone service arrived, it was used for all the things that people expected, of course. But it also proved to be an important part of the social fabric of the community. Since party lines were the norm, it was possible to announce things to every listening subscriber pretty quickly. Older people remember announcements of fresh fruit arriving at the grocery store, funerals, or other news of the day.

To place a call, you would pick up your phone and turn your crank. That caused a bell to ring at the telephone office, which everyone called "Central." The operator would connect to your line and ask whom you wanted to talk to. The operator would then send the distinctive ring for your party down their party line, and patch -- manually -- your call through to them. And, if he was busy, the operator wouldn't listen in on your conversation -- but others on the party line very well might.

Central's hours were published. If you were making a call in the middle of the night, you were going to wake up someone at Central to do it -- plus everyone on the entire party line. So calls after hours were rare.

Fortunately, while some of the old Central operators were still around, some people in the community wrote down some of their stories.

There were some people in the community that were notorious for eavesdropping on other people's conversations. Two brothers one time figured that they knew somebody was listening to their conversations, so they devised a code. One called the other, and said, "I'll be going to McPherson in the morning for band practice." That meant something along the lines of going to town to buy groceries.

A few days later, their prime suspect came up to him and said, "What on earth are you going to band practice for? I didn't know you knew how to play an instrument!" Apparently she realized she was had when he burst out laughing uncontrollably.

The Central operators learned to know the habits of telephone users. Sometimes they would connect calls without even bothering to ask who people wanted to talk to -- and seemed to always get it right.

The phone system supported itself for about 50 years. But as the rest of the world moved on provide direct dialing, this proved a controversial subject in the community. People liked having their operators. The people that worked at Central were everybody's friend. They were people that were there, 24 hours a day, to assist with any emergency. They would gather volunteer firefighters to help fight a fire, or be able to spread community news quickly. This wouldn't be available with the newer phone systems. How would the community be informed of events quickly now? Who would just happen to know whose house the doctor was at when he was urgently needed?

The change was resisted for some years, but eventually the finances of the telephone cooperative turned out to be in deep trouble. Operators grew to be much more expensive than automation, and in the late 1960s, the telephone cooperative was no more -- sold to a phone company from a small town more than twice our size, and a for-profit company at that! Central no longer existed. I remember reading about this event -- it seems people were sad about that for quite some time. They felt that they had really lost an important part of the community when Central went away. Some machine locked in a cabinet doesn't care for people the way Central did. Even today, the older people in the community sound a little sad when they remember telephone modernization, and get the wistful look of somebody that has just remembered something that they miss.

The phone company that bought the system wasn't an AT&T, though. It was a small, independent phone company. To this day, that phone company serves only the two communities. And it was this company that I called yesterday to establish service out at our house.

They had already upgraded our lines -- over a mile of new copper, benefiting only us, at no charge to us -- last fall. The box was already on the outside of the house. Just need to get it activated.

So I called the phone company. They said I needed to drop by their office and sign some papers. Uh-oh, I think -- this is a bad sign. Sounds like a bunch of phone company bureaucracy.

But not so much. I went to the office and signed up. They asked the usual questions: name, address. Plus a few that bigger companies wouldn't ask: who used to have service at that address? Of course, most people would know that answer in our community. I couldn't have told you in Wichita, Dallas, or Indianapolis.

Then they asked when I'd like service to be activated. "As soon as possible," I say, figuring that this would be a couple of weeks like it is with AT&T or Sprint. "Well, we probably can't get out there for a couple of hours. Would it be OK if it's on at about 3?" Yes, that would be fine!

Now, how about DSL? "Well, we're a little backed up on that right now." Uh-oh. Sprint took several weeks when they *weren't* more backed up than usual. "So it'll probably be Monday or Tuesday before we can get out there. Should I just have the installer call you and arrange a time when it gets closer?" Yes, that would be fine, too!

Now, how about finding a phone number.

Out comes a large paper book. Yep, paper. They paged through it, and told me that my grandpa's old number would be available if I wanted it. I said yes -- after all, we've got his old address, so might as well keep the same phone number. OK, no problem. She whips out some white-out, whites out grandpa's name, and writes ours in. Done.

Now, do we want any optional services? Caller ID, call waiting, voicemail? How much is caller ID, I ask. $5 a month. We'll try it for now. "OK". A box was checked on the form and that was that. No high-pressure sales pitch on taking "the works" for some poorly-disclosed price, providing a ton of services I'll never use and don't want. No confusing "discounts" for having The Works and DSL at the same time.

Then I ask about an unlisted number, or at least an unlisted address. We figure that anybody that we really want to be able to reach us will figure out how without using a phone book, and these things get in so many databases these days. Sprint charged almost $10/mo for a fully unlisted number, but only a few dollars a month to just keep our address off the directories.

Our new company charged 50 cents a month for a fully unlisted number. Done.

Now it's time to pay for the first month's fees and the setup. Oops, I've forgotten my checkbook in the car. No problem, the secretary says, I'll watch your baby while you go get it! Jacob was with me, but had fallen asleep, so I brought him inside in his car seat. I went to get the checkbook -- just out the door and close by. I was back in a few seconds later, and the secretary was already on the other side of her desk talking and playing with Jacob. "My baby's 12 now," she said, and for a second, looked like a person that was remembering Central.

Or, Cleanliness is next to godliness The last time when I blogged about Spam fighting Mail Filtering With CRM114 Part 1, I left y'all with visions of non-converging learning, various ingenious ways of working around a unclean corpus, and perhaps a sinking feeling that this whole thing was more fragile than it ought to be. During this eriod of flailing around, trying to get mailtrainer to learn the full corpus correctly, I upgraded to an unpackaged version of crm114. Thanks to the excellent packaging effort by the maintianers, this was dead easy: get the debian sources using apt-get source crm114, download tghe new tarball from crm114 upstream, cp the debian dir over, and just edit the changelog file to reflect the new version. I am currently running my own, statically linked 20061103-Blame Dalkey.src-1. Cleaning the corpus made a major difference to the quality of discrimination. As mentioned earlier, I examined every mail that was initially incorrectly classified during learning. Now, there are two ways this can happen: That the piece of mail was correctly placed in the corpus, but had a feature that was different from those learned before; or that it was wrongly classified by me. When I started the chances were almost equally likely; I have now hopefully eliminated most of the misclassifications. When mailtrainer goes into cycles, retraining on a couple of emails round after round, you almost certainly are trying to train in conflicting ways. Cyclic retraining is almost always a human's error in classification. Some of the errors discovered were not just misclassifications: some where things that were inappropriate mail, but not Spam; for instance there was the whole conversation where someone one subscribed debian-devel to another mailing list, there was the challenge, the subscription notice, the un-subscription, challenge, and notice -- all of which were inappropriate, and interrupted the flow, and contributed to the noise -- but were not really Spam. I had, in a fit of pique, labelled them as Spam; but they were really like any other mailing list subscription conversations, which I certainly want to see for my subscriptions. crm114 did register the conversations as Spam and non-Spam, as requested, but that increased the similarity between Spam and non-Spam features -- and probably decreased the accuracy. I've since decided to train only on Spam, not on inappropriate mails; and let Gnus keep inappropriate mails from my eyes. I've also waffled over time about whether or not to treat newsletters from Dr. Dobbs Journal or USENIX as Spam or not -- now my rule of thumb is that since I signed up for them at some point, they are not Spam -- though I don't feel guilty about letting mailagent squirrel them away mostly out of sight. A few tips about using mailtrainer:

• Make sure you have a clean corpus
• Try and make it so that your have roughly equal numbers of Ham and Spam
• Don't let mailtrainer quit after a set number of rounds. I use a ridiculous repeat count of about 100 -- never expecting to reach anywhere close to that. Instead, I set the streak to a number = number of Spam + number of Ham + 10. This means that mailtrainer does not quit until it has processed every email in the corpus correctly without needing to retrain. Letting mailtrainer quit after it repeated the corpus twice, but before it got a streak of correct classifications left me with a filter with horrible accuracy.
• Make sure you have a clean corpus (yes, really)
• While the ratio of similarities between .css files to the differences between .css files is a fun metric, and I use it as an informal, rough benchmark while training, it is not really correlated to accuracy, so don't get hung up on it, like I was. Usually, the value I get when training from scratch on my corpus is somewhere around 7.5; but over time the ratio degrades (falling to about 5), while the filter accuracy keeps increasing.
• Train only on errors. The good news is that both mailtrainer and mailreaver already do so, so this is easy rule to follow. The reason is that you should only be doing minimal training, in case you have to modify/change the underlying rule in the future. So your filter should be trained to correctly classify the mails you get, but don't shoot for overwhelming scores.
• Use mailreaver. It has the nice behaviour of asking for retraining when it is unsure of some mail, and it caches mail for some time which really helps in training and rebuilding .css files. The reason mailreaver learns faster than mailtrainer is just this feature, I think.
• Stash mails you train on in your corpus, and don't hesitate to re-run mailtrainer over your corpus again after you have been training the unsure emails. When you train single emails, the changed filter may no longer correctly classify some email in the corpus you train with. Running mailtrainer over the corpus adjusts the filter to correctly classify every mail again. I generally retrain about once a week, though I no longer retrain from scratch, unless I change the classification algorithm.

Erich, it's not only killergames which threatens the legal distribution of linux in germany, but also hackertools like ping, ethereal or nmap. Check out this article (german only), which explains the new law which will come into effect in germany on the 15th of march this year.

Obviously you need hackertools, to be able to check if you're system is secure or vulnerable, but unfortunatly common sense is a lost art nowadays.

More maybe later, I'm in a hurry, but I at least wanted to make this remark. More discussion and evaluation of this is definitly needed!

Justification. Currently there is a huge mess of files and folders that start with a “.” in any users home folder. There is no structure or policy on how applications should choose file and folder names for data that needs to be stored in users home directory. Additionally there is no established consistency between Gnome, KDE and most other applications. Gnome application have part of their configuration information in gconf folder and other part in a gnome subfolder. KDE applications have a complex structure under .kde/. And most other applications either have one file directly in users home folder or have their own dot-folder there. Problems. There are two major problems with current ad-hoc approach:

1. Namespace pollution of users home space. In any desktop system “ls -al” in the home folder would be a very lengthy affair. Moreeven this namespace is folded - KDE and Gnome apps live in their own subfolders.
2. Lack of structure in folders makes it much harder to properly understand their purpose. For example, if a user has misconfigured an application and would like to reset it to default settings, it currently could be hard to figure out how to do that. Especially without losing his data in that application.

Proposal. I propose to implement a policy on where an application can store data and configuration information in users home folder. Such specification would be proposed as an extension for FHS, first implemented in Debian and then promoted for inclusion upstream and in other distributions. Structure. I propose a following structure for a users home folder:

• data/appname/ - each application, that stores user data in application-specific format, will have to store this data into this folder. Only configuration information that is essential to load this data would be permitted into this folder (in hidden dot files). For example, Evolution could store emails and email access information here.
• .conf/appname/ - each application that stores any configuration information whould have to store all of it in this folder. GConf will have to also store applications configuration info here. If this folder would be deleted, application must revert to default configuration, but still be able to fully access user data in data/appname/ or other, generic folders (below). For example, if .conf/evolution/ is erased, then on next start-up Evoution should still be able to access all email that is stored in data/evolution/ without any configuration wizards.
• .tmp/appname/ - each application that would need to create temporary files that survive reboot, would put them here. These folders can be safely deleted at any time with no data or configuration loss. For example, Firefox cache information and Nautilus thumbnail cache would have to be here.
• Desktop/ - the desktop folder, just like it is now
• Media folders - (Documents/, Photos/, Videos/, Music/, …) a suggestion of basic document structure for user to use or ignore. Applications dealing with particular media types would default to use files from such folders, but should also cope with working with any other folders on users choice. For example, a screensaver with slideshow function would by default use photos from Photos/, but will have to have a configuration option for user to change that setting.
• Additionally I would propose having a .mounts/ folder where applications could mount user specific media and other mounts, such as FUSE. For example, FUSE could be used for automatic archive introspection.

Benefits The main benefit for such system for me is twofold:

• SBackup can be configured to exclude .tmp/ and .mounts/ by default thus saving significant space in backups.
• A tool can be made to restore settings and/or data for specific applications in an uniform fashion.

So, any suggestions on the idea and on where this should be best discussed?

Just a quick alert since it's quite widely used and since I've completely missed the media coverage and only noticed it when logcheck started sending me long emails every hour: relays.ordb.org was shut down, you may need to adjust your mailer's configuration. At least they've not blacklisted the whole Internet (yet.)