Via the man who should be national security advisor, this fascinating email exchange between a “hacker” and a congressional staffer. The staffer wanted his college GPA “fixed” to improve his grad school outlook. The hacker apparently just wanted to mess with the guy:

If attention comes down, remember that we’re both equally liable in the actions. I’d rather not go to prison, you? I’ll use some snazzy IDS evasion techniques, spoof my IP, use my neighbor’s wireless, send decoy packets from a BitTorrent network, use some old PBX tricks to make them think it is an internal job and more. Think of all the stuff you see in the movies, but better.

Adrian von Bidder notes that he got a message from a random survey group asking for feedback.

[...] we analysed the debian mailing list archives, especially the debian-devel mailing list, as it is a good representation of the whole network. [...] we identified the 100 main actors and therefor came across your email address.

So, somebody who talks quite a bit on the mailing lists, but really only maintains one package is already one of the top 100 "actors" in Debian? I seriously doubt it.

It gets better. In the last couple years I've barely posted on any public Debian mailing lists at all. And yet I'm apparently also in the top 100 "actors". Say what?

I love that one of the benefits to working on free software is that it exposes you to tools that you wouldn't normally be aware of. A prime example is software that you can use for writing documentation, like LaTeX and revision control systems. At work I've been writing up a fairly long document. Normally at work I'm forced to use OO.o or word because I have to trade the file back and forth with my boss, but this time since I'm doing all the actual writing, I've been using LaTeX and git to manage the thing. In addition, I need a bibliography and references, so I basically needed something like LaTeX. I send my boss an rtf (made with latex2rtf) and pdf (made with pdflatex) when I want comments, and I can still use the highly structured style of LaTeX to make sure that things like my lists get formatted properly every single time. Trying to do this in Word was both frustrating and time consuming.

The best part about the experience has been using revision control though. Having atomic changesets with a log message is something we tend to take for granted in free software work, but compare it to the change tracking in OO.o and Word. It's amazing how much easier it makes it. I can actually see what I was thinking a few days ago, which was the last time I had been revising the document. Being able to visualize the changes and history with tools like qgit are even nicer. And tags... I don't think OO.o or Word lets you tag specific revisions, but I know that when I do this with Word, I end up with a ton of documents labeled "DavidRev1" and so forth, and I always mess up the revisions by using "Save" instead of "Save As".

And this doesn't even touch on the things I haven't done yet. I haven't done any branching, for example. I'm sure that when I write my thesis, I'll end up using that, but just knowing that it's available is exciting. The thing I wish I could do most though is actually put the repo up somewhere and have me and my boss actually use it the same way a normal shared source code repository is used.

The standard sorry ending to this story is that getting these things in broad use is basically a pipe dream. It continually amazes me that people will put up with Word or Powerpoint when it's so often the wrong tool for the job. The simplest things, like being able to look at a document's history and the diff and know in a minute exactly what you were thinking when you put down your work a week ago, help so much. It's sad because the technology problem has been largely solved (a small GUI frontend would be the last piece), but the evangelism problem remains. One thing is for sure, I'm not looking forward to writing my next paper in Word.

When I was on the job market last year, by far the toughest hurdle was convincing interviewing eyes and ears that my hacker and survivor project were connected in some fundamental ways. I think in the end, I was successful at times but it was difficult for some folks to see the connection in part because most professors knew little or zilch about survivor politics so I had to lay that ground work and then toss out the connections, which included free speech, the politics of freedom and self-determination, attacks on corporate/closed science. Now with the Zyprexa scandal and the support coming from Free Culture, these two worlds have indeed moved even closer together than before. (Thanks Jonah for pointing out).

So this is a little old news but it is still interesting news about freeing the suppressed zyprexa documents. Here is some additional background information .

You still got some money left after buying all those Christmas presents? How about donating some of it for a good cause:

• Wikipedia

Wikipedia is entering 2007 as one of the 10 most visited websites in the world. That's a great proof of our success. However, with this success comes a new set of challenges and responsibilities. As we plan for the future of Wikipedia and all of the Wikimedia projects, our two most important goals now are the reliability of our content and the long-term sustainability of every project in which we are involved. To meet these goals we have a lot of work to do, so I am asking for your help. In the coming year, the Wikimedia Foundation anticipates dramatically increasing spending to keep up with server and traffic capacity demands, add new staff on the organizational level, improve our software and develop methods to better ensure high quality content, all while making progress toward our goal of giving free knowledge to everyone.

-- Florence Devouard, Chair of the Wikimedia Foundation

Creative Commons

It's once again time to show your support for Creative Commons. Digital technologies are connecting people in ways that were never before possible but that network is fragile. Creative Commons needs your support to help enable a participatory culture a culture in which everyone can actively engage in the creativity that surrounds us. We need your support to assure access to cultural, scientific, and educational content that has been pre-cleared for use by its authors.

Free Software Foundation (FSF)

Freedom is more precious than anything else we have and we need to protect it while we still can.

-- Eben Moglen in his FSF year end video appeal (OGG, 24 MB)

Electronic Frontier Foundation (EFF)

EFF is a nonprofit group of passionate people lawyers, technologists, volunteers, and visionaries working to protect your digital rights.

Tor: anonymity online

Tor is a toolset for a wide range of organizations and people that want to improve their safety and security on the Internet. Using Tor can help you anonymize web browsing and publishing, instant messaging, IRC, SSH, and other applications that use the TCP protocol. Tor also provides a platform on which software developers can build new applications with built-in anonymity, safety, and privacy features.

Or how about some organizations not directly related to Free Culture or Free Software?

• Amnesty International (AI)

Amnesty International (AI) is a worldwide movement of people who campaign for internationally recognized human rights. AI s vision is of a world in which every person enjoys all of the human rights enshrined in the Universal Declaration of Human Rights and other international human rights standards.

Red Cross

The International Red Cross (and Red Crescent Movement) is an international humanitarian movement whose stated mission is to protect human life and health, to ensure respect for the human being, and to prevent and alleviate human suffering, without any discrimination based on nationality, race, religious beliefs, class or political opinions.

New York Times article on female scientists in academia. (Thanks for the link, Kathleen!) Also, the National Academy of Sciences recently published a report on barriers encountered by women in science and engineering in research universities. One thing I found particularly interesting was the observation that around 90 percent of married female science and engineering faculty have spouses that are employed full time, whereas the percentage for male faculty is less than 50 percent.

One of the reasons I have only blogged 21 times in thirty months is because of the very kludgey work flow I had for blogging; I had to manually create the file, and then scp by hand, and ensure that any ancillary files were in place on the remote machine that serves up my blogs. After moving to ikiwiki, and thus arch, there would be even more overhead, were it not so amenable to scripting. Since this is arch, and therefore creating branches and merging is both easy and natural, I have two sets of branches -- one set related to the templates and actual blog content I server on my local, development box, and a parallel set of branches that I publish. The devel branches are used by ikiwiki on my local box; the remote ikiwiki uses the publish branch. So I can make changes to my hearts content on the devel branch, and the merge into my publish branch. When I commit the publish branches, the hook function ensure that there is a fresh checkout of the publish branch on the remote server, and that ikiwiki is run to regenerate web pages to reflect the new commit. The hook functions are nice, but not quite enough to make bloggin as effortless as it could be. With the movge to ikiwiki, and dissociation of classification and tagging from the file system layout, I have followed the lead of Roland Mas and organized my blog layout by date; posts are put in blog/$year/$month/$escaped_title. The directory hierarchy might not exist for a new year or month. A blog posting may also show in in two different archive indices: the annual archive index for the year, and a monthly index page created for every month I blog in. However, at the time of writing, there is no annual index for the next year (2007), or the next month (January 2007). These have to be created as required. All this would get quite tedious, and indeed, would frequently remain undone -- were it not for automation. To make my life easier, I have blogit!, which takes care of the niggling details. When called with the title of the prospective post; this script figures out the date, ensures that the blog directory structure exists, creating path components and adding them to the repository as required, creates a blog entry template, adds the blog entry to the repository, creates the annual or the monthly archive index and adds those to the repository as needed, and finally, calls emacs on the blog posting file. whew.

Zack asked for website meta languages for redoing his homepage. Well, I redid my homepage last february, using XML and XSLT. A monster XSLT stylesheet, because I wanted to keep my template outside of the stylesheet. I can not recommend XSLT. Using it for templating is quite messy. XSLT is okay if you want to transform one representation of the data to another; it's not if you want to add a lot of surrounding markup and things like a sitemap and similar navigation tools. This is next to impossible in pure XSLT, it gets better once you have some extensions (dubbed EXSLT, and supported by pretty much any xslt processor) or maybe with XSLT2. String manipulation is also a pain (at least with XSLT1); I gave up on generating a nice "last modified" date from my subversion tag. Supposedly XSLT2 has some functions that could make this easier (i.e. for parsing and printing datetime information), but the common approach with XSLT is to only supply the required minimum of function you need, and I'm not aware of an easy way to add custom functions, not to mention any large standard library that can efficiently be used (which is the true strength of Java, Python and C#, that they bring a huge collection of pre-written ready-to-use code with them). Of course there are some efforts to write XSLT libraries (especially for XSLT2), and this aforementioned EXSLT is some kind of standard library that even might be efficiently implemented in some interpreters - you can't rely on it to be there and to just work. XSLT isn't useless, but when it comes to presenting data to humans or writing clean, compact code it's not satisfactory at all. I'd give you my Makefile and XSLT file, if they weren't that messy... too many features; I'm generating two languages, a partially expanded navigation menu, etc. - my XSLT is 10k, my sitemap currently 4.5k, the template file is 5k. Anyway. Half a year ago, my favourite templating language was KID templating. I used it for some small projects such as my DNSoupdate tool to edit DNS zones via the DNS protocol (requires a nameserver such as bind which has support for DNS update, uses encryption). It was a perfect match for such tiny pages, but I'm no longer that convinced I still like it for larger projects. What's good about Kid: XML templating language, Python based, easy to use in whichever way you want by writing a few lines of python (i.e. easy to write a Makefile to generate a static version of you homepage) What's not good about Kid: only works with a Python interpreter. I'd prefer to have a templating language that can be used from multiple languages. However the Kid syntax relies on Python, which is really bad. Also I'd prefer to have some component-render model for more complex websites. The current setup is IMHO okay if you have one default layout and one content layout; but if you have multiple components that could be combined differently on the pages, it gets too messy. With my turbogears experiments, Kid was also not very performant (but that might as well be Turbogears fault). Right now, I'd probably still go with Kid. But I've been having a look at JSP 2.1, and JSF / Facelets in particular. There are some things about facelets that I really like (for example, that they use a proper XML syntax, instead of this bastardized almost-XML that JSP usually (ab-)uses). There is also some stuff that I don't like (e.g. the massive overengineering of everything surrounding it), and I have no idea how easy it will be to generate static pages in a scripted fashion, i.e. using it without a real webserver. Or I might just write it all in python, which is a nice language for manipulating XML, usually. Please don't just send me an email with your favourite templating engine. Like zack, I'm only interested in XML-based templating engines, which does not apply to most templating solutions out there. Clearsilver for example is another bastardization of XML. I'm aware of TAL/METAL, and find them quite interesting, but they were also not having this kind of componentization that I'm usually thinking in. [Update: some people have pointed me to Genshi, which is mostly Kid compatible. However, it still has mostly the same problems, e.g. the templates being not reusable in other languages than python, and that certain constructs are a pain to do (e.g. the page_specific_css recipe with more than one css file). Others have pointed me to smarty, but it's string-based and doesn't ensure valid XML output. (Which is very useful for e.g. generating atom or rss output) For example this is probably valid in smarty: if 0 <b> /if </b> - allowing errors is bad. Oh, and smarty is PHP, which is broken by design, a no-go. The best match to my ideas so far is XML::Template (for Perl, Python, Ruby, PHP) which is pretty close to what I've been doing manually when not using a templating solution. I don't know yet how well it handles recursion - I need recursive templates for the navigation menu on my homepage.]

The first step is to configure the CRM114 files, and these are now in pretty good shape as shipped with Debian. All that I needed to do was set a password, say that I'd use openssl base64 -d, and stick with the less verbose defaults (so no munging subjects, no saving all mail, no saving rejects, etc, since I have other mechanisms that do all that). The comments in the configuration files are plenty good enough. This part went off like a breeze; the results can be found here (most of the values are still the upstream default). The next step was to create new, empty .css files. I have noticed that creating more buckets makes crm114 perform better, so I go with larger than norm crm114 .css files. I have no idea if this is the right thing to do, but I make like Nike and just do it. At some point I'll ask on the crm114-general mailing list.

    % cssutil -b -r -S 4194000 spam.css
    % cssutil -b -r -S 4194000 nonspam.css

Now we have a blank slate; at this time the filter knows nothing, and is equally likely to call something Spam or non-Spam. We are now ready to learn. So, I girded my loins, and set about feeding my whole mail corpus to the filter:

 /usr/share/crm114/mailtrainer.crm               \
   --spam=/backup/classify/Done/Spam/           \
   --good=/backup/classify/Done/Ham/            \
   --repeat=100 --streak=35000                  \
         egrep -i '^ + train Excell Running'

And this failed spectacularly (see bug #399306). Faced with unexpected segment violations, and not being proficient in crm114's rather arcane syntax, I was forced to speculation: I assumed (as it turns out, incorrectly) that if if you throw too many changes at the crm114 database, things rapidly escalate out of control. I went o to postulate that as my mail corpus was gathered over a period of errors, the characteristic of Spam drifted over time, and what I consider Spam has also evolved. So, some parts of the early corpus are at variance with the more recent bits. Based on this assumption, I created a wrapper script which did what Clint has called training to exhaustion -- it iterated over the corpus several times, starting with a small and geometrically increasing chunk size. Given the premise I was working under, it does a good job of training crm114 on a localized window of Spam: it feeds chunks of the corpus to the trainer, with each successive chunk overlapping the previous and succeeding chunks, and ensuring that crm114 is happy at any given chunk of the corpus. Then it doubles the chunk size, and tries goes at it again. All very clever, and all pretty wrong. I also created another script to retrain crm114, which was less exhaustive than the previous one, but did incorporate any further drift in the nature of Spam. I no longer use these scripts; but I would like to record them for posterity as an indication of how far one can take an hypothesis. What it did to was have crm114 learn without segfaulting -- and show me that there was a problem in the corpus. I noticed that in some cases the trainer would find a pair of mail messages, classify them wrongly, and retrain and refute -- iteration after iteration, back and forth. I noticed this when I added the egrep filter above, and was not drowning in the needless chatter from the trainer. It turns out, I had very similar emails (sometimes, even the same email) in the Ham and the Spam corpus, and no wonder crm114 was throwing hissy fits. Having small chunks ensured that I had not too many such errors in any chunk;and crm114 did try to forget a mail differently classified in an older chunk and learn whatever this chunk was trying to teach it. The downside was that the count of the differences between ham nd Spam went down, and the similarities increased -- which meant that the filter was not as good at separating ham and Spam as it could have been. So my much vaunted mail corpus was far from clean -- over the years, I had misclassified mails, been wishy-washy and changed my mind about what was and was not Spam. I have a script that uses md5sums to find duplicate files in a directory, and found, to my horror, that there were scores of duplicates in the corpus. After eliminating outright duplicates, I started examining anything that showed up with an ER (error, refute) tag in the trainer output; on the second iteration of the training script these were likely to be misclassification. I spent days examining my corpus and cleaning it out; and was gratified to see the ratio of differences to similarities between ham and Spam css files climb from a shade under 3 to around 7.35. Next post we'll talk about lessons learned about training, and how a nominal work flow of training on errors and training when classifiers disagree can be set up.

Looking at my webserver logs, there seems to be a highly increased demand for my 10 + 100 Creative Commons Christmas Songs blog post from last Christmas... I wonder why... ;-) So I decided to update the list this year, adding a bunch of additional songs; in total there are now more than 230 songs listed. Of course the list is incomplete, so any further links to Creative Commons licensed Christmas music are very welcome. Just leave a comment here or directly on the old blog post and I'll update the list. I'll probably also put up some more Christmas-related songs on my Creative Commons music podcast over the next few days... Image credit: Photo taken from Wikipedia. Author: Kelvin Kay. License: GNU Free Documentation License (GFDL), version 1.2.

So, today, I finally stopped the l10n NMU campaign. It started on Sept. 25th with the "Intent to Work" on hesiod. The last package I worked on has been isync today. The goal of that campaign was closing as many longstanding l10n bug reports as possible. The main goals were decided during the i18n work meeting in Extremadura, back in September. We focused on po-debconf translations, mostly, which are an easy target for NMUs. Thomas Huriaux extracted the data from the BTS and ranked packages with regards to the number of pending l10n bugs and their age. Lucas Wall reactivated a coordination page which we already used back in early 2006 when a first version of such a campaign was run. During that campaign, I worked on 102 packages. 89 of them got all their pending l10n bugs closed, 5 of them got most of their l10n bugs closed (remaining "l10n" bugs were either not really related to localisation and strange things which I left aside), 8 packages were ignored either because their respective maintainers wanted to handle the bugs themselves or because they were tagged for removal. Meny packages have also been handled by their respective maintainers, either because the NMU announcements woke them up...or because of their natural release process. I have to point out that the level of collaboration of package maintainers has been great (except for those who are MIA, and there are quite a few) with only one very proeminent exception (just guess which one). Thijs Kinkhorst and Tobias Toedter also contributed to the campaign. Thanks to them as well. As a side effect, we also made many of these packages as lintian-clean as we could, trying to balance between too invasive NMUs and trivial fixes. IIRC, I only introduced 3 or 4 RC bugs doing so (which I fixed immediately of course). A few RC bugs were also fixed by the NMUs and even a security bug has been hunted down (for screen). Meny thanks to all package maintainers, translators and everybody who helped during that work. It has been a great time. Just look at the result. Now heading to the improvement of the next pet project of the i18n task force: the Debian i18n server.

Debian Etch on Asus U5F Model Asus U5F, with Intel Core 2 Duo T5500 Distribution Debian Etch amd64. Summary Video: works, with 3D and everything. Audio: works, with a little config change. Wireless LAN: works, with drivers from non-free. Bluetooth: works. Ethernet: works. SD card reader: works. DVD reader/burner: works. ACPI features: various problems. Suspend: broken. Installation and wireless card To have the wireless card working you need some package from non-free that is not provided with the standard installer. I installed using the binary 1 CD, then I fetched the following packages from unstable:

firmware-ipw3945_0.3_all.deb
ipw3945d_1.7.22-2_amd64.deb
ipw3945-modules-2.6.18-3-amd64_2.6.18+1.1.2-2_amd64.deb
linux-headers-2.6.18-3_2.6.18-8_amd64.deb
linux-image-2.6.18-3-amd64_2.6.18-8_amd64.deb

Once those packages are installed, wireless just works. Video Intel 945GM/GMS/940GML It just works, with 3D and everything. To turn on AIGLX and Compiz, just follow the HOWTO in the Debian wiki. Audio Intel 82801G To make it work add:

options snd-hda-intel model=laptop-eapd

to /etc/modprobe.d/alsa-base. Rationale: this link gives the quick solution. Note: no need to reboot:

/etc/init.d/alsa unload
modprobe snd-hda-intel

The list of models available for the driver is in /usr/share/doc/alsa-base/driver/ALSA-Configuration.txt.gz A hint about the model to use can be found at the beginning of /proc/asound/card0/codec#0. CPU speed scaling Just modprobe speedstep-centrino. Then the cpufreq_ondemand module works fine. I'm a bit disappointed that it doesn't want to go below 996Mhz (60%) and it has only 3 scaling modes: 1.66 GHz, 1.33 GHz, 996 MHz. It used to display available scaling modes weirdly (it would repeat 1.66Ghz 6 times), then I updated the BIOS to version 304 and now it shows just those 3. The laptop tends to get warm, this disappoints me. I hope things will improve with newer kernels. USB Just works. SD card reader Just works. I put an SD card in and it gets mounted automatically. In the old Asus M2 the card reader was just some unsupported obscure thing. Ethernet RTL8111/8168B Gigabit Ethernet. I'm not sure it was working in the installer, but in the installed system it works out of the box. Very pleasant thing: it can cross the cable if needed. DVD reader/writer Just works. I haven't tried burning CDs or DVDs, but k3b sees it and correctly detects the writing speed. Bluetooth Just works. I connected to the bluetooth GPS just fine. ACPI Still quirky. Brightness control buttons don't work: before updating the BIOS I would get this reaction on syslog:

Dec 17 23:57:35 viaza kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PCI0.SBRG.EC0_._Q0E] (Node ffff81003e98bd90), AE_NOT_FOUND
Dec 17 23:57:35 viaza kernel: ACPI Error (psargs-0355): [\_SB_.PCI0.P0P2.VGA_.LCDD] Namespace lookup failure, AE_NOT_FOUND

After updating the BIOS I get this reaction instead:

Dec 21 21:17:38 viaza kernel: video device notify
Dec 21 21:17:38 viaza kernel: set_level status: 0
Dec 21 21:17:38 viaza kernel: compiz.real[4013]: segfault at 0000000000000060 rip 000000000040b3b0 rsp 00007fff6c47f4e0 error 4

compiz segfaults and I'm back to gdm, with the brightness unchanged. The LCD on/off button works. Controlling the LCD via software does not work:

# echo 1 > /proc/acpi/asus/lcd
Dec 21 22:22:45 viaza kernel: Asus ACPI: Error switching LCD
# cat /proc/acpi/asus/lcd
Dec 21 22:22:49 viaza kernel: Asus ACPI: Error reading LCD status
0

In fact, asus-acpi complains:

Dec 21 17:08:29 viaza kernel: Asus Laptop ACPI Extras version 0.30
Dec 21 17:08:29 viaza kernel:   unsupported model U5F, trying default values
Dec 21 17:08:29 viaza kernel:   send /proc/acpi/dsdt to the developers

Therefore, I sent the dsdt to the developers. The wireless on/off button turns on and off the wlan and bluetooth leds, and apparently also the wireless hardware. I still don't know how to turn off bluetooth and leave wlan on. Suspend Suspend to ram works, but on resume it does not turn on the LCD display. The LCD on/off button has no effect. Other things work fine, so I can blind-type a shutdown command. I have vbetool installed, but it doesn't seem to help. So far, suspend is still not usable. Modem Not tried.

Joey and Josselin, I'm calling you childish, not because you "stand" your principles, but because of the way you try to convince others that your principles are better. What you are using is:

• Insults ("lying", and for the record, FUD means twisting the facts and telling only half the story, and you are twisting the facts, as I detailed in my post; "should read constitution", which of course means to imply that I'm trying to deny you some constitutional right, which I'm not, I'm just asking you to stop spreading FUD)
• Blackmail and threats ("I'll stop contributing" [if dunc-bank is paying anyone], "reduced it much less than I could, probably much less than I should")
• Schadenfreude (the libpng mishap of the new maintainer; noone broke libpng intentionally. That is not a nice character trait, and pollutes the community)
• You seriously annoy people by going on and on and on, while everybody has already understood your opinion and arguments (and just happens to have a different opinion)
• Playing big words without actual content ("read the constitution 2.1.1", which btw. also states that you "must not actively work against these rules and decisions properly made under them"; your attempt at a GR failed, and you are working against these decisions on planet; I've however never said that you're oblieged to still work for Debian if you don't like the decision)

Actually, I'm by now convinced that it would have been better for Debian, if you would just have really left. Your behaviour in the last months makes you qualify as "poisonous people" (except these usually aren't big contributors). In my opinion, your infighting has harmed the project, it gives people the impression that being involved in Debian means mostly having to deal with such things. While it's not - Debian is still fun to me, although I have little time, and didn't manage to go to the last few Debian meetings here in Munich - all these fights are a nuisance to me. Even though I'm not actually affected by them (neither Dunc-Tank, because it doesn't demotivate me, nur the powerpc fights, nor some other fights on -private), the way they are dicussed on the Debian lists does annoy me. Again: if anything is cutting on my motiviation, it's your behaviour (this btw. includes the powerpc issue and fighters, too, especially Sven Luther who has probably sent as many mails as all others together, just reiterating the same things again and again). So if you are planning to become a Debian contributor, learn to use the "delete" button in your email clients, and don't be afraid to just delete whole threads. Certain threads are just a waste of time and bring negative emotions. So: Joey, Josselin: grow up, and accept that people sometimes have a different opinion, and you can achieve more by accepting that your opinions differ and just try to make the most of it anyway. Reminding them of your differences won't make them change their mind, nor will it make them like you any better. Fortunately, most people in the project (well, everybody except me?) seem to obey above "poisoned people" recommendations, and just ignored your posts on this issue. I didn't, and one of the reasons it that I feared Joeys first post could grab some media attention, and I felt there should be at least one post saying that, well, the debian-installer and some other RC bugs are probably also a cause of the delay, and it's probably not worth rehashing the dunc-bank issue. So far, I've only seen one news article citing aba that the delay was caused by the debian installer among others, no "the delay is due to dunc-bank demotivating all developers" FUD.

I have a fairly sophisticated Spam filtering mechanism setup, using MIMeDefang for SMTP level rejection of Spam, grey-listing for mails not obviously ham or Spam, and using both crm114 and spamassassin for discrimination, since they compensate for each other when one of them can't tell what it is looking at. People have often asked me to write up the details of my setup, and the support infrastructure, and I have decided to oblige. What follows is a series of blog posts detailing how I set about redoing my crm114 setup, with enough detail that interested parties can tag along. I noticed that the new crm114 packages have a new facility called mailtrainer, which can be used to setup initial css database files. I liked the fact that it can run over the training data several times, it can keep back portions of the training data as a control group, and you can tell it to keep on going until it gets a certain number of mails discriminated correctly. This is cool, since I have a corpus of about 15K Spam and 16K ham messages, mostly stored on my previous train-on-error practice (whenever crm114 classified a message incorrectly, I train it, and store all training emails). I train whenever crm114 and spamassassin disagree with each other. This would also be a good time to switch from crm114's mailfilter to the new mailreaver, which is the new, 3rd Generation mail filter script for crm114. It is easier to maintain, and since it flags mails for which it is unsure, and you ought to train all such mails anyway, it learns faster. Also, since the new packages come with a brand new discrimination algorithms which are supposed to be as accurate but also faster, but may store data in incompatible ways, I figured that it might be time to redo my CRM mail filter from the ground up. The default now is "osb unique microgroom". This change requires me to empty out the css files. I also decided to change my learning scripts to not send command via email to a 'testcrm' user, instead, now I train the filter using a command line mode. Apart from saving mails incorrectly classified into folders ham and junk (Spam was already taken), I have a script that grabs and saves mails which are classified differently by crm114 and spamassassin from my incoming mail spool and saves it into a grey.mbox file, which I can manually separate out into ham and junk. Then I have a processing script, that takes the ham and junk folders and trains spamassassin, or crm114, or both; and stashes the training files away into my cached corpus for the future. In subsequent blog postings, I'll walk people through how I setup and initialized my filter, and provide examples of the scripts I used, along with various and sundry missteps and lessons learned and all.

For those who've been hiding under a rock for the last few months: dunc-tank is the (controversion) project that was founded by the current DPL, Anthony 'aj' Towns. Controversial, because a lot of people feel that this goes against the spirit of Debian; and that by paying some Debian people and not paying others, we're creating a two-class system that we'd best avoid. And probably a bunch of other arguments, too, but that's not what this blog post is about. Me, I've been staying rather quiet in the whole debate. It's not that I don't have an opinion; rather, I haven't been voicing it, mostly because my thoughts on the matter are rather convoluted, and I'm not sure I can put them in words pretty well. But since the debate seems to be going in circles, and since I think that I might be able to help it out, I'll try anyway. If that doesn't bring the debate any further, well—can't blame me for trying. Do I think that paying people to do work on Debian is necessarily a bad idea? No, not at all. Quite to the contrary; by paying people to work full-time on Debian for a while, we will increase the amount of work that's being put into Debian, which can only be a good thing; and I can only applaud aj for trying to get this done, since it should be obvious to anyone involved that by proposing this, he only had Debian's best interests (i.e., improve our distribution) in mind. The obvious question, however, is who we'll be paying. After all, while I don't think it's necessarily bad to pay people, I do think that with money, you introduce the possibility for corruption; so at the very least, it should be strictly checked what exactly happens with the money that's being spent. Accordingly, the way in which you pay people—your whole set-up and procedures—should have a number of safeguards. Which brings me to my next point. Do I think that dunc-tank, as a project, is a good idea? No, not at all. Quite to the contrary; and I have a number of reasons for that. First, I don't think that deciding by committee who can get paid and who can't is the way to go forward. A committee will always be biased, no matter how diverse its membership. Even if you could find a committee that would not be biased—an impossibility by definition—then there will still be people who will not agree with the committee's decision. Are they right not to agree? Not necessarily. Is it bad to create such controversy if it could be avoided? Sure as hell. Moreover, by creating a project which populated by Debian people and which has the sole purpose of paying Debian people, it's fairly possible to create the impression as if all other ways to get paid to do Debian work would be unsound; that if a developer accepts money for Debian work without going through the committee, he's somehow doing something morally wrong. This would be very bad indeed; I know for a fact that there are cases of people paying Debian Developers to work a few days on a certain package so that a certain bug which impedes their business is fixed, or so that a certain version of a random package which this company is using, is uploaded into the archive. I've done this myself in the past (although I can't disclose more, since the person who paid me requested to remain anonymous); I know of at least two other Debian Developers who've done so as well. So what's the alternative? Not paying people is one, but it would not necessarily be the best. One alternative, which the FreeBSD people have been doing, is just "no organization at all". People who want to get paid just put up a web page with the amount of money they would like to have, and it's up to the community to decide whether their request is valuable enough for it to actually be paid. To some extent, this is also what the Dunc-Tank committee is doing, except that as FreeBSD does it, there is no committee in between. The problem with this approach, however, is that if we all go ahead and ask for money at the same time, it'll not only make us look like a bunch of beggars; it's also very likely that nobody will actually be getting any money, because there's so many "good causes" to choose from that there might not be enough donations to fund them all. Another proposal was one made by Manoj Srivastava, before word got out that dunc-tank would be started (or about at that time). As it was made on the debian-private mailinglist, I can't say much about that; suffice to say that I feel his proposal was overly bureaucratic, and had problems that were similar in nature to the problems I feel exist with dunc-tank. So do I have any answers which might get us a way out of the current status quo? Nope, afraid not. But allow me to make this final observation: In early 2005, a bunch of people decided that Vancouver was a nice place, and travelled there. As it happened, they were all Debian people. What a coincidence. Since they were all there anyway, they decided to talk a bit about Debian, and as a result, came up with some plan for its future. This plan, too, was rather controversial. When I initially read the first few paragraphs, I suddenly fell upon this paragraph which seemed to say that "we suspect we won't be releasing with anything but i386, amd64, and powerpc for etch". Things suddenly turned kinda red from then on. And it wasn't the wine that I hadn't been drinking. I could imagine that what I felt back then is somewhat similar to what some other people felt when they first heard about the suggestion to pay people for Debian work; and this realization is the reason why I haven't been dismissing their arguments or thoughts, much like I've seen other people do. However, I cannot say that I feel the current crisis to have many parallels with the Vancouver one; this is mostly because in the current case, people on both side of the argument seem to be building up walls around themselves, to help them ignore whatever the other side was saying. In contrast, when someone suggested something which had the potential effect of throwing most of the hard work I'd been putting in for Debian during the last five years now down the drain, I went to talk with them—after the heat had cooled down, anyway. Unfortunately for me, the result is exactly the same; for all my effort, I haven't been able to prevent m68k to be kicked out of the release; and to say I'm not happy about this would be an understatement. But at least this way the port has been given a fair chance; if I hadn't been talking to the people on the other side of the argument, I would still be talking about Steve Langasek as if he were the devil himself, and m68k would probably have been kicked out of Debian—along with a bunch of other architectures. At least I got that bit fixed. I guess what I'm saying is that people, rather than farting in eachother's general direction, should be working together in finding a compromise. Sure, that means making concessions, and agreeing to something you'd rather not agree to. But at least this way, the thing you have to agree to isn't as bad as what it could be. And trust me, people will do the stuff you don't want them to do, with or without your agreement. Hell, they've already started. Whoa, long post. I'll stop preaching now.

Dear Santa Pyro,
I've been a really good maintainer this year and I hope not to get coal. Don't let my not-updated-lintian-page fool you. My packages hasn't got any RC bug in unstable or testing and they are all updated.
What I really want for Christmas is Task and Skill step finished. That will look great under Christmas tree :)
Thanks in advance.

Your eternal applicant, luciano

This morning I received an email from the AMD Developer Center advising me that I need to fill out their NDA if I want access to their development machines.

I have a vague recollection that when AMD64 was first released I was very keep to get access to such hardware and had applied to AMD for access to their machines.

Of course now the second-hand market is full of AMD64 machines and I've got one in my server room so it's not as useful as it once was. I don't even know why AMD would still run a developer center given that everyone who wants AMD64 machines can cheaply buy as many as they want and organizations such as Sourceforge and Debian provide access to such machines for their members.

While I appreciate what AMD is doing, it probably would be best if companies could adopt a standard timeout for electronic correspondence. If someone doesn't follow up for X months then you should assume that they are not interested.

Yesterday, I lost many emails because of operation failure. I tried to remove mass spam in Maildir. Unfortunately, I misunderstood to chdir spam directory instead of inbox. So all of email from a month were removed... I need to make a script to prevent such failure.

I am about 2/3 over with my haul from Edmonton to San Juan, PR (laying over in the beautiful JFK) and am too tired to blog anything much of substance but do check out Furious Season’s impassioned and excellent commentary on the recent NYTimes article (and this one )that reveal how Eli Lily knowingly downplayed the risks of Zyprexa. And if you read the second article, you will find out what Viva Zyprexa means. Here is Eli Lily’s press release, which I have included on the next page as these things tend to vanish… Statement from Eli Lilly and Company: Response to The New York Times Article from December 18, 2006 December 18, 2006 INDIANAPOLIS, Dec 18, 2006 /PRNewswire-FirstCall via COMTEX News Network/ — Eli Lilly and Company (NYSE: LLY), vigorously objects to the characterization of company practices in a New York Times article based upon selective documents illegally leaked by plaintiffs’ lawyers. “At Lilly, we do not engage in off-label promotion - as alleged in The Times article,” said Dr. Steven Paul, Lilly’s executive vice president, science and technology. “Lilly is committed to the highest ethical standards and to promoting our medications only for approved uses. We have clear guidelines and extensive training for our sales representatives to help assure that they provide appropriate promotional information that is within the scope of prescribing information approved by the FDA.” Lilly works to bring Zyprexa to physicians who are confronted with the need to diagnose and treat serious schizophrenia and bipolar disorder wherever they practice. * About half of medical care for serious mental illness takes place in a primary care physician’s office. This is due to the fact a large number of people in the United States have no access to a psychiatrist or do not seek psychiatric care. * Our experience in mental health has taught us that primary care physicians are asking for education on severe mental illnesses, since they see many of these patients in their offices. * We believe that it is absolutely appropriate to discuss Zyprexa and its indicated uses with primary care physicians in the interest of meeting a critical medical need. * It is important that the public understand that physicians can and do prescribe medications outside of their approved indications to meet the needs of their individual patients. “While it is accurate to say that mental illnesses like schizophrenia and bipolar disorder begin early in life — with a higher prevalence during early adolescence and childhood — it is simply untrue to assert that these diseases somehow end — and therefore prescribing for them ends — in young adulthood,” added Paul. “To dismiss the devastating impairment of these disorders throughout a patient’s life is wrong. “Bipolar disorder and schizophrenia are often treated in the offices of primary care physicians. In fact Lilly did research with primary care physicians and found that they were challenged in making the diagnosis of bipolar disorder. Many primary care physicians appreciate the educational outreach Lilly provides in mental illness,” he added. The Times failed to mention that these leaked documents are a tiny fraction of the more than 11 million pages of documents provided by Lilly as part of the litigation process. They do not accurately portray Lilly’s conduct. As part of Lilly’s commitment to patients and healthcare professionals, many high-level Lilly physicians and researchers — along with researchers from outside Lilly — were engaged for a number of years to study the issue of Zyprexa and diabetes. Leaked documents involving these discussions do not represent an accurate view of company strategy or conduct. Lilly deplores the illegal release of select confidential documents. This illegal and selective disclosure of incomplete information will cause unwarranted concern among patients that may cause them to stop taking their medication without consulting a physician. This is the unfortunate result we saw when plaintiffs’ lawyers aggressively advertised about Zyprexa in recent years while searching for clients. Lilly, a leading innovation-driven corporation, is developing a growing portfolio of first-in-class and best-in-class pharmaceutical products by applying the latest research from its own worldwide laboratories and from collaborations with eminent scientific organizations. Headquartered in Indianapolis, Ind., Lilly provides answers — through medicines and information — for some of the world’s most urgent medical needs. Additional information about Lilly is available at www.lilly.com. C-LLY Zyprexa (olanzapine, Lilly) (Logo: http://www.newscom.com/cgi-bin/prnh/20031219/LLYLOGO ) SOURCE Eli Lilly and Company Tarra Ryker, +1-317-276-3787, mobile: +1-317-332-7502, or Marni Lemons,
+1-317-433-8990, mobile: +1-317-532-7826, both of Eli Lilly and Company http://www.prnewswire.com Copyright 2006 PR Newswire. All rights reserved News Provided by COMTEX