Experiences with a Trac Server In the last few days after Xmas I set up http://trac.cs.unibo.it, a server for my CS department which will host a bunch of Trac instances, initially for academic projects only then it will be opened to student projects as well. The server set-up has been "powered-by" the following technologies:

1. Debian Etch
2. Apache 2.2 with FastCGI, SuExec, and WebDAV over TLS (for accessing SVN repositories without forcibly passing through Trac)
3. Trac instances backed by Sqlite databases and Subversion repositories
4. Some bits of Genshi templating for generating the projects index
5. Some shell scripts for automating creation and removal of projects

I'm quite happy with the result. From the overall experience I drew some observations though:

• installing Etch is more a dream than ever, in 15 minutes I re-installed the server (which previously was running Ubuntu) with a decent partitioning scheme and all that I needed to work remotely. The sysadm of the department saw me coming out from the server room and asked, unbelievingly, did you already finish?
• apache2.2 in Debian is now annoyingly missing the nice script for creating easily TLS certificates (i.e. apache2-ssl-certificate). I followed-up on #395823 attaching the same script took from Sarge and the needed configuration file. Worked for me
• SuExec is stupid. You have to put stuff under /var/www for it to work at all, the only way to change the path is rebuild apache .... or, you can mount --bind /srv/trac /var/www/trac!
• BTW I like /srv. According to the FHS it's the place where I want to put my Trac projects. Unfortunately the Debian installer neglects that path and per default you end up with a system where /srv is in the same partition as /, and hence with basically no free space. Hence I also bind-mounted /srv/trac from /home/trac
• In Etch we have no available implementation of FastCGI (which really improves Trac performances BTW). libapache2-mod-fastcgi is non-free and for some reason it was not autobuild everywhere and hence is missing from testing. I pinged aba on IRC about that, so the package should probably enter testing again in a few days. Go RMs!

Happy new year birthday trac.cs.unibo.it.

Russel pointed me to Andre Pang's blog entry.  Yep, I share this annoyance.  Especially because I recently received an email with a 3 or 4 line disclaimer at the top of the email, and an additional, longer one at the bottom.  Occasionally, I use that one:

The content of this message may or may not reflect the opinion of me, my
employer, my girlfriend, my cat or anybody else, regardless of the fact
whether such an employer, girlfriend, cat, or anybody else exists.  I
(or my employer, girlfriend, cat or whoever) disclaim any legal
obligations resulting from the above message.  You, as the reader of
this message, may or may not have the permission to redistribute this
message as a whole or in parts, verbatim or in modified form, or to
distribute any message at all.

(Daniel Baumann: Please note the spaces I've added in this post, extra for you!)

What is Debian Desktop?
Debian Desktop is a Debian's subproject created to be a joint effort of
all groups involved or related with the desktop to give our users a
better environment to use and easier to install experience.

Actually we've members from: pkg-gnome, pkg-kde, pkg-xfce, pkg-galago,
tasksel and utnubu. We count with help from d-i and debian-cd members too.
The initiative merged the former Debian Desktop development page, debian-desktop
mailing list for the discussions and debian-desktop alioth project - its svn actually
hosts the desktop-base source package.

We did our best to deliver for our users a interesting desktop experience (including
laptop users) and it includes GNOME, KDE or Xfce (that's your choice) in our upcoming release: 4.0 (Etch).

What will be into Debian Etch Desktop ?
The desktop (task: desktop, installable through aptitude or tasksel):

• X Window System: Xorg 7.1
• Office suite: OpenOffice 2.0.4
• Graphics suite: GIMP 2.2.13

It also includes a simple window manager (twm), zeroconf (avahi), printing (cups) and scanning (sane) support

GNOME desktop (task: gnome-desktop, also installable through aptitude or tasksel):

• GNOME core: 2.14 with some 2.16 modules, eg.: totem (xine backend as default)
• Web Browser: Firefox 1.5 and Epiphany 2.14.3
• Feeds Manager: Liferea 1.0.26 (Liferea is Epiphany friendly)
  
• Mail/Calendar app: Evolution 2.6.3
• Instant Messaging Tool: GAIM 2.0.0beta5
• Multimedia Player: Rhythmbox 0.9.6
• Network (wired/wifi) manager: Network Manager 0.6.4
• Graphical Package Manager: Synaptic 0.57.11 with update-notifier
  
• Graphical Partition Manager: GParted 0.2.5
• CD/DVD Creation: GNOMEBaker 0.6.0
  
• Power Management Tool: GNOME Power Manager 2.14.3

It also includes bluetooth (bluez-gnome) support, multimedia in the browser support (totem mozilla plugin), bittorrent support (gnome-btdownload)

KDE desktop (task: kde-desktop, also installable through aptitude or tasksel):

• KDE: KDE 3.5.5 (base, pim, multimedia, graphics, artwork, network, utils and admin)
  
• Network (wired/wifi) manager: KNetworkManager 0.1
• CD/DVD Creation: k3b

We tried to add all the gnome task features into kde task too. KDE itself already contains a lot of them, if you find something missing please tell us at debian-desktop@lists.debian.org for a better next release, and wait your contribution enter Lenny.

Xfce desktop (task: xfce-desktop, also installable through aptitude or tasksel):

• Xfce: Xfce 4.3.99
• Calendar tool: orage 4.3.99
• Text editor: Mousepad 0.2.8

It also includes multimedia support and xfce goodies.

What if i've Debian testing or sid installed using GNOME or KDE or Xfce?
Open a terminal, check your /etc/apt/sources.list and run as root:
aptitude update
aptitude install desktop $foo-desktop

Please replace $foo with GNOME, KDE, Xfce - e.g: aptitude install desktop gnome-desktop

Note that desktop and $foo-desktop aren't metapackages - packages that usually contains just a few files (the required) to be installed in the system and its main feature is depend, conflict, recommend or suggest other packages. desktop and $foo-desktop are tasks and are in the tasksel source package, maintained by the tasksel team. The debian-installer uses tasksel to install a set of packages, and you should too.

How can i download Debian Desktop for tests?
The easier way to install is downloading a CD or DVD image from http://www.debian.org/devel/debian-installer . Please download
Debian Etch RC1 images or later for GNOME and something newer
(eg: weekly builds) for KDE or Xfce. Please note that DVD usage
is preferred, if you're going to use a CD broadband is recommended.

There are special first CDs images containing not GNOME key packages
but KDE or Xfce, look for debian-testing-i386-$foo-CD-1.iso (don't worry,
we've at least amd64 and powerpc images too) at:
http://cdimage.debian.org/cdimage/weekly-builds/i386/iso-cd/

How can i install Debian Desktop GNOME?
The first DVD image contains the entire GNOME set of packages, just
follow the installer and after the base system install select "default
desktop environment", if the system doesn't suggests it for you and
proceed - no network is required during installation.

The first CD image doesn't contains the entire GNOME set of packages,
just follow the installer and after the base system install select
"default desktop environment", if the system doesn't suggests it for
you and proceed - broadband connection is recommended.

How can i install Debian Desktop KDE?
Before the installer boots, please type:
install tasks=kde-desktop

* The first DVD image contains the entire KDE set of packages - no
network is required during installation.

* The first CD image doesn't contains the entire KDE set of packages -
broadband connection is recommended.

Note: Alternative first CD is available (see above).

How can i install Debian Desktop Xfce?
Before the installer boots, please type:
install tasks=xfce-desktop

* The first DVD image doesn't contains the entire Xfce set of packages
- broadband connection is recommended.

* The first CD image doesn't contains the entire Xfce set of packages
- broadband connection is recommended.

Note: Alternative first CD is available (see above).

Adam Rosi-Kessel made an interesting post about They Might Be Giants producing children's music because their original fan base are now old enough to have children.

From casual inspection of the crowds at events such as Linux Conf AU it seems to me that many serious Linux people are also at the right age to have young children, and several blogs that are syndicated on Linux Planets provide evidence of this. Therefore it seems that there is a market for Linux related children's music.

Many aspiring artists complain about the difficulty of establishing a reputation. I think that if someone was to release OGG and FLAC recordings of a children's version of the Free Software Song under a Creative Commons license then they would get some immediate publicity through the blog space and Linux conferences which could then be used to drive commercial sales of children's music.

While on the topic, it would be good to have a set of children's songs and nursery rhymes to teach children from a young age about the community standards that we share in the Free Software community. There is no shortage of propaganda that opposes our community standards, the idea that sharing all music and software is a crime is being widely promoted to children.

I've been holding off blogging a number of things because I've got a list of time-sensitive tasks as long as my arm to do in my capacity as a volunteer sysadmin for Linux Australia. The immediate pressing one is to swap mirror.linux.org.au over to new hardware with more disk capacity. I've been trying to do this whilst globetrotting to Australia for a holiday, Spain for a Debian QA meeting, and Las Vegas for Christmas. So work on it's been a bit fragmented. Tonight was going to be the night of the cutover. I've had the new machine on a different address while I synced configs (and content), and tonight I swapped IP addresses around so that the old machine would have the IP address that the new machine had, and the new machine would have mirror.linux.org.au's IP address. I rebooted both machines, and... Neither one has come back. Sigh. And of course, it's late afternoon on a Friday in Australia, I'll bet Steve isn't even working this week, so he can't take a look at it. I can't think what I've bollocksed up, but any ARP conflict on the upstream router should have sorted itself out by now. Damn.

I've been feeling sick since the day after Christmas. Maj says that when you push your body too hard, it gives you a cold to tell you to sit back and take it easy for a little while, fercryinoutloud. I think that's what happened to me -- almost as soon as I had some free time, I got a sore throat, body aches, stuffy sinuses, fatigue. It's been OK, though -- I've had a lot of time to sit at my parents' house, fool around on the computer without doing anything serious, and read books. I got a proposal for a new project done, and some API designs, but I also am almost finished with James Fenimore Cooper's The Pathfinder, which has been a real enjoyable book. I picked it up in Lake Placid while we were there at the beginning of the month, and I'm just getting around to finishing it. I guess I'm just getting myself on a Colonial New York kick. I've also watched about 400 episodes of It's a Big, Big World, a great kids show starring a good-natured stoner sloth and some spazzy monkeys. They deal with lots of interesting issues, but mostly the answer to their problems is, "Get over yourself, chill the hell out, and do some tai chi." Which I think is a pretty good piece of advice for everybody. tags: lazy pathfinder james fenimore cooper cold tai chi big big world the wisdom of sloths

Edinburgh So, I just registered for Debconf7, the 7th annual Debian conference, to be held in June 2007 in Edinburgh. I'm excited to go -- I've never been to Scotland, and I'm looking forward to the visit. Nor have I been to a Debconf before! I also submitted an abstract for a paper on the Debian Creative Commons Workgroup, a kind of extended history from the Debian Creative Commons Workgroup Report?. I want to cover the details of the relationship between Debian and CC, the history of the workgroup, and give some lessons learned from the process. I'd also like to draw some conclusions about using one of Debian's greatest assets -- our moral authority in the Free Software world -- to shape the future of FLOSS and Free Content. tags: edinburgh free software debian debconf7 cc free content

Mysteries of Web comics I guess there are just some things I'll never understand about the aesthetics and economics of Web comics. Like, why hasn't Patrick Farley, author of the great e-sheep comics, published anything since 2003? e-sheep is some of the most moving, intelligent, and deep work of any kind on the Internet. I'd really love to see more. Conversely, why did the lurid, absurd, and obnoxious Leisuretown go off the air entirely for a few years? And why did it come back? And if it's back, why hasn't the enigmatic Tristan A. Farnon added any new comics? Also, I still wonder if that's his given name, or if he's stuck with a BBS handle from back in the day when he thought All Creatures Great and Small was really cool. Leisuretown is another of the best things on the Web -- crude, angry, existential. I really like it. tags: webcomics leisuretown e-sheep

In a few days, I hope to launch a little project that I've been working on, that was inspired by some concrete needs at work. That's nothing new, as I've often undertaken new projects after seeing needs out there in the 'real world'. What's going to be different this time, though, is that I'm not going to release it as open source software. Instead, I'm going to make the basic version available for free on the web, and charge money for more advanced versions, in various ways. That's a pretty big change for me, as I'm very dedicated to the idea of open source software. It's simply more fun than the proprietary sort, and I love cooperating with other people on things, and all in all, it's just easier to release stuff that way. However, I want to try something new... I want to see if I can make some money on a few ideas and projects that I have in mind. I've thought about it a lot, and the shortest, most direct feedback loop in the types of end-user applications I have in mind is going the 'commercial' route. If I can make a bit of money on the applications, I can spend a bit more time either improving them or developing new ones. If they were open source, the most I would get is other people contributing a little bit back, but that doesn't matter much, as none of these ideas are all that complex. I'm not giving up on free software... I couldn't. My hope is that by providing myself some extra income, that I will be able to produce some free software as a 'side effect', ala 37 signals or google, or any number of other groups that contribute a lot of 'side dishes' to open source, while saving the main course to make their money with. Hopefully you will see more in this space in the next few days.

This is belated...anyway, I'm taking a day off work to get ready for our vacation this weekend, and I remembered (after writing some other blog posts) that I meant to chime in on a Planet Debian discussion about closed vs not-closed blogs.

My reason for closing comments was quite simple: blogs are spam-magnets. While I was a student and/or unemployed, I didn't mind spending a few minutes a day purging spam messages from my blog. But when I started working full-time, it became a real chore, especially since the amount of spam I got increased with the number of posts on the blog.

One thing that would change my point of view here would be the option to automatically close comments on posts that are more than (say) 60 days old, or perhaps that haven't got a (non-deleted) post newer than 60 days. This would keep the spam problem down, since my biggest problem was ancient blog posts that no-one had commented on (at least for months or years), but that suddenly got picked up by spammers. This might be available if I ran my own blog, but I don't have the time at the moment to administrate a home server, so I'm stuck using someone else's software.

Congrats to Felipe C. for the Bug-not-found bug (aka: the bug with the magic number 404 404) :-) And thanks for the spanish translation.

While its not the normal action people usually expect from me, I just felt the need to have a little posting saying Thanks to somebody. Who? Well. All of you doing work on a regular basis, like * Application Managers, who regularly process people that want to get Debian Developer status. They are doing a lot of work, asking questions, giving tips and in the end submit their report. Usually thats a process of multiple days up to some months work, depending on the time the Applicant and the AM has. * FrontDesk. who makes sure that the process keeps on running by managing all sorts of weird things people come up with, * Sponsors, who help non-DDs to get their package into the Debian Archive, * The release team, who always tries to get the best possible release out of the door, * All those people that make Debian what it is - the biggest and best distribution ever, * everyone that goes and fixes bugs in debian packages to keep the quality as high as possible, * all people that help people by answering questions on one of our mailinglists, on IRC or in web forums, * every upstream author that releases his software under a free license and happily works together with distribution maintainers to make their software the best it can be, * all those that help to organize DebConf, the yearly Debian Conference with all the time-consuming tasks that that includes, * the people that keep the infrastructure running that allows us to develop our own little gems, like the Debian Admins, SPI Admins, Alioth Admins, IRC Network Operators and anyone who provides and maintains machines for a free project out there, * those behind everyone I mentioned - the families. Who have to look for something else during the time people do all the mentioned work, * everyone who can resist feeding a flamewar, Did I forgot someone? I’m sure I did, so - start your own post and say thanks, without people spending a lot of their spare time you would not have such a great distribution to use and ease your life! Have a good new year and continue the good work!

I’ve been asked to keep my blog up to date with regard to Linux support of the MacBook Pro, so here are my latest findings. For more background, see the previous post on the same subject. Anything that I don’t mention here or in another post can be assumed to work (or I just haven’t tested that particular thing yet, though I’m doing some extensive tests).

• Sound: still nothing, none of the various patches floating are good enough
• Battery life: no news here, 1h30 is the max you’ll get. You need to suspend to disk if you want to swap the battery, it’s not possible to do that in suspend to ram like it was on the PowerBook (G4 500, at least). Sad.
• Heat/fan noise: catastrophic. Fans running at 4000-4200 rpm on a typical desktop or on my laps, down to 3000 rpm on a glass desktop. More on the SMC below.
• ATI drivers: still suck. No PowerPLAY support on amd64, which means the GPU is running at the highest clock rate, consuming a lot of power and generating a lot of heat. XVideo makes the X server segfault, watching a short movie using vlc’s OpenGL output managed to crash the box hard, corrupting part of the kernel memory. fglrx kills the machine on resume.

In a previous post, I was reporting a lot of timeouts from the applesmc “driver”. After a (very) quick look at the AppleSMC kext for OS X, I know why this happens. It looks like the OS X driver uses an interrupt and configures the SMC for that. So it probably doesn’t poll the SMC 20 times per second like the applesmc “driver” does. It also looks like the OS X driver uses SMC keys we know nothing about. I could also confirm that the SMC has access to temperature sensors for the CPU, GPU and memory, but it looks like we have no idea of the corresponding SMC keys. With this information, it’d be possible to switch the SMC to manual mode with a userspace governor. What probably happens when we run Linux on this machine is that the SMC stays in automatic mode, and it’s very conservative in this mode. It won’t hesitate to run the fans at high speed to keep the machine safe. It is highly probable that the OS X driver is taking over the fan control to keep the machine both cool and quiet. This is mostly speculation, as the better device power management in OS X plays a role too, and the SMC could very well still be in automatic mode under OS X keeping the fans at low speed just because the machine generates way less heat. Unfortunately, the AppleSMC kext isn’t released as part of Darwin, so we’ll have to wait until it is. Could be a long wait, though. The SMC handles a lot of things. I don’t feel safe running an OS without a proper driver for the SMC on this machine. The fans probably aren’t designed to run at 4000+ rpm for 3 years and will probably fail early or something.

Yay! We finally have snow! I took a picture, but need to charge the batteries in order to get it off of the camera. And for that, I need to find the charger in one of these boxes...

It's an odd feeling, realising that this is the last Christmas that Angie and I get calmly to ourselves. Sprout will be ~9 months old next Christmas, certainly old enough to enjoy shiny light s and music and such, even if (s)he* will be unaware of what's actually happening. We've been making the most of being lazy: Sleeping in, reading books, hacking hppa on Feisty... The types of things that a person can enjoy on a lazy day. =)

Moving has been going really well. While we still have a fair bit to unpack, it's past the overwhelming stage. We spent a bit of time this weekend clearing space for things like cooking and walking. We even managed to get a tree up, although the decorations are quite minimal this year.

We have the last of our moving to do today. A last round of a few plants, and a couple breakables that we didn't want handled by the movers. The new subletters show up tomorrow and then we're basically done with the old place.

Everything so far feels like a great close to 2006.

Chris Samual writes a good summary of Peter Gutmann's analysis of the cost of Vista (in terms of DRM).

The following paragraph in the article however seemed more interesting to me:
Once a weakness is found in a particular driver or device, that driver will have its signature revoked by Microsoft, which means that it will cease to function (details on this are a bit vague here, presumably some minimum functionality like generic 640x480 VGA support will still be available in order for the system to boot). This means that a report of a compromise of a particular driver or device will cause all support for that device worldwide to be turned off until a fix can be found.

Now just imagine that you want to cause widespread disruption - a DOS (Denial Of Service) attack against Windows users. What better option than to cause most of them to have hardware that's not acceptable to the OS? I expect that there will be many instances of security holes in drivers and hardware being concealed by MS because they can't afford the PR problems resulting from making millions of machines cease functioning. But just imagine that someone finds hardware vulnerabilities in a couple of common graphics drivers or pieces of hardware and publicly releases exploits shortly before a major holiday. If it's public enough (EG posted to a few mailing lists and faxed to some major newspapers) then MS would be forced to invoke the DRM measures or lose face in a significant way. Just imagine the results of stopping 1/3 of machines working just before Christmas!

Of course after the first few incidents people will learn. It shouldn't be difficult to configure a firewall to prevent all access to MS servers so that they can't revoke access, after all our PCs are important enough to us that we don't want some jerk in Redmond just turning them off. Of course disabling connections to MS would also disable security updates - but we all know that usability is more important than security to the vast majority of users (witness the number of people who happily keep using a machine that they know to be infected with a virus or trojan).

If this happens and firewalling MS servers becomes a common action, I wonder if MS will attempt the typical malware techniques of using servers in other countries with random port numbers to get past firewalls. Maybe Windows updates could be spread virally between PCs, this method would allow infecting machines that aren't connected to the net via laptops.

Finally, I recommend that people who are interested in such things read Eastern Standard Tribe by Cory Doctorow, he has some interesting ideas about people DOSing corporations that they work for which seem surprisingly similar to what MS is doing to itself. I'll post about my own observations of corporations DOSing themselves in the near future.

I am using Vim a lot to code in Python. One of the most important thing in Python is indent code properly as t uses indent to mark code blocks, and making that right is a bit painful, as you usually end mixing tabstops with white spaces, which makes code almost unreadable when you open it with an editor with a different tabstop setting. To avoid that, using whitespaces is recommended, but it would be nice that those could be inserted automtacally, and that is what the following lines of code added to your ~/.vimrc file make.

" Tip: set indent parameters for python files.
" Version: 0.1
" Date: 13 May 2006
"
" Description: most python scripts use four spaces for indenting, but
"              sometimes you will end up editing a script where tabs
"              are used; in these situations it can be useful to
"              automatically detect whether spaces or tabs were used,
"              and set some parameters (or call some functions) consequently.
"
" Usage: you can put this script in you vimrc and call the PyIndentAutoCfg
"        function with an autocmd associated to python files, or call
"        it manually, or put it in the python.vim syntax script, or... :-)
" Function to set parameters for python scripts that use
" spaces for indention.  This is also the default.  YMMV.
function PySpacesCfg()
  set expandtab " use spaces in place of tabs.
  set tabstop=8 " number of spaces for a tab.
  set softtabstop=4 " number of spaces for a tab in editing operations.
  set shiftwidth=4 " number of spaces for indent (>>, , ...)
  set smarttab
  set autoindent
  set smartindent cinwords=if,elif,else,for,while,try,except,finally,def,class
endfunction
" Function to set parameters for python scripts that use
" tabs for indention.  YMMV.
function PyTabsCfg()
  set noexpandtab
  set tabstop=4
  set softtabstop=4
  set shiftwidth=4
  set autoindent
endfunction
" This function returns 1 if the file looks like a python script
" that uses tabs for indenting, or 0 otherwise.
function PyIsTabIndent()
  let lnum = 1
  let max_lines = 100 " max number of lines to check.
  let got_tabs = 0
  let got_cols = 0 " 1 if the previous lines ended with columns.
  while lnum = max_lines
    let line = getline(lnum)
    let lnum = lnum + 1
    if got_cols == 1
      if line =~ "^\t\t" " at least two tabs, to prevent false-positives.
        let got_tabs = 1
        break
      endif
    endif
    if line =~ ":\s*$"
      let got_cols = 1
    else
      let got_cols = 0
    endif
  endwhile
  return got_tabs
endfunction
" Check the file, and run the relative function.
function PyIndentAutoCfg()
  if PyIsTabIndent() == 1
    call PyTabsCfg()
  else
    call PySpacesCfg()
  endif
endfunction
" Call the PyIndentAutoCfg function.  Uncomment this line if you've copied
" this script in the python.vim syntax file or something like that.
" call PyIndentAutoCfg()
au BufNewFile,BufRead *.py call PyIndentAutoCfg()

Also, if you want to convert a file using tabs to spaces, you can do it by using the following Vim commands.:

:set expandtab
:%retab

Update: Stefano posted about using filetype hook files instead of au groups. I think it is smarter, and I didn't knew about after/ dir in ~/.vim. So following Stefano's advice, you should put the ebove script in ~/.vimrc/after/ftplugin/python.vim, so it gets loaded automatically when you open a Python file.

eBox is growing a lot lately, and updating screenshots is always a cumbersome task, so, prompted by a workmate, I decided to do something to get automatic screenshots. The first step was to get an automatic way to install eBox from scratch to have something to get screenshots from. That was quite easy using the Xen facilities available in Debian. I only had to write a role for xen-create-image and now I can get an eBox up and running in a couple of minutes executing a simple “xen-create-image –role ebox –hostname ebox-screenshots”. Using eBox introspection capabilities and some Perl magic, it was quite easy to autogenerate a list of the URLs that we should take screenshots from. Once we had the URL list and a running eBox all we have to do was to have a browser logging in the eBox web interface and loop through the URLs taking screenshots. To achieve that I resorted to Ruby, Konqueror, KSnapshot and DCOP, and it took less than 40 lines of code to do it. The first problem was spawning Konqueror and KSnapshot instances and getting a DCOP reference for them, but that was easily done using Ruby’s fork, which is quiet convenient:

konqpid = Kernel.fork exec("konqueror --geometry 1024x768+0+0")
ksnappid = Kernel.fork exec("ksnapshot")
konqwin = KDE::DCOPRef.new("konqueror-# konqpid ","konqueror-mainwindow\#1")
ksnapshot = KDE::DCOPRef.new("ksnapshot-# ksnappid ","interface")
The second problem I faced was logging in the eBox interface, I had to enter the password and submit the form. Fortunately Konqueror’s DCOP interface offers a really nice “evalJS” function:

konqwin.openURL("https://# ip /ebox/")
konqhtml = KDE::DCOPRef.new("konqueror-# konqpid ","html-widget1")
konqhtml.evalJS("document.getElementById('credential_0').value = 'ebox'")
konqhtml.evalJS("document.login.submit()")

and we’re in! Now we only have to loop through the urls taking the screenshots. I had a little problem because ksnapshot expects you to click the left mouse button before taking the screenshot. Fortunately that could be easily fixed using xte from the xautomation package to simulate the click.

urls.each url
konqwin.openURL("https://# ip /ebox/# url ")
name = "# konqwin.caption.split(/-/)[1].strip.gsub(/ /,'_') "
ksnapshot.slotGrab
system("xte 'mousemove 100 100' 'mouseclick 1'")
ksnapshot.save("# outpath /# name .png")

The code I pasted lacks a bit of initialization and some sleeps I put there to synchronize the stuff, but basically that’s all. I just love Ruby, KDE and DCOP

When I tried setting a birthday alarm in elric last night, I found out that the year of birth field accepted years of up to 1000. Pity. Anyhow, cheers be to everyone once again; server has rebooted, Jesus has respawned in Bethlehem. Merry Christmas!!!

These are my ten wishes for 2007. Since Santa didn’t arrive in this part of the world, I think I’m asking this to the Three Wise Men (in spite of they were brought here by the Spanish catolicism, but hey):

1. Dignity, land and respect to the Mexican indigenous groups.
2. Return and good health for Commander in Chief, Fidel Castro.
3. Strongest antineoliberalism in the Latin American region.
4. Right wing oppression brought down in Bolivia.
5. Good year socially, economically and politically to Nicaragua with its new hope.
6. An even better year in Cuban trades; they were the best growing economy in Latin America by exporting to South America and China.
7. Stop to the free trade agreement and military bases with the US in Ecuador.
8. Peaceful solution to the Colombia-Ecuador border conflict.
9. Democracy established in some of those countries where it’s more needed, like Central America.
10. More social successful programs in Venezuela.

And my bonus one:

• Peace from R o Grande all the way down to Tierra del Fuego.

A few days ago, I blogged on the popularity of Debian, using Alexa ranking graphs. They showed Debian ahead of the other distributions in terms of web page accesses. Ubuntu definitely was the top growing domain there, but it didn't reach Debian yet, and there was no drop to suggest that Debian was actually losing users to Ubuntu. Here is another chart for you: (Note that it has a referrer block, if you are reading my blog via some planet I'm not aware of, I can add you to the whitelist that is allowed to refer to the image. I added this because some images from my photoblog were heavily linked from myspace and similar sites.) This is a chart from the Debian popularity contest statistics. Submitting data here is entirely voluntary; the data is used to determine which packages to put on which CDs in the CD set. The chart contains four lines, the top line is for bash; pretty much every system has bash, so this is very close to the number of submissions to the popularity contest database. Note that this number has doubled within one year. Sarge was released in June 2005, and thats where the graph changes significantly. Up to then, popularity contest was a rather uncommon package, but sarge promoted its use somewhat IIRC. So lets look only on the post-sarge part of the graph. Ubuntu Dapper was released in June 2006. Any effect? No. The other lines are gnome-session (green), kwin from KDE (blue) and openoffice.org-base (red). KDE is at about 25%, Gnome at about 50% and Openoffice goes up to 50%. This suggests that a significant amount of Debian installations are actually desktop machines (I remove popcon on all my servers, where I'm more concerned with not having too much software on it). For KDE and Gnome, these values have been quite constant. When only counting "active" users, Gnome drops to about 25% (same for OO.o), and KDE to around 10%. VIM would be at 35%, same for apache. I don't claim these numbers are accurate; it's probably more interesting to compare Gnome vs. KDE and similar things. The total number of submissions (represented by bash in this graph) shows that Debian is quite healthy. Oh, and of course I'd like to invite you to install popularity-contest and contribute your software choice to optimizing the CD layout for etch.

A few days ago I installed Debian/Etch on my Thinkpad. One of the reasons for converting from Fedora to Debian is that I need to run Xen and Fedora doesn't support non-PAE machines with Xen. Ironically it's hardware supplied to me by Red Hat (Thinkpad T41p) that is lacks PAE support and forces me to switch to Debian. I thought about just buying a new dual-core 64bit laptop, but that seems a bit extravagant as my current machine works well for everything else.

Feeling adventurous I decided to use the graphical mode of the installer. I found it a little confusing, at each stage you can double-click on an item or click on the continue button to cause the action to be performed. The partitioning section was a little unclear too, but given that it has more features than any other partitioning system I've seen I wasn't too worried (options of creating a degraded RAID array and for inserting a LUKS encryption layer at any level are really nice). The option to take a screen-shot at any time was also a handy feature (I haven't yet inspected the PNG files to see what they look like).

Another nice feature was the way that the GUI restarts after a crash. While it was annoying that the GUI started crashing on me (and would have prevented a less experienced user from completing the install) the fact that it didn't entirely abort meant that I could work around the problem.

I have not yet filed any bug reports against the installer because I have not done a repeatable install (there is a limit to how much testing I will do on my most important machine). In the next few days I plan to do a few tests of the graphical installer on test hardware for the operations that are important to me and file appropriate bug reports. I encourage others to do the same, the graphical mode of the installer and the new encryption and RAID features are significant improvements to Debian and we want them to work well.

I have realised that it won't be possible to get SE Linux as good as I desire before the Etch release, even if the release is delayed again. I'm not sure how many fixes can go in after the release (I hope that we could move to a model similar to RHEL - but doubt that it will happen). So I now plan to maintain my own repository of Etch SE Linux packages and for other packages which need changes to make them work in the best possible manner with SE Linux. I will append something like ".se1" to the version of the packages in question, this means that they will be replaced if a security update is released for the official package. Apart from the SE Linux policy packages (for which any security updates will surely involve me) the changes I am going to make will not be major and will be of less importance than a security update.

I will also add other modified and new packages to my repository that increase the general security of Etch. Apart from SE Linux all the changes I intend to host will be minimal cost issues (IE they won't break things or increase the difficulty of sys-admin tasks), and the SE Linux related changes will not break anything on non-SE systems. So someone who wants general security improvements without using SE Linux might still find my repository useful.

The first step is to configure the CRM114 files, and these are now in pretty good shape as shipped with Debian. All that I needed to do was set a password, say that I'd use openssl base64 -d, and stick with the less verbose defaults (so no munging subjects, no saving all mail, no saving rejects, etc, since I have other mechanisms that do all that). The comments in the configuration files are plenty good enough. This part went off like a breeze; the results can be found here (most of the values are still the upstream default). The next step was to create new, empty .css files. I have noticed that creating more buckets makes crm114 perform better, so I go with larger than norm crm114 .css files. I have no idea if this is the right thing to do, but I make like Nike and just do it. At some point I'll ask on the crm114-general mailing list.

    % cssutil -b -r -S 4194000 spam.css
    % cssutil -b -r -S 4194000 nonspam.css

Now we have a blank slate; at this time the filter knows nothing, and is equally likely to call something Spam or non-Spam. We are now ready to learn. So, I girded my loins, and set about feeding my whole mail corpus to the filter:

 /usr/share/crm114/mailtrainer.crm               \
   --spam=/backup/classify/Done/Spam/           \
   --good=/backup/classify/Done/Ham/            \
   --repeat=100 --streak=35000                  \
         egrep -i '^ + train Excell Running'

And this failed spectacularly (see bug #399306). Faced with unexpected segment violations, and not being proficient in crm114's rather arcane syntax, I was forced to speculation: I assumed (as it turns out, incorrectly) that if if you throw too many changes at the crm114 database, things rapidly escalate out of control. I went o to postulate that as my mail corpus was gathered over a period of errors, the characteristic of Spam drifted over time, and what I consider Spam has also evolved. So, some parts of the early corpus are at variance with the more recent bits. Based on this assumption, I created a wrapper script which did what Clint has called training to exhaustion -- it iterated over the corpus several times, starting with a small and geometrically increasing chunk size. Given the premise I was working under, it does a good job of training crm114 on a localized window of Spam: it feeds chunks of the corpus to the trainer, with each successive chunk overlapping the previous and succeeding chunks, and ensuring that crm114 is happy at any given chunk of the corpus. Then it doubles the chunk size, and tries goes at it again. All very clever, and all pretty wrong. I also created another script to retrain crm114, which was less exhaustive than the previous one, but did incorporate any further drift in the nature of Spam. I no longer use these scripts; but I would like to record them for posterity as an indication of how far one can take an hypothesis. What it did to was have crm114 learn without segfaulting -- and show me that there was a problem in the corpus. I noticed that in some cases the trainer would find a pair of mail messages, classify them wrongly, and retrain and refute -- iteration after iteration, back and forth. I noticed this when I added the egrep filter above, and was not drowning in the needless chatter from the trainer. It turns out, I had very similar emails (sometimes, even the same email) in the Ham and the Spam corpus, and no wonder crm114 was throwing hissy fits. Having small chunks ensured that I had not too many such errors in any chunk;and crm114 did try to forget a mail differently classified in an older chunk and learn whatever this chunk was trying to teach it. The downside was that the count of the differences between ham nd Spam went down, and the similarities increased -- which meant that the filter was not as good at separating ham and Spam as it could have been. So my much vaunted mail corpus was far from clean -- over the years, I had misclassified mails, been wishy-washy and changed my mind about what was and was not Spam. I have a script that uses md5sums to find duplicate files in a directory, and found, to my horror, that there were scores of duplicates in the corpus. After eliminating outright duplicates, I started examining anything that showed up with an ER (error, refute) tag in the trainer output; on the second iteration of the training script these were likely to be misclassification. I spent days examining my corpus and cleaning it out; and was gratified to see the ratio of differences to similarities between ham and Spam css files climb from a shade under 3 to around 7.35. Next post we'll talk about lessons learned about training, and how a nominal work flow of training on errors and training when classifiers disagree can be set up.