This is mostly inspired on this article. The main difference is that I talk about sup instead of mutt, and use SpamAssassin together with bogofilter. Having all these utilities installed and the sources for sup configured, you only need to touch procmail and sup configuration files. My ~/.procmailrc contains a lot of comments, but what it does is basicly the following: If the message is smaller than 250 KiB, it calls SpamAssassin with it. This will create the headers X-Spam-Level and X-Spam-Status in this message. Only messages smaller than 250KiB are considered because most spam isn't bigger than that, and considering big messages would overload SpamAssassin. Then, if SpamAssassin consider this message as spam, tell bogofilter about it. Next, call bogofilter, which will add the X-Bogosity header to the message. Now we have to configure sup. We need to tell it to update bogofilter when we manually mark a message as spam. For this we can use ~/.sup/hooks/mark-as-spam.rb. We must consider two cases: the case where bogofilter thought it was not spam, and the case where bogofilter was not sure about its status. In the first case, we must tell it to forget that the message was not spam and to register it as spam now. In the second, just register it as spam. Also, we don't want spam on inbox. So we can edit ~/.sup/hooks/before-add-message.rb to look for the headers generated by SpamAssassin and Bogofilter and remove the :inbox label. It's important to mark unsure messages with a label so that we don't forget to mark it as spam if it is, and as not spam if it's not. So we need a way to mark messages as not spam. I included a new keybinding in ~/.sup/hooks/startup.rb. Unfortunatelly, I don't know how to include a new keybinding in sup without repeating all the keybindings for the mode I'm dealing with, so that's how I've made it. The command removes the :unsure label and mark it as non-spam. It also sets the undo command, unregistering the message in bogofilter. The case where the message was previously marked as spam, and not unsure, is also handled. I'm sure there's a lot of room for improvement, specially in the Ruby part, but it seems to be working.

The week before DebConf has been traditionally been DebCamp. This has been advertised as a time for core teams to come early to work on their projects together, and this post will talk more about what that means.At DebCamp, there is not much infrastructure set up yet. We have (unfurnished) hacklabs, accommodations, and usually food from the very beginning. We'll have Internet access, but the network and power distribution isn't set up yet. People find what they can and make their own space to work. The DebConf team works on setting up hacklabs starting on day 0 or 1, and the DebCampers serve as the test subjects to make sure that everything works.There intentionally aren't many scheduled activities during DebCamp. The organizers have too much else to do, and events aren't the main purpose of DebCamp. If some people wanted a place to meet, they could probably find a place, though. Some years, there have been one or two workshop type events on the DebCamp schedule.DebCamp has frequently been explained by "Development teams like debian-installer and debian-edu also have the possibility of meeting beforehand for undisturbed collaborative work. ... Interested teams should contact the organizers upfront to make special arrangements." (From the DC10 website) Also included in DebCamp are the DebConf organizers - we desperately need that time to pull organizers from over the world together to get us on the same page locally. Furthermore, anyone without a workplan can attend DebCamp by paying their actual costs.However, saying that DebCamp is limited to these groups of people doesn't reflect reality: Anyone can attend. Thus the DebCamp description doesn't necessarily reflect the actual limits for attending. This has the appearance of resulting in (and some people believe it has) a DebCamp which is biased towards those who well-connected with organizers, since they know that basically anyone can go and get a free extended holiday with Debian people. In fact, getting anyone to review the submitted workplans has met resistance in the last few years.For DebConf10, we did manage to reach agreement to filter out possible tourist-like people, but it was a similar (or only slightly more strict) criteria than DebConf itself. This was done very late, too late to be fair to force adjustments in travel plans of too many people. The difference between the DebCamp listed criteria and who has gotten sponsored for DebCamp in the past caused contention among some new organizers this year. Why didn't they know that they would have qualified for DebCamp past years? The moral of the story is that we need to be more clear on the purpose of DebCamp and make the actual qualifications for attending concrete and well-known so that it remains fair.Of course, it's also true that non-core developers and other interested people to attending DebCamp is a good thing, since it is a strong motivator to get more involved in Debian. Local people are encouraged to attend and learn/hack/volunteer. DebCamp isn't meant to be exclusive, just to reflect the realities of a limited budget.
DebCamp is a very important part of DebConf, both for Debian development purposes and for DebConf set-up purposes. However, since it is a week of limited attendance with mostly the same people attending year after year, we need to ensure that it remains transparent to the rest of Debian.Tags: debconf

There seems to be increasing demand for changing default theme in phpMyAdmin. Well the obvious question is which one to choose :-). There are currently 3 themes compatible with 3.3 branch, what is still far away from current master, however I don't see that they would mean big improvement over current ones. Maybe we should just make some contest for new theme? Anyway if we decided to change the theme, we will still get complains from users that we did the change. At least this is what I remember from last attempt to do so several years ago :-).

In the process of studying for my PhD general examinations this year, I ended up writing summaries about 200 academic books and articles. AcaWiki is a wiki designed to host summaries of academic articles so it seemed like a great place to host these things. Over the last few months, I've uploaded all these summaries. Since I've finished, I've continued to add summaries of other articles as I read them. My summaries tend to be rough. I write them, run them through a spellchecker, and then post them. I don't even reread them before publishing. I hope to improve them as I reread them over time. Of course, because I've uploaded them to wiki, I hope others will add to and improve the summaries as well. AcaWiki uses Semantic Mediawiki and provides nice platform for publishing, editing, and collaboration. Although there are still ways in which the platform can be better, what is needed now is, quite simply, more contributors. I am sad to see that my summaries make up a big chunk of all summaries on the site. So if you are a student, an academic, or anyone else who writes or has written summaries of articles or books or if you might want to do so, you should consider contributing your summaries, in whatever form, to AcaWiki. I've done a little work to help integrate AcaWiki and Zotero which might make things easier. Doctoral students reading for qualifying or general examinations in particular should should consider taking notes and studying with AcaWiki. From the student's perspective, writing summaries can be one of the best way to reflect on and learn a literature. In the process, one can create a great resource for the rest of the world. If a single doctoral student from each of twenty diverse fields of study published summaries of the 200 key articles in their area, AcaWiki would have the critical core of what is most relevant in academia. Help us build it!

deb http://www.coker.com.au squeeze selinux I have an Apt repository for Squeeze SE Linux packages at the above URL. Currently it contains a modified version of ffmpeg that doesn t need execmod access on i386 and fixes the labeling of /dev/xen on systems that use devtmpfs as reported in bug #597403. I will keep updating this repository for any SE Linux related bugs that won t get fixed in Squeeze. Is there any interest in architectures other than i386 and AMD64?

After being completed in German, Spanish, Swedish, dpkg manpages (including the horrific dpkg-source(1) manpage) are fully translated in French. 1937 paragraphs of geek jargon now using the beautiful langue de Moli re (however, I used the 21st century variant and not 17th century). You can thank my daughter for living in Bordeaux and us for visiting her this week-end, indeed. According to the statistics page, we still have some work to do, and 100% will be a no-go for squeeze but I think that the French l10n team can be proud anyway.

Rant follows. I have upgraded my PC to squeeze this weekend. While this generally worked fine, udev (version 160-1) made my day. As usual I had not read the release notes previously and was hit by

Please upgrade your kernel before or while upgrading udev.

AT YOUR OWN RISK, you can force the installation of this version of udev WHICH DOES NOT WORK WITH YOUR RUNNING KERNEL AND WILL BREAK YOUR SYSTEM

dpkg to the rescue:

dpkg -i /var/cache/apt/archives/linux-base_2.6.32-21_all.deb 
dpkg -i /var/cache/apt/archives/linux-image-2.6.32-5-amd64_2.6.32-21_i386.deb  /var/cache/apt/archives/linux-image-2.6-amd64_2.6.32+28_i386.deb 
reboot

Afterwards the upgrade succeded. (For reference: The correct solution would probably have been to run apt-get install linux-image-2.6-amd64 and rebooting, before starting the dist-upgrade.) udev was not finished with me, though, the syntax for udev rules has changed again. ("NAME="%k" is ignored, because it breaks kernel supplied names", "SYSFS = will be removed in a future udev version, please use ATTR =", "BUS= will be removed in a future udev version, please use SUBSYSTEM=", etc.) So I started googling and rewriting my custom rules for USB-sticks and USB cardreader. After many reboots (the only way to test these rules nowadays), thing were still not working. My cardreader is not an exotic but a common one, i.e. it does not report media changes. (The guide for writing udev rules shares this opinion.) Previously I had:

BUS=="scsi", KERNEL=="sd*", SYSFS model =="Compact Flash",SYSFS vendor =="Generic-",NAME all_partitions ="cardreader/cf"

which caused udev to generate /dev/cardreader/cf1, /dev/cardreader/cf2, etc. on bootup (without media present), enabling me to configure /etc/fstab to allow mounting of /dev/cardreader/cf1 without superuser privileges. After googling I ended up with this new rule:

SUBSYSTEMS=="scsi", KERNEL=="sd*", DRIVERS=="sd", ATTRS vendor =="Generic-", ATTRS model =="SD/MMC          ", SYMLINK+="cardreader/sd%n", OPTIONS+="all_partitions"

and while I get ...

argenau:~# ls -l /dev/cardreader/sd
lrwxrwxrwx 1 root root 6 Sep 19 14:42 /dev/cardreader/sd -> ../sdd

... there is still no /dev/cardreader/sd1 (unless I insert a card and run fdisk -l /dev/cardreader/sd as superuser). Looks like OPTIONS+="all_partitions" is ignored. And indeed I find this in udev's changelog:

udev 152
[...]
The option "all_partitions" was removed from udev. This should not be needed for usual hardware. Udev can not safely make assumptions about non-existing partition major/minor numbers, and therefore no longer provide this unreliable and unsafe option.

Joy. Thanks udev for telling me consumer devices have been fixed overnight, USB cardreaders have suddenly started reporting media change events.

Hello fellow developers, The summer is over but I m happy to announce that this year s Summer of Code at Debian has been better than ever! This is indeed the 4th time we had the privilege of participating in the Google Summer of Code and each year has been a little different. This year, 8 of our 10 students succeeded in our (very strict!) final evaluations, but we have reasons to believe that they will translate into more long-term developers than ever, all thank to you. The highlight this year has been getting almost all of our students at DebConf10. Thanks again this year to generous Travel Grants from the Google Open Source Team, we managed to fly in 7 of our students (up from 3!). You certainly saw them, presenting during DebianDay, hacking on the grass of Columbia, hacking^Wcheering our Debian Project Leader throwing the inaugural pitch of a professional baseball game or hacking^Wsun-tanning on the tr s kitsch Coney Island beach. Before I give the keyboard to our Students, I d like to tell you that it will be the pleasure and honor of Obey Arthur Liu (yours truly, as Administrator) and Bastian Venthur (as Mentor) to represent Debian at the Summer of Code 2010 Mentors Summit on 23-24 October 2010, at the Google Headquarters in Mountain View. Like last year, we expect many other DDs to be present under other hats. We will be having 2 days of unconference on GSoC and free software related topics. We all look forward to reporting from California on Planet and soc-coordination@l.a.d.o! All of our students had a wonderful experience, even if they couldn t come to DebConf, that is best shared in their own voice, so without further ado, our successful projects: Multi-Arch support in APT by David Kalnischkies, mentored by Michael Vogt apt-get install MultiArch does mostly work now as most code is already merged in squeeze, but if not complain about us at deity@l.d.o! Still, a lot left on the todo list not only in APT so let us all add MultiArch again to the Release Goals and work hard on squeezing it into wheezy. Debbugs Bug Reporting and Manipulation API by David Wendt Jr., mentored by Bastian Venthur Hello, I m David Wendt, and I went to Debconf10 to learn more about the development side of Debian. Having used it since the 9th grade, I ve been intimately familiar with many of Debian s internals. However, I wanted to see the developers and other Debian users. At DebConf, I was able to see a variety of talks from Debian and Ubuntu developers. I also got to meet with my mentor as well as the maintainer of Debbugs. Content-aware Config Files Upgrading by Krzysztof Tyszecki, mentored by Dominique Dumont Config::Model is now capable of manipulating files using shorter and easier to write models. Thanks to that, packagers may start experiment with creating upgrade models. Further work is needed to support more complicated config files Dominique Dumont is working on DEP-5 parser, I ll shortly start working on a cupsd config file parser.
The best thing about DebConf10 is that every person I talked with knew what I was doing. I had a mission to get some feedback on my project. Everybody liked the idea of making upgrades less cumbersome. On the other side, it was my first visit to United States, so I decided to go on a daytrip on my own (instead of staying inside the building, despite heat warnings). I had a chance to visit many interesting places like Ground Zero, the UN headquarters, Grand Central Terminal, Times square and Rockefeller Center that was a great experience. Hurd port and de-Linux-ization of Debian-Installer by J r mie Koenig, mentored by Samuel Thibault Debconf10 was great! Among other people working on the installer, I met Aur lien Jarno from the Debian/kFreeBSD team and we worked together on a cross-platform busybox package. Besides, the talks were very interesting and I ve filled my TODO-list for the year.
For instance I learned about the Jigsaw project of OpenJDK, and how Debian would be the ideal platform to experiment with it. More generally, some people think Debian could push Java 7 forward and I d like to see this happen. Smart Upload Server for FTP Master by Petr Jasek, mentored by Joerg Jaspert I must say that it was great time for me in NY, I ve met and talked and coded with people from ftp-master team like Torsten Werner who helped me to push the project a bit further and with some other people who were looking forward to release of the tool which I hope they will use quite soon. Everybody interested, everybody excited, really cool place and time. And I can t forget the Coney Island beach and stuff, lot of fun, lot of sun;) Aptitude Qt by Piotr Galiszewski, mentored by Sune Vuorela Currently, development branches support full features searching, viewing extended package s informations, performing cache and packages operations. Code and GUI still require a lot of work which will be continued. Informations about further progress could be found on aptitude mailing list and repository rss channel. Debian-Installer on Neo FreeRunner and Handheld Devices by Thibaut Girka, mentored by Gaudenz Steinlin For me, DebConf 10 started at the airport, where Sylvestre Ledru (whom I didn t know of before) was wearing a GSoC 2007 t-shirt, that is, given the circumstances, almost equivalent to say I m a hacker, I m going to DebConf 10 .
I ve spent my time at the conference attending various talks, hacking, meeting DDs and other hackers (amongst others, my co-mentor Per Andersson, Paul Wise, Julien Cristau, Christian Perrier, Cyril Brulebois, Martin Michlmayr, Colin Watson and Otavio Salvadores who I have to thank for his patience while dealing with my questions), chatting, cross-signing keys, rushing to finish eating before 7pm, getting sunburnt, sightseeing (thanks, Arthur, for the lightning-fast tour of Manhattan!), and so on. Debian Developers and community, we count on you. See you next year! (cross-posted to debian-devel-announce@l.d.o and soc-coordination@l.a.d.o)

Wanted to post this yesterday, but somehow didnt work out, so here we go now Another meeting day passed, another lot of commits into our repository, another day without entirely breaking the system. But wait, one more chance to break it today, maybe we finally manage. :) Yesterday we got volatile sorted out for Squeeze (Lenny stays like it is, seperate). Its now a suite on ftp-master, and we have all the neccessary information and workflow around it defined. At least so far that we can operate the suite, we don t define what gets into it, we just take that from the volatile team (or, I think, in future, from the SRMs) in a kind of textfile similar to how testing works. Besides having Alex unbreak our bts categorize script, Mark fixing up our BYHAND processing and implementing the volatile stuff I mentioned above, we also merged a patchset from Luca that allows us to provide the packages.debian.org service with a tree containing changelogs, copyright files, NEWS.Debian and README.Debian. Metadata basically. Which they extracted in the past, and that was taking up quite a lot of resources. Much easier if we provide it and they just use it. This is now done for both, the main archive as well as the backports archive. As soon as I upgrade the security archive it will also be done there, which should additionally enable packages.debian.org to display those files for all archives (run by FTPMaster). This update is scheduled for the upcoming week, DSA already gave me the needed bits and pieces on a host so I can do it without breaking security during the process. We currently export this metadata set only to the packages.debian.org host. We do want to provide it to more, but I am waiting for a reply from DSA as to which machine I should use for the open access to those files. When I have that reply it should be available for you all. As a sidenote from that: If you need to compute any data that has to do with the Debian archive (information out of source/binary packages or something around it) and if you do that with your own scripts - you are most likely doing it wrong. Talk to us. We might be able to help you extracting the data you need and keep it updated. Recent examples for that are the above changelogs, but also things like the package-file-mapping snapshot uses to easily determine which file belongs to which source/version, but also information like the Maintainers index, the Uploaders index and whatnot. The simple way is - ask us. We are processing a shitload of data, it might not be too complicated to let your set fall out somewhere at the side, but if you never ask you will never find out Today we start working on some internals of dak and will have some more discussion on agenda points like data.debian.org and the git source format. We already had some words on other points, but those are ones which will take a long time more to fix, like generally reworking the internals of our database to enable us having multiple archives with pretty different access levels inside one machine. (Think of security archive on ftpmaster host. You want to ensure embargoed security issues actually stay embargoed until they are released. Means a good set of neccessary changes inside the database to handle this. We have ideas, but getting them implemented will take a bit).

---

And of course we had been out for food too, helps to stay alive. Turkish restaurant. All nice and well - until they told us they have NO BAKLAVA. Args. Suckers. They ruined this year! WAAAAAAAAAAAAAAAAH. I wish them a windows OS for their live! If not worse! (Can it be worse than windows?) ;-)

Given the impending deprecation of vservers, I've decided to make the switch to KVM on my laptop. Although lxc is a closer approximation to vservers, I decided to go with KVM due to it's support in Virtual Machine Manager. My first step was to confirm that my CPU would support kvm:

egrep -o "svm vmx" /proc/cpuinfo

If that command outputs either svm or vmx (depending on whether you have Intel or AMD hardware) then your CPU supports virtualization. I'm working on a host machine called chicken, which has a logical volume called vg_chicken0. All vservers on chicken operate on a root filesystem that is backed by their own logical volume. In this post, I'll describe the steps to convert the vserver hobo (which operates on a filesystem mounted on the host in /var/lib/vservers/hobo and is backed by the logical volume called vg_chicken0-hobo_root). Both chicken and hobo are running debian squeeze. vservers don't have a kernel installed or grub. KVM virtual servers need both. I was hoping I could simply enter the vserver, install both a kernal and grub and be ready to go. However, grub installation will fail miserably because grub can't figure out how to install on the underlying disk (which is hidden from the vserver). Next, I tried launching a kvm instance, passing a debirf generated ISO with the -c (cdrom) option. However, grub recognized that it was being installed onto a device that did not have a partition table (the logical volume was directly formatted with a file system). So, since I had disk space to spare, I created a new logical volume:

lvcreate --size 5GB --name hobo_root_new vg_chicken0

I then added a gpt partition table (why not prepare for the coming 2TB disks?) and created two partitions. One partition for grub2 and one for everything else:

parted /dev/mapper/vg_chicken0-hobo_root_new mklabel gpt
parted /dev/mapper/vg_chicken0-hobo_root_new unit s mkpart biosboot 2048 4095 
parted /dev/mapper/vg_chicken0-hobo_root_new set 1 bios_grub on 
parted /dev/mapper/vg_chicken0-hobo_root_new unit s mkpart primary 4096

When prompted for the end of the last partition, choose: -1 and accept the adjustment. I had to eyeball cat /proc/partitions to figure out which dm device was the second partition (dm-19). I then created a file system:

mkfs -t ext3 /dev/dm-19

Mounted it:

mount /dev/dm-19 /mnt

And rsync'ed the data:

rsync -a /var/lib/vservers/hobo/ /mnt/

With the data in place, I chroot'ed and installed the packages I needed. When prompted, I chose not to install grub to the disk, because I wanted to wait until I had an environment in which the proper disk would be available to grub as it will when the virtual server boots (see below):

chroot /mnt
mount /proc
aptitude install linux-image-2.6-amd64 grub2
umount /proc
exit

Then, I cleaned up:

umount /mnt
umount /var/lib/vservers/hobo
lvremove vg_chicken0/hobo_root
lvrename vg_chicken0/hobo_root_new hobo_root
dmsetup remove /dev/mapper/vg_chicken0-hobo_root_newp1
dmsetup remove /dev/mapper/vg_chicken0-hobo_root_newp2
kpartx -d /dev/mapper/vg_chicken0-hobo_root

And I removed it from /etc/fstab. Next, I created a new kvm virtual server, using the disk /dev/mapper/vg_chicken0-hobo_root and passing a debirf cd image with -c: virt-install --name hobo --ram 512 --disk /dev/mapper/vgchicken0-hoboroot -c /usr/local/share/debian/ISOs/debirf-rescuesqueeze2.6.32-5-vserver-amd64.iso After logging in, I installed grub2 (aptitude update; aptitude install grub2) and then I installed grub:

mount /dev/sda2 /mnt/
grub-install --no-floppy --root-directory=/mnt /dev/sda

After running grub-install, edit /mnt/boot/grub/device.map so it reads:

(hd0) /dev/sda

Then, rerun grub-install command. I tried generating the grub.cfg file, but got an error message indicating that grub-probe would not detect the device providing / (because I was running on a ram file system from debirf). I added the following to /mnt/etc/fstab:

/dev/sda2  /     ext3 errors=remount-ro 0 1
proc       /proc proc defaults          0 0

And then re-generate the initrd image:

chroot /mnt
mount /proc
update-initramfs -u

So, I rebooted the virtual machine by typing:

exit
reboot

This dropped me into a grub shell. I manually typed:

root (hd0,gpt2)
linux /vmlinuz root=/dev/sda2 ro
initrd /initrd.img
boot

Once booted, I logged in a completed the task with:

update-grub

I ve been sitting on this one since more than a month. I ve contacted upstream on the 19.08.2010 and the Debian maintainer on the 02.09.2010. No reaction from them till today, and no, my spamfilter does not eat mail ;) Still, I won t tell you the name of the software (but you could easily guess or check ) So let s put the grey hat on and begin ;) Foreword: when I write spam-filter, I mean some DNSBL/SPF/blah filter, not SpamAssassin, crm114 or other content-based filters. common spam-filters There are plenty of spam-filters out there which work after the following schema:

for check in checks:
    if check(mail) == BAD:
        reject(mail)
        break

And the checks often look like this:

def check(mail):
    errors = False
    result = some_magic(mail)
    if not errors: return result
    else: return GOOD

Here some_magic is a function that do the actual checks (DNSBL lookups etc) and which stores possible happened errors in errors. If an error occurred, it s safer to say the mail was good (or at least not bad) than bad. the problem This looks like a sane approach on the first sight: iterate over all checks and execute them until a mail is found to be spam (and the corresponding check did not end up in an error (imagine some weird DNS error for DNSBL or SPF checks here)). But then the author realizes that the errors might be global , like with DNS: when the first check notices that it can t resolve anything (ie. it ran into a DNS timeout three times in a row), the second check (which also utilizes DNS queries) will most probably result in an error too. After this he adds a global counter for DNS errors and skips all the checks that use DNS if this counter is >=3. Great idea , you say, saving resources is good . Bad idea , I say The reason is simple: DNSError != DNSError. Why? The local resolver might be broken, then every DNS query will end up with an error and DNS-based checks should be really disabled. The first three (out of eg. ten) DNSBLs might be down, but that s not a reason to skip the other seven and the SPF check. Etc Now let s assume all configured DNSBLs are working properly, so is the local resolver and the checks are:

checks = [check_HELO, check_SPF, check_DNSBL]

How could one attack this filter? the attack Set up the following zone

spam.example.com IN NS ns1.example.com
spam.example.com IN NS ns2.example.com
spam.example.com IN NS ns3.example.com
spam2.example.com IN MX mx1.spam.example.com
spam2.example.com IN MX mx2.spam.example.com
spam2.example.com IN MX mx3.spam.example.com
spam2.example.com IN MX mx4.spam.example.com
ns1.example.com IN A 10.1.1.1
ns2.example.com IN A 10.2.2.2
ns3.example.com IN A 10.3.3.3

and greet the to-be-spammed-MTA with EHLO client1.spam.example.com
What happens? The spam-filter tries to resolve client1.spam.example.com in check_HELO to check whether the hostname matches the IP-address the connection is coming from, fails three times (there are no reachable DNS servers in 10.0.0.0/8) and continues with the next two checks, but these are skipped because of previous DNS errors. As no checks could identify the mail as being spam, the mail is delivered to the users mailbox. With some creativity this can also be used in SPF records (via spam2.example.com), sender-domains etc. conclusion If you want to save resources by skipping DNS-based checks after another (DNS-based) check failed don t! Think whether this is really the same error you are seeing Additionally this can be used as a sort of DoS against the not said software: according to the logs each of my tries needed between 30 and 36 seconds (!) to complete imagine some hundred simultaneous requests the end You think I did something wrong with that post? Throw the first stone then, I don t care.

So a while back I blogged about maintainable test suites. One of the things I ve been doing since is fiddling with the heart of the fixtures concept. To refresh your memory, I m defining fixture as some basic state you want to reach as part of doing a test. For instance, when you ve mocked out 2 system calls in preparation for some test code that represent a state you want to reach. When you ve loaded sample data into a database before running the actual code you want to make assertions about that also represents a state you want to reach. So does simply combining three or four objects so you can run some code. Now, there are existing frameworks in python for this sort of thing. testresources and testscenarios both go some way towards this (and I and to blame for them ), so does the zope testrunner with layers, and the testfixtures project has some lovely stuff as well. And this is without even mentioning py.test! There are a few things that you need from the point of view of running a test and establishing this state:

• You need to be able to describe the state (e.g. using python code) that you wish to achieve.
• The test framework needs to be able to put that state into place when running the test. (And not before because that might interfere with other tests)
• And the state needs to be able to be cleaned up.

Large test suites or test suites dealing with various sorts of external facilities will also often want to optimise this process and put the same state into place for many tests. The (and I m not exaggerating) terrible setUpClass and setUpModule and other similar helpers are often abused for this. Why are they terrible? They are terrible because they are fragile; there is no (defined in the contract) way to check that the state is valid for the next test, and its common to see false passes and false failures in tests using setUpClass and similar. So we also need some way to reuse such expensive things while still having a way to check that test isolation hasn t been compromised. Having looked around, I ve come to the conclusion we ll all benefit if there is a single core protocol for doing these things, something that can be used and built on in many different ways for many different purposes. There was nothing (that I found) that actually met all these requires and was also tasteful enough that folk might really like using it. I give you fixtures . Or on Launchpad. This small API is intended to be a common contract that all sorts of different higher level test libraries can build on. As such it has little to no policy or syntatic sugar. It does have a nice core, integration with pyunit.TestCase, and I m going to add a library of useful generic fixtures (like temporary directories, environment isolators and so on) to it. I d be delighted to add more committers to the project, and intend to have it be both Python 2.x and 3.x compatible (if its not already my CI machine isn t back online after the move yet, I m short of round tuits). Now, if you re writing some code like:

class MyTest(TestCase):
    def setUp(self):
        foo = Foo()
        bar = Bar()
        self.quux = Quux(Foo(), Bar())
        self.addCleanup(self.quux.done)

You can make it reusable across your code base simply by moving it into a fixture like this:

class QuuxFixture(fixtures.Fixture):
    def setUp(self):
        foo = Foo()
        bar = Bar()
        self.quux = Quux(Foo(), Bar())
        self.addCleanup(self.quux.done)
class MyTest(TestCase, fixtures.TestWithFixtures):
    def setUp(self):
        self.useFixture(QuuxFixture)

I do hope that the major frameworks (nose, py.test, unittest2, twisted) will include the useFixture glue themselves shortly; I will offer it as a patch to the code after giving it some time to settle. Further possibilities include declared fixtures for tests, and we should be able to make setUpClass better by letting fixtures installed during it get reset between tests.

Since last year, I've been slowly working on introducing a new 4096-bit RSA key (complete overkill, I know) with all the modern hash preferences and so forth so that I can retire my old 1024-bit DSA key. After DebConf in New York City at the beginning of August, I now have enough signatures on the new key to finish this transition. If you have signed my old key and are willing to sign the new key on the strength of a key transition document, here is key transition document signed with both the old and the new keys. While my new key has a reasonable number of signatures, I'm always happy to get more. My old key is 0x0AFC7476, and my new key is 0x82004173. You can retrieve the new key (and the old key, for that matter) from any public keyserver or from my web site (new key and old key). Shortly after posting this, I'll submit the RT ticket to transition my key in the Debian keyring. The new key has a 2048-bit signing RSA subkey which I may export to a few places other than my primary secure system so that I can do package uploads in some more convenient ways. The new primary key has a three year expiration period (which will be periodically extended), and the signing subkey has a one year expiration period (likewise).

There are corners of the Internest where foolish people congregate, and invent stories. These foolish stories are then read as gospel by trusting people, and reposted, until the original made-up source is concealed from view. As an attempt to stem this flow of disinformation, here are some commonly held but incorrect beliefs about the Mono framework, and an explanation of the reality of the situation, as far as I understand it. The next Mono version is co-developed with Microsoft There is a grain of truth behind this one, but it s a gross mischaracterisation. Mono 2.8, when it ships, will bundle, for convenience, a number of Free Software libraries, which are released by Microsoft under a license considered Free Software by the Free Software Foundation, the Ms-PL. These are:

• System.Web.Mvc and System.Web.Mvc2: ASP.NET MVC, a library for writing ASP.NET web pages with a model-view-controller design, similar to Rails for Ruby. This is not news System.Web.Mvc has been bundled in Mono since Mono 2.4.2 (June 25th 2009). System.Web.Mvc2 has been bundled in Mono since Mono 2.6.7 (July 14th 2010).
• System.Numerics: Mono re-uses Microsoft s implementation of BigInteger.cs from the Dynamic Language Runtime (see below). This is indeed new to Mono 2.8, as it is part of .NET 4.0 (Mono 2.8 targets .NET 4.0 compatibility by default).
• System.Data.Services.Client: The OData SDK client library. This is indeed new to Mono 2.8.
• System.ComponentModel.Composition: The Managed Extensibility Framework, a library for writing and using plugins. This is similar in scope to the existing Mono.Addins project. This is indeed new to Mono 2.8, although it was added to Mono Trunk in 2009.
• System.Web.Extensions: Microsoft Ajax, a library for using Ajax inside ASP.NET projects. This is not news MicrosoftAjaxLibrary has been bundled in Mono since Mono 1.2.6 (11th December 2007).
• Microsoft.Scripting.Core and Microsoft.Dynamic: The Dynamic Language Runtime is a framework permitting implementation of dynamic languages (such as Python or Ruby) on top of the core .NET runtime. It is used for IronPython and IronRuby, amongst others. The DLR has very recently changed license from Ms-PL to Apache 2.0, but the version bundled in Mono is still under the older license. This is not news the DLR has been bundled in Mono since Mono 2.6 (14th December 2009).

So, to summarise, there are five Free Software libraries written by Microsoft under the Ms-PL included in Mono 2.8 but only two of those five are new, and none of them were co-developed in any meaningful sense. Mono development is dead There have been reports that Mono development has ended, on the basis that no commits have been made to Novell s Subversion server for a few weeks. However, these reports miss one minor detail Mono has moved to Github.com. There were 35 commits from 9 different people to the main mono.git repository (of dozens of repositories under the main Mono project) in the last 2 days, at time of writing far from dead. Mono lacks features found in Microsoft.NET Mono lacks libraries found in Microsoft.NET, such as the Windows Presentation Framework GUI toolkit. In terms of features, i.e. things implemented at a compiler or runtime level, Mono is typically more advanced. This is helped in no small part by Mono s Free Software development model, allowing experimentation and what if changes to the core runtime which a sluggish corporate behemoth like Microsoft cannot accommodate. To give three real-world examples, Mono allows embedding of its compiler as a service, and provides a REPL shell this is a planned feature for .NET 5.0, but has been available for years in Mono; Mono.Simd provides a number of data structures which will run on any version of Mono or Microsoft.NET, but will use optimized CPU extensions like SSE when run on a sufficiently new version of Mono, on an appropriate architecture as far as I m aware, there is nothing like this available or planned for Microsoft.NET. Mono is able to produce fully compiled, static executables which do not need to JIT anything at runtime this is used for iPhone compilation, for example, where JITters are not permitted. There is no comparable feature in Microsoft.NET. Clearly, Microsoft.NET can only be thought of as more featureful if one defines features in terms of does it have lots of libraries? in terms of functionality, Mono is ahead. Mono can sneak onto your system without your knowledge If you don t have the mono-runtime package installed, you can t run Mono apps. It is possible to install some Mono apps alongside the awful-yet-popular mononono equivs package, since the popular equivs script fails to place Conflicts on the correct packages (F-Spot will be blocked, Tomboy will not). No package in Debian or Ubuntu embeds its own copy of the Mono runtime, and we have no plans to make any changes to packaging which would allow execution of any C# application without mono-runtime installed. If one is using a different OS, then things may be different e.g. The Sims 3 for Mac & PC uses embedded Mono, which you wouldn t know about without looking. Canonical are pursuing a pro-Mono agenda, and are responsible for it being pushed in Debian Mono development has been happening in Debian for longer than Canonical has existed the first upload was made in April 2002. Ubuntu is made primarily from software already available in Debian deemed of sufficient quality, and when F-Spot and Tomboy became parts of the default Ubuntu desktop system in 2006, both pieces of software were already available in Debian and deemed of sufficient quality. Nobody working on Mono in Debian is paid by Canonical actually, that s not entirely true, three packages related to accessibility are officially maintained by someone who was hired by Canonical after doing the initial packaging work when he worked for Novell. But the Mono runtime itself, Canonical have no influence over its direction in Debian. As for a pro-Mono agenda , they ve always taken an extremely pragmatic approach to language choice, never showing any real preference for one language or another when it comes to app selection. They don t exhibit any overt anti-Mono policies, which is not the same thing. The names of people contributing to Mono in Debian are not secret check the pkg-mono team page on Alioth. The System.Windows.Forms namespace is protected by Microsoft patents The truth is, nobody knows for sure if SWF, or any other part of Mono, or any other framework such as Vala or Python, is covered by Microsoft patents. The way the US patent system is contrived, it is actively dangerous to check whether something contains any patents when you write it, as you are liable for triple damages should it later emerge that there WAS a patent, even if your searching missed the fact. You cannot patent an API or namespace only a specific implementation of a software concept, in those countries where software patents are permitted. There has never been any evidence shown that Mono s implementation of SWF, or indeed any part of Mono, infringes any Microsoft-held patents, because if that were the case, then the code would be rewritten to avoid the issue much like the approach taken by Linux kernel developers when a patent becomes apparent. The belief within the Mono community is that the core parts of Mono as defined in ECMA334/335 are safe (they are covered by the Microsoft Community Promise patent grant); any of the Ms-PL libraries mentioned above is definitely safe (Ms-PL includes an explicit patent grant); and the rest of the package is likely safe too (on the basis that it is a named component of the Open Innovation Network s list of protected software and on the basis that there s unlikely to be anything patentable in one of many implementations of basic ideas like database connectors). Nobody knows for sure, because that s how the system works. But, again, nobody knows for sure that Microsoft patents don t apply to other frameworks such as Python there is simply a belief and an assumption that they do not. MonoDevelop contributors removed GPL code from MonoDevelop, in an attack on the GPL This is somewhat disingenuous, given MonoDevelop is LGPL. The MonoDevelop team excised the remaining GPL code (and there wasn t much of it) in order to grant greater Freedom to developers. Previously, the entire MonoDevelop IDE was a GPL combination, meaning any add-ins for the IDE also needed to be GPL, regardless of developer choice. Now, any developer can write an add-in for MonoDevelop, using the license of their choice, whether another Free license like the Mozilla Public License, or a proprietary closed-source add-in. They are still welcome to produce GPL add-ins, if they want to, as well. Mono won t run random Microsoft.NET apps anyway, so isn t really cross-platform Actually, this one is often true. When developers write apps for Windows primarily, they rarely take the time to think is this the correct way to do it? and will often plough on with an assumption about the Windows way of doing things. It might be simple things like filesystem assumptions (assuming a case-insensitive filesystem, or assuming a backslash is used to separate directories, not a forward slash). It might be more involved, such as using P/Invokes into Windows-specific C libraries, when a more cross-platform alternative is possible. So, often, random applications for Microsoft.NET won t run on Mono. The reverse is also true F-Spot or GNOME Do are fairly heavily tied to Linux (or to UNIX-like OSes with X11, at any rate), through the libraries they invoke being fairly platform-specific. You can write platform-specific Java, with one quick piece of JNI, too. It should be noted, however, that Mono makes the chance of .NET applications being ported to Linux (and/or Mac) much more likely, since even in the worst case scenario, a company only needs to fix a portion of their source to make it cross-platform. The Mono team have a tool called MoMA, which will scan an application and its libraries, and give you detailed reports on the app s portability. This info is used at both ends by app vendors who want to become more portable, and by the Mono team who want to fill in the most frequently encountered blanks. And, it should be stressed, writing cross-platform apps is entirely possible if one desires it e.g. the IRC client Smuxi is pure cross-platform C#, and the executables compiled on Mono on Linux will run fine on Microsoft.NET on Windows. Portability in this direction is important too consider how many people have been introduced to Free Software thanks to availability of Free apps on Windows like OpenOffice.org, Firefox, Pidgin or GIMP. You can download Tomboy for Windows, and work is ongoing on fixing Banshee portability issues (which are mostly caused by gStreamer). Mono on Android is a terrible idea which will be super slow There are two efforts to enable developers to write Android applications with Mono a paid proprietary product from Novell called MonoDroid, and an untitled porting effort by big-name Android hacker Koushik Dutta. I have no insight into MonoDroid since it hasn t been released yet, but Koush did some benchmarks of his work which exhibited some remarkable figures compared to Dalvik (and even compared to Sun Java for ARM). Those numbers haven t been updated to reflect the Froyo Dalvik JITter, but Mono on Android is still very exciting from a performance perspective. Mono isn t really Free Software The source code is all available under Free Software licenses. So, um, yes it is. We don t need Mono because we have $LANG By the same logic, we don t need $LANG because we have Mono. By all means, use the language of your choice other developers will use the language of their choice too. If you want to use Vala or Python or Java, then by all means, go ahead. It doesn t mean there s no room for more languages, better suited to other usage patterns. Haskell is great for some types of development, so is Fortran, and so is C#. You might not need Mono because of $LANG, but there are others in the world with different needs. Mono apps are all slow, and crash your computer, and stuff No userland app can crash your Linux system, unless there s a bug in the kernel, or you have some severe problems with your hardware. If you ve ever observed a crash as a result of running a Mono app, then it s really coincidence that Mono is tickling whichever part of your system is busted. As for slow startup time may well be slower than for apps written in C or Vala. There s a delay due to the JITter needing to compile all the libraries used by the application (we may AOT the most common libraries in a future Mono package version, at upstream s recommendation). Once an app is running, it should be fast compared to a Python app, and memory-light compared to a Java app. The new garbage collector in Mono 2.8 should also offer significant improvements to performance, especially under heavy workloads. Ubuntu s default GTK+ themes require Mono This is my favourite recent nonsense to emerge from the Internest. There are reports mostly restricted to IRC and blog comments, that (paraphrasing here) removing Mono removes the Ubuntu themes . Here s the reality: in Ubuntu 10.04, a new visual style was implemented throughout the distro. As part of this, small icons (e.g. notification area icons) were set to be black-and-white by default, and colourised when attention is needed (e.g. the power-off icon turns red when a reboot is needed, the messaging indicator turns green when there s a new message). All of these new monochrome icons are in a package named ubuntu-mono . Removing the ubuntu-mono icon package will also remove the new Ubuntu GTK+ themes, in the light-themes package. So there s your explanation: the themes have a dependence on some monochrome icons, not on the Mono framework.

I've just committed some patches to the rawhide (not F14) tree that re-enable USB autosuspend on some devices. This set includes a workaround in the bluetooth input code that should handle the case where people were seeing their input devices become laggy when autosuspend was enabled, but there's still some chance that other bluetooth devices will behave slightly oddly. If that's the case then try:

echo on >/sys/class/bluetooth/hci0/device/power/control

and see if it improves things. If so then please file a bug and include information about the device you're trying to connect to.

The other thing this patchset does is enable USB HCD runtime power management by default. This means that if you have a host controller with nothing connected, or with connected but suspended devices, the host controller will be powered down. It'll automatically power back up if a device is connected or a connected device sends a wakeup event. The obvious failure mode here is that your USB ports stop working. If that happens, please file a bug and include the output of lspci and dmidecode (you may want to edit the dmidecode output to make sure that your serial numbers are removed). In both cases, assign the bug to me (mjg at redhat.com).

Once this seems fairly stable I'm going to be looking at powering down more mostly unused PCI devices. SD readers and firewire controllers are typically unused and both support generating wakeup events, so they're the next step.

Like many people I use Apache's mod_proxy to proxy from *:80 to a bunch of servers running upon 127.0.0.1:XX. (I've mentioned this too often; but in short I have a bunch of sites all running with thttpd under their own UID). Why do I use apache soley as a reverse proxy, instead of pound, varnish, nginx, or lighttpd? After all it is pretty heavy-weight. Well the answer to that is that I have a bunch of mod_rewrite rules. So I'm wondering, could I drop apache if I were to hack together a simple network proxy that would listen upon port 80, reading requests, and directing them to local servers? The answer to that is plainly "yes". There are many reverse-proxies around and writing them isn't hard. So what would be the point? Imagine a reverse-proxy that understood mod_rewrite rules. That would rock. In short we'd have to define three things:

• Matching vhost name.
• Destination to proxy to.
• (Optionally) the mod_rewrite rules

Given something like this:

LISTEN 1.2.3.4:80
host example.net or host www.example.net
 
   #redirect traffic here
   proxy_to 127.0.0.1 1011
   RewriteRule /about /cgi-bin/index.cgi
 

When the proxy received an incoming request to http://example.net/about it would actually send the request /cgi-bin/index.cgi to the host 127.0.0.1:1011. That seems neither too complex nor too impossible. The hard part would be emulating mod_rewrite 100%. Especially chained requests. I would be willing to write the trivial version, but I suspect the full emulation would be a job of diminishing returns. Am I right? ObSubject: You crossed the line first, sir. - "The Dark Knight"

While gzip is the standard Unix tool when it comes to compression, there are other tools available and some of them are performing better than gzip in terms of compression ratio. This article will explain where you can make use of them in your Debian packaging work. In the source package A source package is composed of multiple files. The .dsc file is always uncompressed and it s fine since it s a small textual file. The upstream tarballs can be compressed with gzip (orig.tar.gz), bzip2 (orig.tar.bz2), lzma (orig.tar.lzma) or xz (orig.tar.xz), so choose the one that you want if upstream provides the tarball compressed with multiple tools. Put it at the right place and dpkg-source will automatically use it. Note however that packages using source format 1.0 are restricted to gzip, and the main Debian archive currently only allows gzip and bzip2 (xz might be allowed later) even if the source format 3.0 (quilt) supports all of them. The debian packaging files are provided either in a .diff.gz file for source format 1.0 (again only gzip is supported) or in a .debian.tar file for source format 3.0 (quilt) . The latter tarball can be compressed with the tool of your choice, you just have to tell dpkg-source which one to use (see below, note that gzip is the default). In a native package, dpkg-source must generate the main tarball and you can instruct it to use another tool than gzip with the --compression option. That option is usually put in debian/source/options:

# Use bzip2 instead of gzip
compression = "bzip2"
compression-level = 9

For 3.0 (quilt) source packages, this option is not very useful as the debian tarball that gets compressed is usually not very large. But some maintainers like to use the same compression tool for the upstream tarball and the debian tarball, so you can use this option to harmonize both. In native packages, it s much more interesting: for instance the size of dpkg s source package has been reduced of 30% by switching to bzip2, saving 2Mb of disk. In the binary packages .deb files also contain compressed tar archives and by default they use gzip as well:

$ ar t dpkg_1.15.9_i386.deb
debian-binary
control.tar.gz
data.tar.gz

data.tar.gz is the archive that contains all the files to be installed and it s the one that you can compress with another tool if you want. Again this is mostly interesting for (very) large packages where the size difference clearly justifies deviating from the default compression tool. Try it out and see how many megabytes you can shove. Another aspect that you must keep in mind is that those alternative tools might use important amount of memory to do their job, both for compression and decompression. So if your package is meant to be installed on embedded platforms, or if you want to build your package on low-end hardware with few memory, you might want to stick with gzip. Now how do you change the compression tool? Easy, dpkg-deb supports a -Z option, so you just have to pass -Zbzip2 for example. You can also pass -z6 for example to change the compression level to 6 (it s interesting because a lower compression level might require less memory depending on the tool used). The dpkg-deb invocation is typically hidden behind the call to dh_builddeb in your debian/rules so you have to replace that invocation with dh_builddeb -- -Zbzip2 . If you are using a debhelper 7 tiny rules files, you have to add an override like in this example:

%:
	dh $@
override_dh_builddeb:
	dh_builddeb -- -Zbzip2

If you are using CDBS, you have to set the variable DEB_DH_BUILDDEB_ARGS:

include /usr/share/cdbs/1/rules/debhelper.mk
[...]
DEB_DH_BUILDDEB_ARGS = -- -Zbzip2

I hope you found this article helpful. Follow me on identi.ca or on twitter. One comment Support my work

Is Low Quality in Italian Academia related to the choice of Low Quality Software? Diego Gambetta and Gloria Origgi wrote an interesting paper titled L-worlds: The curious preference for low quality and its norms [1]. The paper describes how in Italian universities (and large portions of Italian life) there are covert agreements that both parties in a transaction (the employee and the employer or the buyer and the seller) will deliver less than agreed while pretending that they are offering the agreed exchange. People who offer high quality in exchanges are discriminated against because they make people who offer low quality in exchange feel guilty. Nathan suggests that this is the explanation for people choosing to pay for inferior software from Microsoft instead of getting superior software for free [2]. Now it does seem quite plausible that someone who is offering low quality goods in the manner of Italian academia would refuse to consider software from any company other than Microsoft, after all the easiest way of selecting software is to phone a MS representative and be told exactly what to buy. But I don t think that this explains even a significant fraction of the people who refuse free software. There is no direct analogy between bilateral agreements to produce low quality and the choice of MS Software because MS Software is quite expensive (they demand what would be considered a high quality trade in the jargon of the paper). If someone was to buy one of the cheaper laptops in Australia (around $650 new) and upgrade to the full version of Windows 7 along with purchasing the home version of MS Office then the cost of software would be almost as much as the cost of hardware. If they wanted one other MS product then the cost of MS software would probably be greater than the cost of hardware. Hardware costs are steadily falling and MS prices are only increasing, for people who use MS software we should expect that soon the MS tax will be the majority of the costs of running a typical PC. Good Reasons for Choosing MS One thing we have to consider is that there are people who have good reasons for using MS software. One example is the companies that depend on proprietary applications which are central to their business. When the entire company s data is stored in an undocumented proprietary database it s really not easy to change to a different application even when everyone in the company knows the software to be of amazingly low quality. If the vendor of the proprietary application in question decides to only support MS Windows then it s customers (victims?) have no choice about which OS to use. One interesting thing to note about such companies that are locked in to proprietary software is that the amount that they spend per year on license and support fees is usually greater than the cost of hiring one good programmer. If a few such companies formed a consortium to develop free software to manage their business where each company paid the salary of one programmer then after a couple of years of development they could move to the free software and reduce their operating expenses. Another category of users who have a good reason to choose MS is the people who play games seriously. If you want to play games then MS Windows does offer some real advantages. The price of games will usually be a fraction of the hardware cost (the serious gamers spend a lot more on hardware than most people) and MS Windows is apparently the best PC OS for commercial games. Personally I ve found that there are more than enough free games on Linux to waste my time, Warzone 2100 [3] is one that I currently play, and I ve tried Battle for Wesnoth [4] in the past and found it too time consuming and addictive. How a Preference for Low Quality could lead to Microsoft I think that everyone who has any significant experience in the computer industry has encountered companies that have large areas of low quality. This generally tends to be in large corporations as small companies can t afford the waste. In some large corporations Linux on the desktop is never considered, even when people are hired as Linux sysadmins and there are obvious productivity benefits to having the same OS on the desktop as on the servers (even if two desktop PCs are required so that proprietary software can be run on MS-Windows). Major wrote a good satire of the corporate IT non-working culture with a comparison to medical work [5], it illustrates the principle of a coalition to ensure low quality. He later documented how he was sacked by the low quality coalition at a company that uses a lot of Microsoft software [6]. So it does seem that when customers don t care at all about the quality of the result it does help drive some sales for Microsoft. But that doesn t explain the market share that they have. It takes a lot of work to get Market Share without Quality Microsoft has spared no effort in gaining market share. Every possible effort including buying out small competitors, aggressively FUDing competition, using all manner of legal attacks (including the threat of patent suits), and deliberately breaking standards has been used. I think it s reasonable to assume that the MS senior management are not entirely stupid, they do so many things that are unethical and possibly illegal because they know that they need to do so to maintain their market share. The result is that the market capitalisation of MS is almost as high as that of Apple and Apple makes vastly superior products. Given the amount of effort that MS uses to keep market share it seems apparent that they aren t just relying on customers not caring about quality. Should users have to Understand Computers? My observation is that most users don t want to know much about how their computers work. The desire to understand computers seems to be about as common as the desire to understand cars, people just want to buy one that looks good and have it work. The difference is that cars are very compatible while computers aren t. Cars have the same controls in the same places and large parts of the design are specified by law so that they can t differ between models. Proprietary software is usually intentionally incompatible with other software (both open and proprietary) to try and gain a competitive advantage. Hardware is often incompatible due to the rapid developments in technology and the requirements for new interfaces to take advantage of new features. In concept it seems reasonable for someone who is about to spend $30,000 on a car and $1000 on a computer (for hardware and software) to spend 30 times longer considering which brand of car to buy. One could argue that more than 30 times as much consideration should be given to the car as most people can t afford to discard a car that they don t like. As people spend a few minutes considering which brand of car to buy they can be excused for spending a few seconds considering which type of computer to buy. But once a choice has been made about which software to use it s very difficult to change to something else, while in comparison it s easy to drive a car that was manufactured by a different company. So a poorly informed choice made at an early stage can have costly long-term affects when buying software. If we had mandated open standards for file formats and data interchange then users would be able to make choices that don t result in their data being locked in to some proprietary format. Such standards could be set through government tender processes, if every government agency was to only buy software that complies with open standards then the proprietary software vendors would scramble to make their products less incompatible. The result would be that bad choices in purchasing software could become learning experiences that result in better purchases in future instead of being a lock on users that forces them to keep using the same software that doesn t satisfy their needs. Conclusion I think that the best thing about the paper by Diego Gambetta and Gloria Origgi is that it highlights the issue of low quality. No-one wants to be considered a loser, so maybe this can encourage people to strive for high quality (or at least try to make their work suck a little less). Regardless of the conclusion they eventually reach, it s probably good for people to occasionally wonder do I suck? .

• [1] http://www.sociology.ox.ac.uk/documents/working-papers/2009/2009-08.pdf
• [2] http://www.advogato.org/person/ncm/diary.html?start=405
• [3] http://wz2100.net/
• [4] http://www.wesnoth.org/
• [5] http://blog.ninbtma.org/2007/10/will-i-miss-it.html
• [6] http://blog.ninbtma.org/2007/11/just-forward-fucking-ticket_6090.html

The Apple universe is happy to live with whatever Steve Jobs allows to get in. Numerous examples going back at least 20 years are available; most recent Apple's law action against a manufacturer offering compatible batteries and now an external harddisk for iPods. Microsoft is obviously the traditional enemy for many (personally, I think there's not much difference between MS, Apple and Google anymore.) So no surprise that MS Exchange plays a role in the story about a remote wipe function of many smartphones. Just being connected to a MS Exchange server allows the administrator of that server to remotely factory-reset the smartphone. I'm not sure if there's a complete list of vulnerable smartphones, but at least iOS, webOS and Android based phones are affected. Personally, I'd say this is a serious security issue. But of course, according to the manufacturers, it is a function for the convenience of network adminstrators so they can wipe stolen phones remotely. Update: Nico: If I'm somebody who is really after your data, I've got your smartphone and I'm clever enough not to connect it to the net before I got the data out. So you're only catching the Oh look what I accidentally found case, which I think is really just giving you a false sense of security. Anonymous: Google is the company with the Do Not Be Evil company motto and the very good press and marketing team touting everywhere how much they do for the good of everybody. Except when it cuts into their earnings. Like really cooperating with the Linux kernel folks. Like living up to what they announced they'd do in China. It's also the company covertly collecting WLAN data, and only admitting to exactly as little as they can get away with when it comes into the open. And, in the end, simple greed will do the rest: it's a powerful big company with plenty of very well paid career positions, and in my experience this company tends to attract people who will use this power in any way they can to increase their income. At this level of business, a company motto like the one of Google is just a few meaningless words. The real company motto is Earn Money. In Whatever Way. Other examples are told by anybody who needed to interact with their so called customer support, either because they got their free account locked or because they had a specific request as a paying customer placing ads on Google (and I'm talking about a company with at least several CHF 10k per year in ads, not just a small nobody.)

I'm a keen amateur photographer and have been since I was at school. Of course back then it involved smelly chemicals, strange lights and dark rooms. The expense and hassle of film eventually did it in for me and in 2000 I bought my first digital camera. After a series of increasingly sophisticated devices, back in 2007 I bought a Nikon D80 and soon thereafter I discovered that you can do things with raw camera files that have some analogies to the darkroom processing of my youth. For most of the time since then I've used UFraw for processing my raw images, combined with increasingly sophisticated batch files to harass it into shape. It has increasingly frustrated me that UFraw doesn't do several things that are essential to processing most files, since once the image leaves that programming it usually ends up in an 8-bpp processing path. It seems to me that the saving the processed image into an 8bpp format like jpeg is the last step before leaving the image alone. Of course this can't always be the case since sometimes a judicious use of the 'Resynthesizer' plugin can make some unsightly elements just disappear, or one needs to carefully apply some censorship to the 14 year old in the background so that the 7 year-old's parents can put the photo on the mantelpiece without worrying about what Granny will say about how much better children's manners were in her day. I'm hoping too, that when I have grandchildren I'll be able to tell them that 'in my day we only had 8 bits per pixel, and we were lucky', but The Gimp isn't there yet, no matter how much promise Gegl seems to offer. The thing that has particularly annoyed me is sharpening. UFraw doesn't, and as a result my workflow has either involved applying some blanket sharpening, much like the camera does, or moving the image into a second tool just to apply some sharpening. This week has been particularly demanding, as the school asked me to 'take some photographs of the production'. So of course that's a thousand-odd photos that I want to do a cull down to a hundred or so, and then I want to quickly do justice to the good ones so I can move on to other stuff. Well this time my frustration with UFraw hit that ceiling that got me to try something else. I've done this in the past, and found myself going full circle and ending up right back at UFraw with a few extra lines added to my script, but this time I tried rawstudio and in processing the 'can you do the class shots overnight please' test it produced much nicer results in much less time than I have ever been able to with UFraw. So I've switched, and the more I use it the more convinced I am that UFraw will be getting uninstalled pretty soon. That's not to say that rawstudio is perfect: it isn't. In particular it has some user interface failures that are kind of annoying to me, but they aren't enough to stop me using it. In fact I liked it so much that I immediately had to pull the nightly repository and build that, just to get the new saturation handling. The user-interface niggles that do get to me are:

• The keyboard key to switch between zoom 1:1 and zoom to fit is '*', which I have to access with 'shift' down since I'm on a laptop. This was less bad when I discovered I could right-click, but it's still annoying. 'z' would be a much better choice for right-handers, or '/' for left-handers. Or perhaps they could both be made to work. When working on an image I often want to switch between some particular area and the whole image when checking colour balance, sharpening, noise, etc, so this is one of the most used keyboard functions.
• Using the mouse wheel to adjust sliders varies greatly in it's effect. A single detent for contrast or saturation often overshoots the useful range, so I have to drag the fiddly slider a fiddly pixel-and-a-half to go from a contrast of 1.00 to 1.03.
• Not all modifications to an image show in the UI. When I crop or align an image RawStudio remembers the settings for these things (which is fantastic) but if I come back three months later I might not realise I'm looking at a cropped image. I can see all of the hue, saturation, sharpening and so forth, but there should be something in the UI to indicate that the crop and alignment have happened also.
• It confuses me what settings get applied to a fresh image. I seem to get some kind of random selection of which DCP profile is being applied, and there's no way to set a default sharpening or noise reduction. Of course some of this is hard: noise reduction defaults really need to change depending on camera & ISO, and sharpening defaults probably should vary by lense and aperture too, to some extent.
• The icon view runs across the top, taking up important screen real-estate. On my widescreen laptop I'd like the option to run it vertically where otherwise there's wasted space.

Things that I immediately love, over what UFraw gave me:

• Sharpening.
• Straightening images.
• Cropping images (in some versions of UFraw this works, and in some it's broken, but the UI is useless in all of versions I've seen).
• Colour profiles.
• Lens corrections.

Things that I immediately miss from UFraw:

• White balance presets.
• Wavelet noise reduction.

There are a bunch of things that RawStudio also does that I haven't particularly found a use for yet, but which I suspect I will quite like when I do. The image classification and stuff seems likely to be useful, for sure. In choosing to switch RawStudio I also looked at RawTherapee, which has recently relicensed under the GPL, but I chose RawStudio because it has vastly better performance on my laptop, and because I found the interface much more immediately accessible. RawTherapee also exceeds what UFraw can do in many ways, and looks like it might be useful for the really difficult cases, but learning to drive it with any facility could take some time. Especially if you only have an i7 740QM with 6G RAM, it would seem.