Nothing is more embarrassing than a website announcing that something will happen on a particular date -- e.g., a product will be released, a feature will be turned on -- after the date has come and gone! Even worse, putting things off repeatedly can be a lot of work! To help such people, I just did a very quick 10 minute hack I'm calling The Vaporizer. It looks like just a date on a webpage. However, if the date originally listed has come and gone, a little bit of Javascript will change the site so that it shows tomorrow's date instead. Vaporware providers of all types can use it to safely (and effortlessly) put things off without worrying about looking overdue! I have seen the future, and the future is tomorrow.

Missed the morning talks.. looking forward to replays being available.. rails Projects not bothering to release tarballs, and just publishing a git repo (hopefully with some tags!) will only become more common, I think, and it's interesting to see this is already very common with ruby gems. Faking up an orig tarball is still a reasonable workaround now, but will probably look very retro in 5 years. Re Debian vs Rails communities, I wonder if there's a generation gap in here somewhere.. rest (will update throughout the day)

po4a - or PO for anything. PO is the de facto standard for translations and po4a tries to help maintainers prepare packages for translation.

1. I18n (internationalisation) is making a package accessible for translators and is an upstream/maintainer action. i18n is done once for all translation support and updated when the content changes (typically during the build).


2. l10n (localisation) is using i18n support to create an individual translation for a specific locale. l10n is done many times per package, with individual translations being updated each time the i18n content changes. (Tdebs will support this area but are a separate topic from po4a.) PO files that exist when the POT is updated are also updated, tagging modified strings as "fuzzy".

The natural burden, therefore, falls within l10n but without i18n, l10n is all but impossible (it means having to do the i18n "outside" the package).
I'm using po4a and I've been doing various translation-support stuff now for a few years, so I'm going to add po4a into my blog entries as a series to help maintainers make their own packages i18n aware.
This is particularly important for:

1. all Debian native packages, particularly scripted packages. My example will be based on perlpod documentation often embedded wthin the perl scripts themselves as well as DocBook XML for manpage generation. po4a supports many other formats.


2. All manpages (or other documentation) created by Debian maintainers

To make things easier, I'll be creating a new category in my blog ("i18n") for such entries. This is helpful for me too because the first time I used po4a (current versions of emdebian-tools) I basically got things wrong and made more work for myself than was necessary. This entry primarily concerns providing i18n support from manpage sources and perl scripts. i18n support in compiled packages is a separate issue. The fixes described here will be included in the next version of emdebian-tools (which will be uploaded to Emdebian but not into Debian until after Lenny).
Terminology

• PO - (Portable Object) files contain the POSIX C locale strings abstracted from the "master document" which in many cases is the source file in the relevant package(s), alongside the translated version of that string (if any). Multiple master documents are typically combined into a single collection of translatable strings. PO files are translated into individual locales, one PO file per locale.


• POT - PO Template - the definitive collection of translatable strings from which all PO files are prepared and updated. The POT file is never translated directly, it is maintained, updated and packaged by the maintainer.


• gettext - the binary application that handles the translatable strings. gettext abstracts the strings out of the source files during the package build and inserts the translated strings back into the text actually displayed to the user at runtime. If no suitable PO file exists, the user gets the untranslated string, exactly as it appears in the POT file. If a suitable PO file does exist but the strings have been modified since translation, gettext drops "fuzzy" translations that do not match precisely and the user gets a "mixed" translation containing some translated strings and some untranslated. Most translators try to maintain a 100% translation because the ugliness of the "mixed" output is a partial incentive to keep up with the updates.

Using po4a
po4a needs a config file where the maintainer keeps all metadata relating to the master documents and the relevant formats. po4a then converts the master documents into a collection of strings for translation and puts those strings into a POT file (a PO template file). The maintainer packages the POT file (and optionally sends the POT file to relevant mailing lists seeking translations to be returned as PO files) and keeps the POT file up to date during the build process. With a config file (e.g. called po4a.config), all the maintainer needs to do is call:

po4a po4a.config

po4a then parses the list of files defined in the config file, generates the lists of strings, collates the lists and identifies where the same string is used in multiple files and writes out a POT file containing references to where the strings are used and the unique strings.
po4a config file
Comments start with '#' in the usual way. Instructions are enclosed in square brackets.
po4a_paths defines where the POT file will be put (it can be extended later but I'm keeping this simple). The name of the POT file is important for compiled packages but can just be the name of the source package for scripted packages and/or manpages.

[po4a_paths] po/emdebian-tools.pot

type identifies *each* master document to be processed by po4a. The syntax is well suited to being generated from some other list within the package but I'll give just a snippet of the final output here:

[type: xml] doc/xml/em_autobuild.1.xml

[type: xml] doc/xml/embug.1.xml

[type: pod] emtargetcmp

[type: pod] emdebcheck

That's it. The complexity of my current script was entirely unnecessary.

Also, po4a is more than capable of handling multiple document types within the same source package and the same POT file which is something that I had missed so far. One more hack that can be removed.

All I do now is use xmllint to "clean-up" the DocBook XML and pod2html and some XSLT to generate HTML documentation for the website. The loops to do those tasks already contain the list of files to be processed by po4a, so a few echo commands is all that is needed to create po4a.config at build time and run po4a to generate a combined POT file for all strings in the package.

#!/bin/sh

CONFIG="po4a.config"

echo "[po4a_paths] po/emdebian-tools.pot" > $CONFIG

for file in  ls doc/xml/*.xml ; do

	xmllint --format "$file" --output "$file"

	echo "[type:xml] $file" >> $CONFIG

done

for file in emdebcheck emtargetcmp emprunecross Emdebian/Tools.pm \

 em_installtdeb emrecent emcache; do

	if [ "$file" = "Emdebian/Tools.pm" ]; then

		pod2html -outfile html/EmdebianTools.html -title $file  ../$file

	else

		pod2html -outfile html/$file.html -title $file  ../$file

	fi

	if [ "$file" = "Emdebian/Tools.pm" ]; then

		pod2man ../$file > man/Emdebian::Tools.3

	else

		pod2man ../$file > man/$file.1

	fi

	echo "[type: pod] $file" >> $CONFIG

done

po4a $CONFIG

He probably will hate me for it, but still: If you want to see our DPL dance, this 10 minute video will be what you want. :) (If you are at DebConf8, download it from our local machine instead.) Or you just wait a bit, the video team said they want to include it in our streaming, and it will also be put into our meetings-archive. There is also a, way shorter, video showing Wouter (local DebConf8 location) dancing. If you like it and want to see our DPL doing more of this - feel free to comment here, maybe we get him drunken enough :) Comments: 0

This is part 1 of a series on articles about the Firewire security issues mentioned below. For many years now, attacks via Firewire / i.LINK / IEEE 1394 have been a known security issue. Basically, if you gain physical access to a PC or laptop which has Firewire ports (or PCMCIA/Cardbus/ExpressCard, more on that later) you can

• read arbitrary RAM contents from the victim's system,
• overwrite arbitrary RAM contents with whatever you want,
• and perform many, many severe attacks based on the two issues above. Examples include grabbing a full RAM dump via Firewire (takes only a few minutes), grabbing ssh-agent keys, grabbing screen contents, modifying screen contents, bypassing login/password screens, and many, many more...

All of this is done by exploiting a "feature" of the Firewire spec (OHCI-1394) (PDF), namely that it allows read/write access to physical memory (via DMA) for external Firewire devices. Worse, as this is DMA, the CPU/OS will not even know what's going on. Even worse, this works regardless of whether you have locked your screen with a password-protected screensaver, or xlock, or vlock, or whatever. As long as the system is running, you're vulnerable. In this article, I intend to give a fairly complete overview of the available papers published on this issue, tools for testing the attacks, as well as mitigation techniques for various OSes. If I'm missing some important papers or tools, please post a comment! Papers, Attacks, and Tools Over the years a number of presentations and papers have been released with information about these Firewire issues. Maximilian Dornseif et. al. The first publication that I know of was done by Maximilian Dornseif, Michael Becher, and Christian Klein. They gave a number of talks on various security conferences on this topic:

• Pacsec 2004: 0wned by an iPod (PDF)
• 21C3: Hacking with Fire / Feuriges Hacken - Spa mit Firewire (PDF)
• Cansecwest 2005: Firewire: all your memory are belong to us (PDF)

They also released a number of tools, Firewire libraries for Mac OS X and Linux, as well as small demo scripts which use those libs:

• pyfw-20041111.tar.gz: Mac OS X Firewire bindings for Python, various demos
• http://c0re.23.nu/c0de/pyfw/: Other versions of the above code
• pyfw_linux.tar.gz: Linux Firewire bindings for Python (can be used with above demo code on Linux)
• ipod-linux-pyfw.tar.bz2: Instructions (+ disk image) on how to turn your iPod into a Firewire attack tool

Adam Boileau In 2006 Adam Boileau (a.k.a. Metlstorm) gave a talk called Hit by a Bus: Physical Access Attacks with Firewire (PDF) at Ruxcon 2006. In 2008 he then released a set of tools:

• pythonraw1394-1.0.tar.gz: Python bindings for libraw1394 (Linux). Tools: businfo, romtool, 1394memimage
• winlockpwn: Python script which can circumvent a locked Windows XP screen (an arbitrary password will log you in)
• bioskbsnarf: Grabs/shows the BIOS keyboard buffer via Firewire (which often contains your BIOS password)

Peter Panholzer As of early 2008 Peter Panholzer from sec-consult.com published a two-page whitepaper which says they were able to run a winlockpwn-like attack on Windows Vista via Firewire. There's not much information in the PDF unfortunately, and no tools were released, as far as I know.

• Whitepaper: Physical Security Attacks on Windows Vista (PDF)
• Bugtraq Thread about the whitepaper

David R. Piegdon The most recent toolset and papers I know of are from David R. Piegdon (a.k.a. IosTrace), who gave a number of talks in 2007/2008 about the issue, and also released a toolset called SEAT1394.

• 2007: OpenChaos: hacking in physically addressable memory (no extra slides)
• Easterhegg 2008: FireWire Hacking: paper (PDF), slides (PDF)
• More papers here...
• http://0null.net/software/SEAT1394/: tools and libraries for pyhsical memory access via Firewire/DMA: 1394csrtool, liblinear, libphysical, debug-sshkey, dump-memory, remote-ps, show-stack, snarf-sshkey, test-all-pagedirs, dmashell, ...

I'll go into much more detail on how the tools are used and what they can do in another follow-up article. Mitigation There are ways to eliminate or at least mitigate these attack vectors. The simplest and most secure way is to not have any Firewire ports installed (don't put Firewire PCI/PCIe cards in your PC, don't use Firewire PCMCIA/Cardbus/ExpressCard cards). Now, if you have a laptop with built-in Firewire ports, you have a problem, of course. In that case you could still physically destroy the port (by opening the laptop and cutting/desoldering stuff, or by putting glue/epoxy in the port in order to prevent any Firewire cables being attached). These are slightly drastic (but effective!) measures. Note: Even if you don't have any Firewire ports, you're not automatically safe and secure. If your laptop has a PCMCIA/Cardbus/ExpressCard slot, an attacker can simply insert a PCMCIA Firewire card (for instance) in that slot. Chances are, that your OS will automatically load the driver for that card and also the Firewire drivers you'll need if you want to use the card for attaching Firewire devices. Game over. Your "secure" laptop is now vulnerable... If you cannot (or don't want to) remove/destroy/disable your Firewire ports, the next best thing is to ensure that nobody except yourself ever gets physical access to your PC/laptop. This is hard to do for a PC, and almost impossible for a laptop, mind you. Finally, there are some software measures you can use to prevent at least physical DMA access for Firewire devices: Mitigation: Linux Pretty much every Linux system with the "old" Firewire drivers loaded (kernel module ohci1394 et. al.) is vulnerable to these issues. Newer kernels now also ship with a new Firewire stack called "juju" (kernel module firewire_ohci et. al.) which may or may not have the same issues (not fully tested by me so far, will report back later). Per default, all recent kernels, e.g. 2.6.26, are vulnerable, but see below. Under Linux, simply using a kernel which doesn't have any Firewire support (neither built-in, nor as a module) is the most secure option. If you must have Firewire support you can load the ohci1394 module with the phys_dma=0 parameter to at least disable physical DMA support:

  $ modprobe ohci1394 phys_dma=0

I have personally tested this on some boxes and I can confirm that it renders the currently published tools useless. As for the new "juju" Firewire stack, I'm not so sure. A few quick tests showed that the currently available tools don't work with the new stack, but you shouldn't feel too secure! AFAIK the new stack does support (or will support soon) physical DMA for Firewire, so it's probably just a matter of adapting the tools a bit (I'll do some testing/research on this later, as time permits). Mitigation: Mac OS On Mac OS you might also be able to completely remove Firewire support from the kernel (but I don't know if/how that can be done, not sure if you can easily recompile Mac OS kernels, and/or if you even have buildable source code and toolchains for that). However, you can at least remove the Firewire support in the default Mac OS installation by unloading AppleFWOHCI.kext:

  $ sudo kextunload /System/Library/Extensions/IOFireWireFamily.kext/Contents/PlugIns/AppleFWOHCI.kext

Thanks to a Daniel Reutter for letting me abuse his MacBook via Firewire and for finding the above kextunload command line. We have successfully tested that after unloading AppleFWOHCI.kext the current tools won't work anymore. The tests were done on a Mac OS X Tiger (I think) with all recent security updates applied. Please leave a comment if you can test other versions of Mac OS X... Mitigation: Windows As for Windows, well, I guess you're screwed. While Windows XP does implement sort of "protection" in that it only allows physical DMA access via Firewire to devices which "deserve it", e.g. iPods (or any other Firewire mass storage device, I guess) this can be easily defeated by having your attack PC/laptop pretend to be an iPod (see the romtool Python script by Adam Boileau). The only remaining option I know of (short of removing/destroying Firewire ports or preventing physical access alltogether) is to disable the Firewire ports/drivers in the device manager (untested by me so far). If you do that, remember to also disable all PCMCIA/Cardbus/ExpressCard controllers, of course (see above). So far I've tested Windows XP SP2/SP3 successfully with Adam Boileau's tools. I haven't yet been able to test Windows 95/98/Vista, if you can verify one of them, please leave a comment. Mitigation: OpenBSD/FreeBSD/NetBSD/OpenSolaris/... On OpenBSD you're likely not vulnerable as OpenBSD doesn't have any Firewire drivers at all, as far as I know ;-) As for FreeBSD, NetBSD, OpenSolaris, and other OSes I don't have any information. I might be able to test one or two of them in the nearer future, but please leave a comment if you have some information about whether they are vulnerable and/or how you can secure your system... Conclusion That's it for now. I hope you now have a good overview of these issues and how to protect. I can only urge you to take this problem seriously! Three or four minutes of leaving your laptop unattended are fully sufficient for an attacker to get a full forensic image of all your RAM contents for later analysis. This is at least as critical as the Cold Boot attacks, if not worse. I will follow-up with more articles about some more interesting details on these Firewire issues, how to use the above tools, and I'll report on some of the stuff I was able to find in RAM dumps gathered via Firewire...

I have been thinking for quite some time about migrating the maintenance of the wormux package to git, but in a really smart way so that:

• we preserve history currently stored in svn
• a new integration branch should exist - we used the incomplete package model with svn-buildpackage, but now it makes more sense to have the complete history, including the upstream code at the time
  
• all previous orig tarballs should exist in the git repo in pristine-tar format
• upstream's svn repo should be included entirely so that we can cherry-pick any patches we might want or need

Since both the package repo and the upstream repo are in SVN, you can't simply use git-svn for both in a single repo, so I started thinking of some scheme to be able to join both repos and the (uncommitted) orig tarballs (which are backed up on DGT's webspace).


Tonight I have experimented a little with my git-svn clones which I made previously and eventually thought that:

• the main repo should track the upstream repo via git-svn
• I should properly tag the debian package versions in the svn.d.o repo, so svn tags no longer look like branches in git
  
• ideally all the branches in the package repo should be reproduced in some corresponding branch in the final repo
• the integration branch (that should contain the real debian complete packages) should be created with a non-existant merge strategy which I call overlay (basically, for two branches A and B, the strategy resumes to: all files as in A are present, any files unique to B are present)
• using pristine-tar forces me to keep the 'upstream' branch reserved so I need to avoid using that

It seems the last point could be obtained in some way via this small set of commands (I am sure it needs some modification to make the integration work in another branch C):

git merge --no-commit -s ours B
git checkout B -- .
git checkout A -- .
git commit


Note: Looking back, I think I can do a better job next time and I will se if I can use the resulting repo to create a new repo and drop the svn package maintainance for wormux. If this works, I might be able to create a script to allow migration of svn-bp maintained packages to git, at least for the Debian Games Team.


Since you can't fix a problem if you don't take things one at a time, I started by cloning the initial git-svn generated repos.

So after waiting some time (both git-svn fetch commands took long enough), I had to git-svn repos:

• repo A - the package repo
• repo B - the upstream svn clone repo

I tried to clone both A and B, to avoid distroying the original A and B clones and noticed that the svn branches were not visible from the cloned repo since the origins didn't had any other local branches than master - the svn trunk for each of them. So I added tracking branches in A for the experimental and sarge versions of the package. I also added some tags in A so that the svn tags don't get lost in between.

In the end I got my A' repo that had:

0 eddy@heidi ~/usr/src/wormux/tt/debian-bare $ git br -a
debian-experimental
debian-sarge
debian-sid
* master
origin/debian-experimental
origin/debian-sarge
origin/master

Yes, and having master in A' is not a good idea, but since I made A' a bare repo I had to suffer. Well, one more thing to better in iteration 2.


Excellent, now I can go in B and add the new remote repo which is A' which I should have called pkgsvn, but I ended up call it 'origin'. Since git-svn doesn't treat the svn repo as a true remote, this was possible.



Due to some confusion I also added 'ups-' as a prefix to the svn remote trunk and tags, but I am still unsure if this was necessary - I'll have to see more clearly into the pristine-tar restrictions area.

After all of this I got this branch configuration:
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch
* master
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch -r
0.8-final
0.8beta4
origin/debian-experimental
origin/debian-sarge
origin/debian-sid
origin/master
ups-tags/wormux-0.7
ups-tags/wormux-0.7.2
ups-tags/wormux-0.7.3
ups-tags/wormux-0.7.4
ups-tags/wormux-0.7.9
ups-tags/wormux-0.7.9rc1
ups-tags/wormux-0.7beta3
ups-tags/wormux-0.8
ups-tags/wormux-0.8alpha1
ups-tags/wormux-0.8beta1
ups-tags/wormux-0.8beta2
ups-tags/wormux-0.8beta3
ups-tags/wormux-0.8beta4
ups-trunk
wormux-0.7
wormux-0.7.9
wormux-0.8beta1
wormux-0.8beta2

So more tracking branches were necessary; I fired 'gitk --all' and got a nice picture, after adding tracking branches for branches originating from A':


This is not yet complete and now I am feeling sleepy, so I'll continue once I experiment more.

Thousands of the worlds brightest, most handsome, and fragrant-smelling developers gathered in Bryant Square Park, New York City, for PreventaConf 2008 Project founder Jaldhar H. Vyas gave a stirring keynote covering such interesting and informative topics such as "Is Preventa just a big joke or are you really working on it?" The audience were shocked to learn that he actually is. Attendees gather for lunch sponsored by Heinz baked beans. Lunch is marred slightly by rain pouring through the roof and an altercation which resulted in the forcible expulsion of a known Ralph Nader sympathizer and his Move On organizer "girlfriend." Wonderful news! The government of Zimbabwe has offered to sponsor Preventa to the tune of ONE HUNDRED BILLION DOLLARS. The Preventa project will use that money to cover infrastructure costs and subsidize developer travel expenses. Thanks Zimbabwe! And finally here is a scene from the lab where developers are busy trying make Preventa a reality. Look forward to some real news soon!

To the lovely person who subscribed me to around 200 mailing lists this evening:

I will find you.

The best way to run a blog is to run your own blog server. This can mean running an instance on someone else’s web server (some ISPs have special hosting deals for bloggers on popular platforms such as Wordpress), but usually means having shell access to your own server (I’ve previously written about my search for good cheap Xen hosting [1]). There are platforms that allow you to host your own blog without any technical effort. Three popular ones are Wordpress.com, LiveJournal.com, and Blogger.com. But they give you less control over your own data, particularly if you don’t use your own DNS name (blogger allows you to use their service with your own DNS name). Currently it seems to me that Wordpress is the best blog software by many metrics. It has a good feature set, a plugin interface with lots of modules available, and the code is free. The down-side is that it’s written in PHP and has the security issues that tend to be associated with large PHP applications. Here is a good summary of the features of various blog server software [2]. One that interests me is Blojsom - a blog server written in Java [3]. The Java language was designed in a way that leads to less risk of security problems than most programming languages, as it seems unlikely that anyone will write a Blog server in Ada it seems that Java is the best option for such things. I am not planning to switch, but if I was starting from scratch I would seriously consider Blojsom. But for your first effort at blogging it might be best to start with one of the free hosted options. You can always change later on and import the old posts into your new blog. If you end up not blogging seriously then using one of the free hosted services saves you the effort of ongoing maintenance.

• [1] http://etbe.coker.com.au/2008/05/28/xen-hosting/


• [2] http://asymptomatic.net/blogbreakdown.htm


• [3] http://wiki.blojsom.com/

Share This

Last weekend I updated the Debian package of MySQL to 5.0.67, the latest Community Edition release. It was not uploaded into the Debian archive, but is available from my home directory on gluck (amd64 only at the moment). The next Debian release lenny is frozen already, which means no newer upstream releases get accepted into testing before lenny is released, hence lenny definitely gets shipped with MySQL 5.0.51a. We will not see MySQL 5.0.67 or 5.1 in lenny, but we will continue providing backports of newer MySQL releases.

Our current primary goal regarding the MySQL packages in Debian is to get 5.0.51a in shape for lenny, so please do not send us mails asking about MySQL 5.1 or MySQL Cluster. When lenny got released we will upload MySQL 5.0.67 (or any newer community release) to unstable, and will then take care of MySQL 5.1 and MySQL Cluster.

Hi folks! We’re almost done for the official Summer of Code program and I thought that I couldn’t let it end without another update, so here we are (you may thank Daniel Burrows for additional poking). The Gtk+ interface for Aptitude is making great progress. The product is not final yet but already implements many of the planned ideas, with others to come. Here are the screenies: So, let’s see the tasks presented in my previous update:

• A real user interface <- much progress here
• Being reliable <- check! we don’t crash all the time anymore
• Packages dependencies, pre-dependencies, etc. <- check!
• Hyperlinks between packages <- check!
• Advanced package search <- check!
• Tags support (tag clouds ?) <- well, no clouds yet, but we have tags
• Linking with interesting data sources like popcon <- not done yet
• A lot of interesting things <- well..

In some more details: we are now much closer to the final interface. Some things are not done yet, for example, notification bars la Iceweasel (or is it Firefox ?) are in the works for unobtrusive notifications about important things and a little guidance. We are much more reliable now. We eliminated lots of memory management problems that resulted from, mmh, imperfect coding discipline. We also lock the interface when important backend stuff reload to prevent the user from shooting himself in the foot when triggering an install when the APT cache is reloading for example.. The backend refactoring took a sizable part of this second term but it was worth it. The code is now much cleaner and uses no more (at least much fewer) ugly unscalable hacks. One very important feature of Aptitude is the ability to easily display all kinds of dependencies and navigate through them by hyperlinking. It’s now working in the Gtk+ interface as well. Feel free to click on whatever looks like a package or a package version. We now support the powerful Aptitude package searching syntax. Daniel Burrows is talking with Enrico Zini to bring the power of apt-xapian-index to Aptitude: autocompletion! lightning fast full-text searches! tag clouds! ponies! There is still a lot to be done. I’m not going away after the GSoC program. I’m here to stay, for Aptitude and Debian. Daniel Burrows has been very helpful and supportive. The Debian community is as awesome as I knew it before participating in the program. Daniel Burrows is planning to merge the Gtk+ interface into the main branch in September and packages may appear into Experimental. With the very interesting evolutions of the upcoming Adept-3, Lenny+1 will show very interesting changes in the area of packages managers. Now for you. I need you to test my interface. All necessary instructions are on the development trac : http://dev.graffit.net/aptitude/trac.

Jimmy Kaplowitz writes:

“Software in the Public Interest, Inc., will hold a public board of directors meeting on Wednesday, August 20, 2008, 19:00 UTC. [...] SPI meetings are held on the OFTC IRC network, irc.oftc.net, in #spi. [...] Agenda“

A couple of weeks ago, I asked why people do and don t join SPI? The answers I got can be grouped into a few headings:-

Don’t know about SPI

“I admit that I was unaware of SPI. However seeing its age, a few familiar names on the board and also on the project list it looks like a solid organisation.”

“I too am unfamiliar with your organization”

“I guess I would be yet aother person who was not aware of this group.”

“Lack of awareness of SPI is probably the biggest hurdle. I too, was unaware of SPI. I am aware of the Linux Foundation and the FSF and others, but how does SPI differentiate itself and its goals from those organizations?”

Don’t know about joining

“The membership page on the SPI website is buried at the bottom of a secondary menu. One has to really go looking for it. And this definitely gives the impression that adding members is not a very high priority.”

“I ve been looking a while for how to join, is not the first time, I did before and never found it until now”

Don’t see why to join

“Even as an open source software advocate, I have not joined SPI because my particular interest and focus is on Plone. The Plone Foundation handles largely the same functions that SPI does for smaller, independent groups.”

“Why, what’s in it for me? The betterment of mankind or bragging rights when a corporations steals all of my work and gets rich off of it.”

“If you are someone who is very interested FOSS (free and open source software) rights and licenses, you are probably more inclined to become a part of the FSF (Free Software Foundation) or the OSI (Open Source Initiative).”

I’d welcome any thoughts you have on how we should overcome these barriers, either in the comments here, or on IRC #spi - I think I should be there a bit before or after the meeting. I’ll try to summarise the best over the next Wednesday or two.

[ Taken and modified from a post to the hecl-devel mailing list ] I've been talking with a company about a consulting job regarding a programming language for mobile apps. Sounds familiar, right? There is, however, a bit of discussion regarding the syntax - my contact at said company maintains that Hecl's syntax is unfamiliar to many programmers, and I think on the face of it, I'd have to agree. Of course, Hecl's syntax is like it is because when I started the project, being very small was a necessity, and a very simple syntax facilitated that goal nicely. Additionally, the syntax allows for extreme flexibility, letting you create new control commands, and "DSL's" (domain specific languages) easily. Also, sticking with small/simple means that the interpreter and parser can both reside on the phone, instead of doing some kind of solution where the development/host computer compiles the scripting language to byte code, and the interpreter on the phone runs the byte code. That sort of strategy has some advantages, but has disadvantages too: if you don't have an on-board parser, you don't get "eval".

---

A quick Hecl syntax refresher: In Hecl, everything is a command, seperated by spaces, and grouped with :

if   ...     ...  

Is a command that takes two arguments. It evaluates the first one, and if it's true, evaluates the second one. This is different from languages like C, Python or Ruby, where 'if' is part of the language. For those unfamiliar with languages like Lisp, where this looks strangest is probably in conditions or other expressions:

if   = $foo 10     puts "foo is ten"  

Or

set foo [+ [* $bar 10] [- $beebop $doowah]]

Which is certainly a different take on things than most 'mainstream' languages. That radical simplification is, however, one of the ways that I used to save space in the interpreter.

---

Personally, I'm pretty happy with how Hecl looks, but I am sensitive to the 'marketing' argument: popularity helps, and this isn't the first time I've heard complaints about Hecl looking "funny" to people. What do you think? A) "I like Hecl just the way it is and wouldn't want to change it!" B) "Hecl is quite useful, but I just want a mobile scripting language, the syntax doesn't matter much." C) "I think the syntax is ugly, but I use it because I need an interpreter." Also related: 1) "I don't care too much about code size. My current apps target N kb as an acceptable size." 2) "Small is really important. The smaller the better." And: I) "I don't care about eval, or being able to run human readable code on the phone." II) "I really like the fact that you can download or even write code directly on the phone." And finally: X) "I don't care about midp1.0" Y) "The fact that Hecl runs on midp1.0/cldc1.0 is important to me." Now, I am not going to run off and change Hecl! Code that currently runs ought to continue to run, modulo some tweaks here and there, api changes, and the normal stuff. If I do something 'new', it would probably be a clean break (although of course code that could be reused would be), and will be in addition to Hecl, rather than instead of Hecl, because I would be working on the 'new' language as a day job. I do that with Hecl too, but not all the time, as I often have consulting/contracting work.

Third day of DebConf8 now - and it has been a pretty nice time since I arrived. The local orga team is doing a marvelous job, making this whole conference a really nice experience for about every attendee. Thanks to that we had a lot of time doing actual Debian work, which meant that we got a nice number of changes in dak either done or planned for the future, including a redesign of our database schema. Which is urgently needed if we want to follow some of the points we discussed. Like throwing away apt-ftparchive and generating the Packages files ourselve. Or various other enhancements. So for dak, the changes are summarised into:

• Adapt an old patch for process-new, so that NEW packages only adding new binary components are sorted higher during the processing.
• Imported a patch from Joachim that adds an (effectively) anti-s390-buildd-maintainer option to our cruft-report full mode. Well, actually it lists a set of packages which are marked “not-for-us” on the buildd, but still have a (outdated) binary in unstable, effectively blocking updates of the packages going into testing. Ideally a buildd maintainer would report to the maintainer (through a bug or even just a mail) when he sets not-for-us (which is a short-term-solution, long-term P-a-s should be used), but that doesn’t seem to happen.
• Replaced a 128 line NIH syndrom perl script with one of those new-fangled sarge-and-later options in rsync :)
• Fixed bugs in code-pathes that are nearly never used - but which we managed to trigger twice while fixing a gpl violation (missing source) in our archive.
• Merged a patch that changes NEW handling of packages for stable/oldstable.
• Added signing of the buildd incoming.debian.org directory. Still need to announce this on d-d-a, but buildd maintainers should now be able to enable the gpg verification.
• Automate the i18n package description translation import so we finally get rid of the “one-time-manual-sync-back-in-2006” outdated translations. And have an automated but secure way to get updates.
• Started the layout of the new projectb.

But we did not only do work on dak, we also did some work for the conference. In the days down here we (re)wrote the Pentabarf entrance controller, enabling our frontdesk to check people in, keeping track of various details they need to know about them. We wrote entirely new controllers for Pentabarf, one of it being the part that manages this years Assassins game. Other orga team members (hello triple des :) ) wrote a video and a volunteer controller. I guess after DebConf we will have some fun time to merge our changes back to upstream.

---

Travel in .ar Oh well, as some people know, I stay in Argentina for about 10 days more after DebConf is over (my flight back goes on 27. August, in the evening). Up to now I haven’t decided - or even started planning - what I do here. I should probably look at that soon. Hrm. Anyone out there doing something similar the next days? :) Comments: 0

For some time people have been telling me about the benefits of SpamAssassin (SA). I have installed it once for a client (at their demand and against my recommendation) but was not satisfied with the result (managing the spam folder was too complex for their users). The typical configuration of SA has it run after mail has been accepted by the server. Messages that it regards as spam are put into a spam folder. This means that when someone phones you about some important message you didn’t receive then you have to check that folder. Someone who sends mail to a user who has such a SA configuration can not expect that the message will either be received or rejected (thus giving them a bounce message). Even worse it seems to be quite common for technical users to train the Bayesian part of SA on messages from the spam folder - without reviewing them! Submitting a folder of spam that has been carefully reviewed for Bayesian training can increase the accuracy of classification (including taking account for locality and language differences in spam). Submitting a folder which is not reviewed means that when a false-positive gets into that folder (which will eventually happen) it is used as training for spam recognition thus increasing the incidence of false-positives! Spam has been becoming more of a problem for me recently, on a typical day between 20 and 40 spam messages would get past the array of DNSBL services I use and be re-sent to pass the grey-listing. Also I have been receiving complaints from people who want to send email to me about some of the DNSBL and RHSBL services I use (the rfc-ignorant.org service gets a lot of complaints - there are a huge number of ignorant and lazy people running mail servers). So now I have installed spamassassin-milter to have SA run during the SMTP protocol. Then if SA checks indicate that the message is SPAM my mail server can just reject the message with a 55x which will cause the sending mail server to generate a local bounce (if it’s a legitimate message) or to just be discard it in the case of a spam server. Here is how to set it up on Debian/Lenny and CentOS 5: Install the package yum install spamass-milter or apt-get install spamass-milter spamassassin spamc (spamassassin seems to be installed by default on CentOS). On a Debian system the milter will be setup and running. On CentOS you have to run the following commands:
useradd -m -c "Spamassassin Milter" -s /bin/false spamass-milter
mkdir /var/run/spamass-milter
chown spamass-milter /var/run/spamass-milter
chmod 711 /var/run/spamass-milter
echo SOCKET="/var/run/spamass-milter/spamass.sock" >> /etc/sysconfig/spamass-milter On CentOS edit /etc/init.d/spamass-milter and change the daemon start line to ‘runuser - spamass-milter -s /bin/bash -c "/usr/sbin/spamass-milter -p $SOCKET -f $EXTRA_FLAGS"‘ Then add the following lines below it:
chown postfix:postfix /var/run/spamass-milter/spamass.sock
chmod 660 /var/run/spamass-milter/spamass.sock The spamass-milter program talks to the SpamAssassin daemon spamd. On both Debian and CentOS run the command “useradd -c Spamassassin -m -s /bin/false spamassassin” to create an account for SA. The Debian bug #486914 [1] has a request to have SA not run as root by default. On CentOS it seems that SA wants to use a directory under the spamass-milter home directory, the following commands alllow this. It would be good to have it not do that, or maybe it would be better to have the one Unix account used for SA and the milter.
chmod 711 ~spamass-milter
mkdir ~spamassassin/.spamassassin
chown spamassassin ~spamassassin/.spamassassin On Debian edit the file /etc/default/spamassassin and add “-u spamassassin -g spamassassin” to the OPTIONS line. On CentOS edit the file /etc/sysconfig/spamassassin and add “-u spamassassin -g spamassassin” to the SPAMDOPTIONS line. To enable the daemons, on CentOS you need to run “chkconfig spamass-milter on ; chkconfig spamassassin on“, on Debian edit the file /etc/default/spamassassin and set ENABLED=1. Now start the daemons, on CentOS use the command “service spamassassin start ; service spamass-milter start“, on Debian use the command “/etc/init.d/spamassassin start“. Now you have to edit the mail server configuration, for Postfix on CentOS the command “postconf -e smtpd_milters=unix:/var/run/spamass-milter/spamass.sock” will do it, for Postfix on Debian the command “postconf -e smtpd_milters=unix:/var/spool/postfix/spamass/spamass.sock” will do it. Now restart Postfix and it should be working. For correct operation you need to ensure that the score needed for a bounce is specified as the same number in both the spamass-milter and SA configuration. If you have a lower number for the spamass-milter configuration (as is the default in Debian) then bounces can be generated - you should never generate a bounce for a spam. The config file /etc/default/spamass-milter allows you to specify the score for rejecting mail, I am currently using a score of 5. Any changes to the score need matching changes to /etc/mail/spamassassin/local.cf (which has a default required_score of 5 in Debian). You can grep for “spamd..result..Y” in your mail log to see entries for messages that were rejected. One problem that I have with this configuration on Debian (not on CentOS) is that spamd is logging messages such as “spamd: handle_user unable to find user: ‘russell’“. I don’t want it to look for ~russell when processing mail for russell@coker.com.au because I have a virtual domain set up and the delivery mailbox has a different name. Ideally I could configure it to know the mapping between users and mailboxes (maybe by parsing the /etc/postfix/virtual.db file). But having it simply not attempt to access per-user configuration would be good too. Any suggestions would be appreciated. Now that I have SpamAssassin running it seems that I am getting about 5 spams a day, the difference is significant. The next thing I will do is make some of the DNSBL checks that are prone to false-positives become SpamAssassin scores instead. When I started writing this post I was not planning to compare the sys-admin experiences of CentOS and Debian. But it does seem that there is less work involved in the task of installing Debian packages.

• [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=486914

Share This

It seems to me like basic business common sense not to attack your software’s users, because they’re people who might fund your future development work, when they want some new feature adding or some adaptation making. We should hope that this audience will become co-promoters, co-developers or co-producers of free and open source software in the future. Indeed, one of the basic lessons at our webmaster cooperative is to try to direct enquiries about software we’ve co-produced to somewhere useful, even if the question isn’t really much to do with us (because it’s about some part we didn’t produce or it’s something we’re no longer funded to work on), or even if we suspect the questioner has done something very silly. This approach has worked well so far. So, just don’t attack your users. It’s not on. Don’t do it. This basic lesson of not attacking users doesn’t seem to have been learned in software industries in general. Most software comes with a very restrictive “End User License Agreement” that seems to treat the user as if they are a potential criminal and grabs as many rights as possible for the software producer. I expect that Microsoft’s new post-Windows operating system will continue this approach and that the user-tracking possibilities of internet-centred operation will be exploited as far as legally possible. Hopefully, we’ll continue to see pre-installed Linux growing strongly in UK as Bristol Wireless describe. However, this pales into insignificance compared to many media companies. One of the worst offenders is Bristol-based TV Licensing who send us a “pay up or else, you potential criminal”-style letter each year. If there’s ever a reform that makes BBC subscription voluntary, they will probably reap the seeds of hatred that TVL have sown by attacking their audience. I have quite a lot of admiration for people like Martyn Drake who are fighting a noble fight in things like TV Licensing - the war begins! but I wonder whether media companies will learn not to attack their users before they cease to exist.

Today I figured out why the awesome window manager's tags are so eh, awesome. I'm attending debconf via irc and web stream, and have that on tag 9. Which I mostly think of a a desktop, since I generally view one tag at a time, and flip between them like desktops. But often I want to browse the web, or do something else too, without losing the video and irc offscreen. So from tag 9, I can press mod+ctrl+3 to display tags 9+3 at the same time. And after I've done this, can mod+left/right to go up and down between tags, and tag 9 stays selected and visible. Awesome automatically lays everything out.

Only thing I don't like is that if I use mod+N to switch to a specific tag, it will drop tag 9.

OpenSync update To give some update on the state of OpenSync in Debian, I have uploaded libsycml-0.4.7 to experimental a couple of days ago. This is significant in sofar as a lot of development and bug-fixing (mostly by Michael Bell) happened for this release, as well as some committment to maintaining an API and at least responsively versioning the library. In order to use libsyncml-0.4.7 with OpenSync, a newer libopensync than 0.36 is needed; however, current OpenSync trunk has seen a lot of changes in plugin handling and plugins need to get ported to the new API. So I uploaded the last known-working revision of OpenSync along with corresponing revisions of the file-sync and syncml plugins, the vformat module and a rebuild of msynctool to experimental for now. I did not have the time or energy to migrate/upload the other plugins yet, and as it seems that OpenSync-0.37 will only ship with ported file-sync and syncml plugins, it might not make much sense. I also took over maintainership of the related wbxml2 package, and upload a patch by Michael Bell which seem to fix a lot issues people are having with SyncML. The good news is that it seems all of the new features for a 0.40 stable OpenSync release have been finished according to the roadmap , most notably a common plugin configuration system and the machinery for a migration path from 0.22 to 0.40 configurations (plugins still need to support/implement that I believe), so no more big API changes are expected and the focus will be on bugfixing and plugin discovery from now on. This means developers will be able to start porting their plugins to the 0.40 API once 0.37 is out and front-end authors can start to take a look at the architectural changes which were made to facilitate their jobs. My hope is that conduit will be able to leverage the OpenSync technology and introduce a solid GUI for this (as kitchensync does for KDE), making syncronization finally work on the desktops. From the Debian packaging point of view, I have been mostly on my own now for the last couple of months. However, I recently registered an Alioth project in order to maintain the packages in a subversion repository (I have not yet decided whether it is worth importing the 0.22 packages targetted at lenny), and people who are interested in helping should contact me.

More notes and reactions from DebConf video. debbugs Hadn't realised how handy bts select is! My CVE-XXX-XXXX tags are a fairly large percentage of the 3000+ unique user tags. 2461 to be exact. :-) It would be nice to be able to view a list of all users of usertags, and their tags, in the BTS web interface. Yay! Don implemented the summary idea already.
(And I implemented bts summary during his talk.) Local partial debbugs mirror: Crazy idea, but better than bts cache. :-) (And if it provides an incentive for a current and proper debbugs deb to be maintained, that's a win.) vcs-pkg I won't bother once again repeating my responses to tired arguments like "git's archive format will change!" Been there, done that, been ignored; stopped maintaining any non-native packages; your loss. Topgit looks generally handy, and it was neat to have its author on irc during the talk. video team Hundreds of remote watchers -- impressive.. Looking forward to the remote controlled telepresence robots at debconf 10! (Also to a lack of tango loop music.) bugs in large packages Bug summaries are not yet shown in the "extra information" in the bug index page.

...at least sometimes it is. Take this gem, for example: In probability theory and statistics, the probit function is the inverse cumulative distribution function (CDF), or quantile function associated with the standard normal distribution. It has applications in exploratory statistical graphics and specialized regression modeling of binary response variables. A well-trained statistician will be perfectly able to understand the above, and perhaps fix any errors in it. However, anyone who isn't a highly trained statistician will not; and a trained statistician probably doesn't even need wikipedia to know the above. I did get some basic statistics during my last studies, and have an understanding of some of the basic terms and concepts of statistics. Yet the above makes no sense to me, at all.

As an example, consider the familiar fact that the N(0,1) distribution places 95% of probability between -1.96 and 1.96, and is symmetric around zero. It follows that

... Or something. Suuuure. (posted belatedly because my server was down yesterday)