A few days ago, I discovered From Punched Cards To Flat Screens, the "technical memoirs" of Philip Hazel (of exim and PCRE fame). This document, by itself a very good read, mentioned something about him having written a piece of software for the Acorn RISC OS platform, starting in the early 90s, to typeset music based on a text file; very much the way in which lilypond operates. The document also mentioned that after the demise of Acorn's platform, Philip had ported that application, PMW, to the unix platform. Being interested in music, this had me curious, so I did a bit of investigation. Lilypond is an interesting piece of software, but as I've written before, I do have a few gripes with it. Once you get beyond the basics, the lilypond syntaxis is somewhat counter-intuitive in my opinion, and there are a number of things that are just wrong with it. Upon reading through the first few examples, I quickly learned that most of the major issues that I have with lilypond's syntaxis simply do not exist with PMW, and that it also has quite a number of features which lilypond in its (in the mean time) fairly long history, still does not support. And while lilypond does have a slightly higher amount of output options, that simply does not weigh against the much higher processing speed and much more sensible input syntaxis of PMW. I've made a short comparison of the two, which shows some of their differences, and contains some notes on my opinion of the two. This table may grow in the future... I've only been playing with PMW for a few days now, but I've already decided that I'm going to ditch Lilypond.

Third week and I still have picture to publish. It was taken at Continental cup in nordic combined at Harrachov.

Filed under: English Photography 1 comments Flattr this!

Since waiting for long running builds is boring and having to check if the build already finished distracts one from doing other stuff, git-buildpackage 0.5.16 reports the build status of a package via desktop notifications if python-notify is installed: To disable this feature use:

[git-buildpackage]
notify = off

in your ~/.gbp.conf. This blog is flattr enabled.

The CRANberries service (which reports on new and updated CRAN packages for the R language and environment) is now tweeting about new packages. Simply follow @CRANberriesFeed to receive theses messages. For the technically minded, adding this to the existing 200-line program which runs all of CRANberries was very easy. CRANberries relies only on R itself and a few Unix tools like diffstat as well as the simple blosxom txt-to-html/rss 'blog compiler'. The tweeting itself is now done by this new function

tweetNewBlogEntry <- function(curPkg, curVer, reposurl)  
    ## tests reveal that pipe(), cat(), close() is easiest
    ## to send multiple messages, may need --background option
    con <- pipe("bti --config bti.conf", "w")
    cat("New CRAN package", curPkg, "with initial version", curVer, " http://goo.gl/pgljT\n", file=con)
    close(con)
 

which simply pipes the message in Greg KH's bti program (which is now a new dependency). Special thanks to its Debian maintainer Gregor Herrmann for some helpful emails; I am using the newest release 0.29 which itself needs liboauth0. Once OAuth tokens are set-up (and see here for how to do that) all we need is the three-liner above. At this point I am not too sure what to do about updated packages. One message per updated package seems too noisy. To be seen---comments or suggestions welcome.

It will be my first time there, I hope I'll manage to meet lots of people I only know through mailing-lists. See you there!

Yesterday afternoon, I was at the first OrgCamp in Paris. It was my first attendance to a BarCamp, and I really liked it. It's basically the first geek event I do not find boring nor useless. There was about 18-20 persons participating, which was quite high, since we all initially though we would have been only 5. We had several presentations of various features and personal usages of Org-mode. For my part, I've quickly presented the agenda, and my BBDB replacement named org-contacts (I'll probably talk about it on this blog in another post later). The only downside was that Bastien (the new Org-mode maintainer) was not able to come and join us. On the other side, there were so much to tell for a first time, I did not have so much time to code. I only have been able to fix one bug reported during my agenda presentation. In the end, the overall atmosphere was very enthusiastic and friendly, which was extremely pleasant. The #org-mode-fr IRC channel has been created on Freenode, following this event. Feel free to join us. Since people liked it so badly, it seems there should be another barcamp in the next months. Stay tuned.

James Purser wrote Stop Using Autism to Push Your Own Brand of Lunacy about the organisations that use Autism for their own political agenda [1]. He references the ongoing vaccine issue (which didn t even stop when Andrew Wakefield was proven to have engaged in fraudulent research) and a campaign of PETA.PETA s ClaimsPETA claims that many autistic kids improve dramatically when put on a diet free of dairy foods [2]. They also speculate about the possibility of Autism being caused by dairy.One thing that confuses the issue is the impact of stress on people with Autism Spectrum Disorders (ASDs). Whatever you are trying to do you will probably do it less effectively if under an excessive amount of stress, and the background stress levels of people on the Autism Spectrum tend to be quite high at the best of times. So add in some extra stress from a food allergy and the ability to emulate an NT can decrease a lot.There are people who make credible claims about significant behavior improvements in Autistic children after changing their diet. Eliminate foods that trigger a minor allergic response and the behavior of NT kids will improve too.I m sure that there are some Autistic kids who have a minor intolerance to lactose or casein who s behavior would improve if they had no dairy products. There are probably just as many Autistic kids who have a minor allergy to peanuts who would get the same benefit from avoiding peanut butter. It s not inconceivable that some Autistic people may be allergic to soy or wheat and find that they can cope better if their consume more milk, cheese, and steak and less vegetables!Also some people on the Autism Spectrum have a better sense of smell than NTs, and it s fairly common to be more fussy about the appearance of food. Force children to eat food that smells bad and their behavior might deteriorate.The issues of diet and Autism are more difficult than those of vaccines because while people just imagine the vaccines to have a bad affect, there are people who are proven to get a bad affect from certain foods.Some Ideas for ResearchI think that the biggest problem with PETA s research in this regard is citing a study of only 20 individuals and only testing the removal of milk. A proper study would involve more subjects (maybe 100) and separate tests of removing milk, eggs, wheat, rice, seeds, nuts, soy, fish, spices, and food colorings and additives.It would also test various methods of reducing stress, having a more quiet environment etc.PETA do deserve some credit for leaving this URL in the comments [3], it s a study that disproves the link between milk and Autism. But it s still a small sample size of 14.Why They Target AutismIn spite of psychologists moving to define Asperger Syndrome and Autism as the same thing, the people who make such claims aren t talking about people like me. They want to make claims about people who can t refute them, such as people who fit the Rain Man stereotype. The Low Functioning Autistic (LFA) people have less ability to communicate and are easier to ignore, so if you want to make some stuff up about a group of people then they are an easy target.Parents of LFA kids are often desperate and will latch on to any idea that seems to help. They can imagine that things have improved in spite of a lack of evidence and also fail to realise that children can improve rapidly just due to their natural development, 1 month is not much time for an adult but for a 4yo it s 2% of their life!Also any supposed research that includes testimony of parents should be regarded as rubbish. If you talk to any child psychologist they will tell you about how difficult it is to convince parents that their child is on the Autism Spectrum, a Google search for autism + denial also brings up plenty of hits. Parental denial doesn t stop at refusing to accept a diagnosis, it includes wanting to reject it at any later time based on any shred of evidence that may be available. So when there is a diet change or some other supposed cause followed by some apparently better behavior they want to claim that Autism is cured.I ve written a document titled Some Suggestions for Parents of Aspies [4] which is largely based on the childhood memories and adult experiences of various Aspies. My document covers many issues and few kids on the Spectrum will benefit from all of them, but I think that there is something there that can benefit every child on the Spectrum and probably a lot of geeky NT kids too. It seems to me that when dealing with Autistic kids it s a lot better to seek advice from adults on the Autism Spectrum than from random Neuro-Typical people (NTs), particularly if those NTs happen to have an agenda to push. The sensible parents are the ones who seek advice from psychologists and people like me rather than from random unqualified NTs.

• [1] http://jamespurser.com.au/blog/stop-using-autism-push-your-own-brand-lunacy
• [2] http://www.peta.org/features/got-autism-learn-about-the-link-between-dairy-products-and-the-disease.aspx
• [3] http://www.reuters.com/article/idUSTRE64I6EZ20100519
• [4] http://doc.coker.com.au/aspie/suggestions-parents-aspies/

In the discover-data package in Debian, there is a script to report useful information about the running hardware for use when people report missing information. One part of this script that I find very useful when debugging hardware problems, is the part mapping loaded kernel module to the PCI device it claims. It allow me to quickly see if the kernel module I expect is driving the hardware I am struggling with. To see the output, make sure discover-data is installed and run /usr/share/bug/discover-data 3>&1. The relevant output on one of my machines like this:

loaded modules:
10de:03eb i2c_nforce2
10de:03f1 ohci_hcd
10de:03f2 ehci_hcd
10de:03f0 snd_hda_intel
10de:03ec pata_amd
10de:03f6 sata_nv
1022:1103 k8temp
109e:036e bttv
109e:0878 snd_bt87x
11ab:4364 sky2

The code in question look like this, slightly modified for readability and to drop the output to file descriptor 3:

if [ -d /sys/bus/pci/devices/ ] ; then
    echo loaded pci modules:
    (
        cd /sys/bus/pci/devices/
        for address in * ; do
            if [ -d "$address/driver/module" ] ; then
                module= cd $address/driver/module ; pwd -P   xargs basename 
                if grep -q "^$module " /proc/modules ; then
                    address=$(echo $address  sed s/0000://)
                    id= lspci -n -s $address   tail -n 1   awk ' print $3 ' 
                    echo "$id $module"
                fi
            fi
        done
    )
    echo
fi

Similar code could be used to extract USB device module mappings:

if [ -d /sys/bus/usb/devices/ ] ; then
    echo loaded usb modules:
    (
        cd /sys/bus/usb/devices/
        for address in * ; do
            if [ -d "$address/driver/module" ] ; then
                module= cd $address/driver/module ; pwd -P   xargs basename 
                if grep -q "^$module " /proc/modules ; then
                    address=$(echo $address  sed s/0000://)
                    id=$(lsusb -s $address   tail -n 1   awk ' print $6 ')
                    if [ "$id" ] ; then
                        echo "$id $module"
                    fi
                fi
            fi
        done
    )
    echo
fi

This might perhaps be something to include in other tools as well.

I've been doing a bit of work recently on the netcfg portion of Debian Installer, with the largest work so far being adding IPv6 support. Unlike most of the rest of d-i, netcfg is written in C, and as such suffers from some difficulties in the realm of debugging. Worse, the whole d-i environment is rather bereft of the usual debugging tools (strace, lsof, etc) -- but most importantly, in this case, gdb. Thanks to some hints from Julien Cristau, I was reminded of the existence of gdbserver, a small part of gdb that allows you to debug a program running on a remote machine, using a copy of gdb running on your local machine. It's designed for debugging embedded systems, I think, because all of the heavy lifting (symbol lookups, etc) happens on your local machine, not where gdbserver is running. Here are the steps I took to get a useful gdbserver session inside d-i:

1. Get the gdb sources (apt-get source gdb) and drop into the gdbserver tree (cd gdb-x.y.z/gdb/gdbserver)
2. Configure gdbserver to build a static binary (LDFLAGS=-static ./configure). This is necessary because you don't want to have to faff around with library dependencies inside the d-i environment. Then build gdbserver (make).
3. Copy gdbserver over to your d-i machine/VM (I like using netcat; nc -l -p 31310 >/bin/gdbserver on the d-i machine, and nc IPADDR 31310 <gdbserver where you built gdbserver), then make it executable (chmod /bin/gdbserver)
4. Fire up gdbserver in d-i, attaching it to the process you wish to debug: gdbserver host:31337 --attach PID
5. Now things get a little trickier... on your local machine, you need to have an unstripped copy of the binary you're debugging. For me, this is easy, because I'm building netcfg myself (to keep the symbols, I just run DEB_BUILD_OPTIONS=noopt dpkg-buildpackage -b -rfakeroot) but I'm not sure what you'd do if you were just tracing around for fun.
6. Once you've got your binary, just run gdb BINARY on your host machine to start gdb running.
7. For the "even trickier" part of the show, your local machine also needs copies of all the libraries your binary is linked against. If you don't have this, when you try to debug you'll get all sorts of "wrong library or version mismatch?" errors, and your backtraces will likely break (amongst other things). Again, for me, this is easy, because I'm only running the d-i initrd, but if you're debugging something later in the installation process you might need to unpack some udebs somewhere. All I have to do is keep the initrd build tree around (which lives in the installer tree, under build/tmp/netboot/tree). Once you've got your tree of libraries, you can continue.
8. You need to tell gdb about that tree of libraries, by running the following in gdb: set solib-absolute-prefix /path/to/library/tree
9. Now, tell gdb to connect to your gdbserver: target remote IPADDR:31337 -- gdb should tell you "Remote debugging using IPADDR:31337", and the gdbserver should say "Remote debugging from host LOCALIP:31337".
10. Now, you should be able to interact with the program being debugged just like any other. Enjoy!

For some years now I have kept a private journal, sort of a private blog. It is mainly related to my work and hobby development activities, though some private life gets poured into it as well. It's a private journal. It is not meant for others to read. It has a dual purpose in life:

• a venue for me to develop thoughts, rant, express feelings, and what not, as uncensored as possible
• a way to go back and check what I was thinking when I made a particular stupid thing

I find this journalling activity to be beneficial to software development. Since I mainly work alone, at home, it is a bit like having someone to talk to across the desk. However, since it is recorded, it is more practical to grep things for useful stuff later on. Re-reading entries also gives me a wonderful sense of how fast or slow things happen. For example, on January 23 last year, exactly one year ago tomorrow, I wrote:

That concludes my list of next actions that should be done before obnam is theoretically able to do normal backups.

I then spent most of last year actually making Obnam be capable of doing backups for real. Recently, I thought that it might be interesting for some to read another programmer's thought processes when designing, writing, and debugging code. For this reason, I've collected some of my journal entries from the past year or so, those related to Obnam and my Python B-tree library (which Obnam uses). I have lightly sanitized them, to protect other people's privacy. Other than that, it's me as raw as you could wish. The dump is on my website. Please be gentle and laugh at me behind my back.

My paternal grandmother died a month ago today - I may have been quieter than usual since December. The most vivid memory I have of her is helping to collect runner beans from her garden - apparently this had been going on since the World War II "dig for victory" campaign. Her funeral was in Hastings on 10th January, and she was buried in Fairlight, in the Roman Catholic area of St. Andrew's - it was a very peaceful setting.

Do not waste my hard disk! I recently acquired an 2 TB hard disk drive, which I immediately formatted with ext4. Given that mkfs.ext4 defaults to 5% reserved blocks for too, that amounts to 100 GB of lost* space. Wow. 100 GB. On a desktop machine. For root. While mkfs belongs to e2fsprogs, I think this requires action on the distributions side. Let me explain. If you are installing a server, 5% makes perfect sense: there are a lot of logfiles, updates, potential break-ins and other disasters-to-happen for which root-reserved space will be preciously required. 5% is a good choice here. If 5% is too much (or too little), it will not matter: any decent sysadmin knows mkfs.ext4 -m and tune2fs -m (if you don t, Martin will teach you). On the other hand, if you are installing Ubuntu on your laptop, you probably don t know shit about mkfs, tune2fs, 5%, etc. You only know by removing Windows and installing Linux you ve magically lost 100GB of space. Vanished. So here is my request to Linux distributions: when installing a distribution in desktop profile (i. e. not Debian in server profile, RHEL, SLES, Ubuntu Server, etc), DO NOT RESERVE 5% to root. 100 or 200 MEGABYTES should be more than enough. * Yes, I know it s not really lost but just unusable by normal users but that technicality doesn t matter to 99% of the people out there.

Kernel 2.6.38 will add support for tracing ASoC using trace points. Previously all logging for ASoC had been done using printk(), meaning that changing the active logging required a kernel rebuild and that when trace was enabled the volume of trace could easily become very disruptive to other logging within the system. Trace points solve these problems by providing a framework for enabling and disabling individual traces dynamically and separate logging infrastructure designed for easy filtering and post-processing. When using trace points it s reasonable to leave them enabled all the time, making life much easier especially when debugging intermittent problems. There s a bunch of existing documentation out there for the trace subsystem, including some in the kernel under Documentation/trace, which I won t repeat here. The ASoC events are grouped together under asoc . The raw log can be viewed by looking at the file tracing/trace in debugfs. Here s a sample output from starting playback of an MP3 on a Samsung SMDK6410 reference system using WM8580, a high performance six channel CODEC with a very simple software interface:

mplayer-1673  [000]   183.920000: snd_soc_dapm_start: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Rear val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Center+Sub val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=Front val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT3L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT2L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1R val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=VOUT1L val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC3 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC2 val=1
mplayer-1673  [000]   183.920000: snd_soc_dapm_widget_power: widget=DAC1 val=1
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=2
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=32 val=1e
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=32 val=2
mplayer-1673  [000]   183.920000: snd_soc_bias_level_start: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_bias_level_done: card=SMDK-I2S val=3
mplayer-1673  [000]   183.920000: snd_soc_dapm_done: card=SMDK-I2S
mplayer-1673  [000]   183.920000: snd_soc_reg_read: codec=wm8580-codec.0-001b.27 reg=13 val=10
mplayer-1673  [000]   183.920000: snd_soc_reg_write: codec=wm8580-codec.0-001b.27 reg=13 val=0

Each line represents a trace event, showing the process that initiated the event, the CPU it ran on and the time it was recorded at together with event-specific details. On this system the time is reported with 10ms accuracy and the CODEC is very simple so the timing information is not terribly informative, but more accurate timers can be really helpful in analysing what s going on with more complex CODECs. The set of events available will change over time, the key ones currently present in the kernel are:

• snd_soc_dapm_start and snd_soc_dapm_done: These events show the start and stop of the DAPM algorithms, showing the time spent on power management decisions.
• snd_soc_dapm_widget_power events are generated for each of the widgets which DAPM has determined needs to change state. The val=1 means that all these widgets are being powered up, val=0 would show power down.
• snd_soc_reg_write events show values written to registers.
• snd_soc_reg_read events show values read from registers these reads may be done from the register cache, or may be from the hardware.
• snd_soc_dapm_widget_event_start and snd_soc_dapm_widget_event_done show the time spent in per-widget event callbacks.

The set of events traced will change as the subsystem develops and as people gain more experience with using the trace API.

If you convert a docbook document in a chunked HTML form, it would presumably be very useful for you to place one of the various available javascript based trackers on each of your pages to see what is most popular, linked to etc. I'm assuming you've already got to the point where you have your DSSSL stylesheet going (see something like my prior entry). It doesn't appear you can get arbitrary output into the <head> tag -- the %html-header-tags% variable is really just designed for META tags and there doesn't appear to be anything else in the standard stylesheets to override. So the trick is to use $html-body-start$. But you have to be a little bit tricky to actually get your javascript to slip past the SGML parser. After several attempts and a bit of googling I finally ended up with the following for a Google Analytics tracker, using string-append to get the javascript comment filtering in:

I ve been lasy about the reporting lately, so this is a 3 weeks update at once ! The bug count according to udd is down to 818. I guess this is due to Kibi s triaging on input related bugs, but I helped to close a few one too ! I kept my week numbering as before, so now I know the first bugs I pinged are more that 11 weeks old. I guess if some submitter did not answer in that delay, they will probably never do it. I think the bug count will go down again !

• #451708 ping & close xserver-xorg-input-synaptics: Option GuestMouseOff true not working.
• #424743 ping & forwarded X11 pen driver doesn t work on tabletPC Fujitsu Stylistic 2300 (fix included).
• #499664 ping, try nouveau X fails to start with xserver-xorg-video-nv 1:2.1.12-1.
• #512614 confirmed & upstream xset dpms rejects reasonable values as illegal.
• #512711 ping and later closed All keypresses repeated roughly three times each, makes login impossible.
• #513128 ping X server crashes every time when gnome-screensaver starts.
• #486356 merge with #482592 numlock led is inversed (using numlockx) .
• #515737 ping & closed After upgrading from Etch to Lenny, mouse pointer is very slow and keys don t repeat.
• #515840 ping Desktop is not using full screen after upgrade from etch to lenny.
• #516860 closed after looking at the git history Xorg with -sharevts use almost 100% of the CPU.
• #362434 ping & closed, suggested to try nouveau nv has problems with xcursor-themes.
• #502131 ping & closed System gets into a state and fails to process modifier keys.
• #453296 closed (old ping unanswered) MacBook: cannot assign right mouse click to lower Enter key.
• #504537 ping X server eats CPU on sparc.
• #502313 ping Applications are getting ghost Alt keypress events, but I press no keys.
• #527118 ping autoadded input devices kill X server.
• #270887 forwarded startx and xinit do not preserve client arguments.
• #542542 closed after looking at the git history non-VGA compatible graphics devices won t run X.
• #492888 ping xorg ignores keystrokes.
• #543210 merge with #492783 Alt-F7 to return to X, passes F7 key to top focus d window.
• #524413 closed xauth fails with xorg 7.4, this was actually an xinit bug and the git log lead to #549377 being closed

The X Strike Force (still) needs you ! You can have a look at the X Strike Force Bug Closing Procedure and check XSF unstable bugs sorted by date.

Joey has proposed a new concept of Continuously Usable Testing for Debian [1], basically testing should be usable at all times and packages that aren t usable should be dropped. But to properly achieve this goal we need continual testing of usability.The Plan For SE LinuxTo do this for SE Linux I m setting up a Xen server which will have a number of different DomUs for testing a variety of server applications. The system has 1.5G of RAM and 160G of mirrored storage. An image of a typical server will take about 4G of disk space, so we could have something like 40 images online and ready for testing. I have already setup Squid on another system on the same LAN to cache Debian packages, so running apt-get dist-upgrade on a number of DomUs won t take that long. With 256M for a typical server image I could have 5 images running at the same time. If the hardware isn t enough then I can expand it, I hope to get some donations of DDR-266 or DDR-333 RAM (or maybe DDR-400) to upgrade the system to 4G of RAM, I can add more hard drives, and I could even install more servers.I want to have testing be very usable for SE Linux throughout the development cycle so that I don t have to rush things before release.At this stage I m not sure whether to track Unstable or Testing for this. I guess it might be best to track Testing most of the time and only track Unstable for daemons that are changing rapidly. It might get boring testing every version that comes through Unstable, but if people want to do this then I won t stop them.Setting up the TestsWhat I need are interested people who want to install server configurations for testing. If you have some favorite combination of daemons that you want tested for SE Linux support (even if it s daemons that have no current policy) then I can give you root access to a DomU to develop test cases. Ideally there would be automated tests used for most things for example testing a mail server by using swaks to deliver mail and a POP or IMAP client script to retrieve it. But some things can t be tested properly without human intervention.For the automated tests I want to script the creation of DomUs, upgrading the packages in the DomU, testing it, and then shutting down the DomU if it all works. If at any time the tests fail (or the upgrade fails) then it would wait for human intervention. That would be me fixing SE Linux problems and other people fixing the application problems. I think that discovering SE Linux issues will only be a part of this project.For the manual tests I will grant access to create and destroy the DomUs in question to people who can run the tests.I m thinking of having a couple of DomUs running permanently for things which are test candidates but also useful for the project, such as a MediaWiki instance. It really depends on the interest of people who might use such things.Also I m thinking of setting up some Ubuntu DomUs too, I probably should join Ubuntu and get involved with SE Linux there.Sharing the ImagesI have a web server in Germany with almost unlimited bandwidth and storage. For every image that is created I want to upload a version to the server in Germany to allow anyone in the world to test it. There are lots of possible ways of using this for software development. For example if you had a patched version of Apache that you wanted to test then you could download every image that had Apache installed and test that they all work. That would be easier than configuring Apache in different ways and also possibly provide better coverage.Also if someone can t figure out how to configure a daemon correctly then downloading a Xen image of a working configuration could be helpful (if a little bandwidth intensive). Note that deploying such an image in production would be a really bad idea, among other things there are lots of places where passwords are stored and you wouldn t want to risk missing one.I also plan to share the scripts used in running the Dom0 and anything else that seems useful along the way.What We NeedThe main thing we need is volunteers to configure virtual machines with their favorite daemons. Note that I don t plan to have only one daemon per DomU, if we can get multiple daemons running that don t conflict (EG file server and mail server) or multiple daemons that can interact (EG database server and a mail server or anything else that can be a database client) running on the same system then that s a good thing. So there will be some degree of interaction with other people.I m happy to accept contributions from people who aren t interested in SE Linux. But SE Linux will run on all DomUs.Finally I also need more RAM for a HP D530S, DU875PA (that s a Celeron 2.4GHz). I ll accept donations of complete systems too once my HP system gets full, preferably relatively low power systems as they will be housed in a location that s not as well ventilated as I would like (cost and availability of IP addresses were the main criteria). A laptop with a broken screen would be great!The system won t go live until Monday, but I think that probably people won t be ready to do much work with less than two days notice anyway.

• [1] http://kitenet.net/~joey/code/debian/cut/

I discovered, in not a very amusing way that if you use query_posts() then the variables for paginating with The Loop stop working. Fixing this is easy, just add to paged option on the array the variable $paged, in this way:

query_posts(
    'posts_per_page'    => 15,
    'paged'                 => $paged,
);

Or if you don t use it like this, then:

query_posts('posts_per_page=15&paged=' . $paged);

After this, using posts_nav_link() actually works.

Match package names across distributions What would happen if we had a quick and reliable way to match package names across distributions? These ideas came up at the appinstaller2011 meeting:

• it would be easy to lookup screenshots in the local distro, and if there are none then fall back on other distributions;
• it would be easy to port Debtags to other distributions, and possibly get changes back;
• it would be trivial to add a [patches in $DISTRO] link to the PTS
• it would be easy to point to other BTSes

We thought they were good ideas, so we started hacking. To try it, you need to get the code and build the index first:

git clone git://git.debian.org/users/enrico/distromatch.git
cd distromatch
# Careful: 90Mb
wget http://people.debian.org/~enrico/dist-info.tar.gz
tar zxf dist-info.tar.gz
# Takes a long time to do the indexing
./distromatch --reindex --verbose

Then you can query it this way:

./distromatch $DISTRO $PKGNAME [$PKGNAME1 ...]

This would give you, for the package $PKGNAME in $DISTRO, the corresponding package names in all other distros for which we have data. If you do not provide package names, it automatically shows output for all packages in $DISTRO. For example:

$ time ./distromatch debian libdigest-sha1-perl
debian:libdigest-sha1-perl fedora:perl-Digest-SHA1
debian:libdigest-sha1-perl mandriva:perl-Digest-SHA1
debian:libdigest-sha1-perl suse:perl-Digest-SHA1
real    0m0.073s
user    0m0.056s
sys 0m0.016s

Yes it's quick. It builds a Xapian index with the information it needs, and then it reuses it. As soon as I find a moment, I intend to deploy an instance of it on DDE. It is using a range of different heuristics:

• match packages by name;
• match packages by desktop files contained within;
• match packages by pkg-config metadata files contained within;
• match packages by [/usr]/bin/* files contained within;
• match packages by shared library files contained within;
• match packages by devel library files contained within;
• match packages by man pages contained within;
• match stemmed form of development library package names;
• match stemmed form of shared library package names;
• match stemmed form of perl library package names;
• match stemmed form of python library package names.

This list may get obsolete soon as more heuristics get implemented. Euristics will never cover all corner cases we surely have, but the idea is that if we can match a sizable amout of packages, the rest can be somehow fixed by hand as needed. The data it requires for a distribution should be rather straightforward to generate:

1. a file which maps binary package names to source package names
2. a file with the list of files in all the packages

For example:

$ ls -l dist-debian/
total 39688
-rw-r--r--  1 enrico enrico  1688249 Jan 20 17:37 binsrc
drwxr-xr-x  2 enrico enrico     4096 Jan 21 19:12 db
-rw-r--r--  1 enrico enrico 29960406 Jan 21 10:02 files.gz
-rw-r--r--  1 enrico enrico  8914771 Jan 21 18:39 interesting-files
$ head dist-debian/binsrc 
openoffice.org-dev openoffice.org
ext4-modules-2.6.32-5-4kc-malta-di linux-kernel-di-mipsel-2.6
linux-headers-2.6.30-2-common linux-2.6
libnspr4 nspr
ipfm ipfm
libforks-perl libforks-perl
med-physics debian-med
libntfs-3g-dev ntfs-3g
libguppi16 guppi
selinux selinux
$ zcat dist-debian/files.gz   head
memstat etc/memstat.conf
memstat usr/bin/memstat
memstat usr/share/doc/memstat/changelog.gz
memstat usr/share/doc/memstat/copyright
memstat usr/share/doc/memstat/memstat-tutorial.txt.gz
memstat usr/share/man/man1/memstat.1.gz
libdirectfb-dev usr/bin/directfb-config
libdirectfb-dev usr/bin/directfb-csource
libdirectfb-dev usr/include/directfb-internal/core/clipboard.h
libdirectfb-dev usr/include/directfb-internal/core/colorhash.h

interesting-files and db are generated when indexing. To prove the usefulness of the idea (but does it need proving?), you can find in the same git repo a little example app (it took me 10 minutes to write it), that uses the distromatch engine to export Debtags tags to other distributions:

$ ./exportdebtags fedora   head
memstat: admin::benchmarking, interface::commandline, role::program, use::monitor
libdirectfb-dev: devel::lang:c, devel::library, implemented-in::c, interface::framebuffer, role::devel-lib
libkonqsidebarplugin4a: implemented-in::c++, role::shared-lib, suite::kde, uitoolkit::qt
libemail-simple-perl: devel::lang:perl, devel::library, implemented-in::perl, role::devel-lib, role::shared-lib, works-with::mail
libpoe-component-pluggable-perl: devel::lang:perl, devel::library, implemented-in::perl, role::shared-lib
manpages-ja: culture::japanese, made-of::man, role::documentation
libhippocanvas-dev: devel::library, qa::low-popcon, role::devel-lib
libexpat-ocaml-dev: devel::lang:ocaml, devel::library, implemented-in::c, implemented-in::ocaml, role::devel-lib, works-with-format::xml
libgnutls-dev: devel::library, role::devel-lib, suite::gnu

Just in case this made you itch to play with Debtags in a non-Debian distribution, I've generated the full datasets for Fedora, Mandriva and OpenSUSE. Others have been working on the same matching problem. After we started writing code we started to become aware of existing work:

• whohas
• PackageMap
• Equivalent-Packages, statistically generated from package contents, more info in this post

I'd like to make use of those efforts, maybe to cross-validate results, maybe even better as yet another heuristics.

Cross-distro Meeting on Application Installer I have been to a Cross-distro Meeting on Application Installer which to the best of our knowledge is also the first one of its kind. Credit goes to Vincent Untz for organising it, to OpenSUSE for hosting it and to the various sponsors for getting us there. It went surprisingly well. We got along, got stuff done, did as much work as possible to agree on as many formats, protocols and technologies as we possibly could. The timing of it is very important, as most major distros would like to adopt some of the features that just became popular in the various new app markets and stores, such as screenshots, user comments and ratings. It looks like a lot of new code is about to be written, or a lot of existing code is about to gain quite a bit of popularity. For my part, I presented the work on Debtags and apt-xapian-index. With regards to Debtags, other distros seem to be missing a compehensive classification system, and Debtags is, well, it. With regards to apt-xapian-index, we just noticed that it's the perfect back-end for what everyone would like to do, and the index structure is rather distribution-agnostic, and it's been road-tested with considerable success by at least software-center, so it attracted quite a bit of interest, and will likely attract some more. Just to prove a point I put together a prototype webby markety appy thing in just a few hours of work. The meeting was also the ideal place to create a joint effort to match package names across distributions, which means that a lot of things that were hard to share before, such as screenshots, tags and patches, are suddenly not hard to share anymore.

A prototype webby markety appy thing What better way to introduce my work at an Application Installer meeting than to come with a prototype package browser modeled after shopping sites developed in just a few hours? It's a little Flask webapp that just works on any Debian system, using the local apt-xapian-index as a backend. It has fast keyword search, faceted navigation and screenshots, and it runs on your system showing the packages that you have available. To try it:

git clone git://git.debian.org/users/enrico/pkgshelf.git
cd pkgshelf
./web-server.py
Then visit http://localhost:5000

It didn't have much interface polishing, as it's just a quick technology demo. However you can see that:

• keyword search is fast (fast enought that it could be made to search as you type);
• relevant tags appear on the left, grouped by facets;
• the most relevant tags are highlighted;
• the less relevant tags could be hidden behind a [more] expander;
• you can choose several strategies to hide packages you may find irrelevant.

Things that need doing:

• hiding uninteresting facets;
• making it pretty.

It's essentially JavaScript and CSS work. Anyone wants to play?