A couple of days ago, I commented about my thoughts on finding a better way to listen to music, and asked for suggestions. I ve checked out the avenues suggested, and here are my thoughts so far. Streaming service: Spotify Of the streaming service offerings, Spotify seems the most compelling. It has both Linux and Android clients. The Linux client is unofficial and a bit buggy, but serviceable. The service is fairly nice, and the serendipity of listening to radio of tracks like any track or artist I may pick is an incredible feature. It will mean a different mindset; rather than carefully choosing what I buy, I have a vast catalog at my fingertips. I d have to give up some control, but might be better off for it. Spotify can play local files, but sadly lacks support for FLAC that most of my collection is in. A little work with the shell, and I had a quick multi-core FLAC to MP3 conversion done. It can also identify local files and match them up with its catalog, but it does a somewhat poor job of it, despite the fact that my files are tagged by Musicbrainz. There are also a fair number of albums in my collection (from Magnatune and from local artists) that aren t in Spotify s catalog and probably never will be. I may wind up using git-annex or something like it to sync them between PCs. Surprisingly, it doesn t have any kind of web-based player available. It has a lot of features, but many of them are under-documented. Subsonic / Supersonic The idea of being able to stream music anywhere certainly has appeal, and the idea of streaming that music from my own machine my own collection is quite the appealing one. Subsonic seems to be the standard that people recommend for this. It has a web-based player that works well, as well as mobile clients for various platforms. The Android mobile client is particularly well-regarded and is probably the best one here, aside from Spotify. The playlists work well, as expected, and generally it has the appearance of a well-rounded system. There is a lot of flexibility in bitrates; maximum bitrates can be set per-user and per-client. It, of course, transcodes on the fly. My chief gripe about Subsonic is that it has no true album/artist/genre browser. Its main browser is mostly a filesystem browser. I have used clients that show album/artist/genre panes or sortable lists, based on tags, for so long that my filesystem organization is rather different centered around bitrate of rip, origin, etc. It is not very easy to navigate my collection in that way. The search feature can search artist and album, and this can partially make up for it. But there is simply no way to see a list of all artists in the collection. The server is written in Java, and takes up 362MB at startup. That s, I guess, decent for a Java server, but doesn t make me pleased as I d be running it on lower-end hardware. Supersonic is a git-maintained actively synced subsonic fork which removes the license cost (the source is GPL3 anyhow so that wasn t a big deal). Ampache Often mentioned in the same breath as Subsonic, this is yet another frequent suggestion for streaming your own collection. Its primary player is web-based, like Subsonic. It has full artist/genre browsers, and adds tag cloud and various other options as well by default, it loaded the tag cloud with my genre tags, which was a nice touch. The search feature is a little less robust, as it simply searches anywhere and returns a list of all matching tracks, rather than listing matching albums and artists before going into the list of all tracks. But that is somewhat of a minor nit. Ampache s web-based player, liks Subsonic s, uses Flash, but the implementation is really quite awkward. You can t simply click a play button and have playback start. Rather, you must add things to the playlist (queue), a frame on the web interface. Then, you must click play in THAT frame, which launches a new window for the player and transfers its contents to it. You thus have two separate running playlists. And if you think you can just add to the playlist of the player window, think again; you can only replace it. It is simply not possible to add a track to the end of that list without losing your existing place. This makes the usefulness of the web-based player rather, well, low. Playlist functionality in Ampache is rather odd. You can have playlists, but if you want to add a track to a saved playlist, first you have to clear you current playlist, then add the track to it, then save it to the existing playlist. Frontends for Ampache exist in banshee and in amarok. Banshee s is broken, and simply silently fails to do anything. Unless you look in the console, in which case you ll see a .NET backtrace. Banshee has had these mysterious errors for me for years. The Amarok plugin is so basic that you can browse by artist but not by album, and is almost unusable. There is also a rhythmbox plugin in Debian, but it is apparently for an old version of Rhythmbox and no longer works. The best frontend is a dedicated Ampache frontend called Viridian. It has a nice interface, provides features such as taskbar integration, and generally works pretty well. The only thing it s bad at is playlists, and here it s even worse than the web interface. It doesn t support modifying the server-side playlists at all. And accessing one is an annoying load playlist operation, that replaces your current play queue (also called the playlist ) with its contents by default. Several Android clients exist for Ampache, though it seems only one has seen any updates since 2010. mpd This is an interesting program, as it is designed more to provide multiple points of control for a single player than it is to stream a single collection to multiple devices. Nevertheless, I tested its HTTP and icecast stream modules. (It can also stream with PulseAudio, but PulseAudio causes devious breakage on my workstation, so I refuse to bother.) This could be made to work reasonably well on my LAN using the gmpc client. There are several Android clients as well, though none of them integrate as well as Subsonic or Spotify. Most seem focused on controlling a remote player. The ones that do stream do so with some hiccups, and don t integrate controls with the lock screen. gmpc is a reasonable control program and, combined with mplayer, makes a decent player. It can browse the database every way I d want to. It doesn t let me right-click on tracks to add them to playlists, but I can use copy/paste to do so, or add them to the play queue and from there to a playlist. It s annoying, but not completely unworkable. But here s the real concern. mpd has no provision whatsoever for any sort of encryption such as SSL. If I want to be listening to my music collection from on the road, I don t really wish to open up a streaming music service for the entire Internet. Nor do I really want to mess with SSH tunnels and the like, which are cumbersome on Android (and can be in general). This is pretty much a fatal flaw to me. Google Music An interesting service, but ultimately one I don t see the point for in my situation where I already have my music on a PC that is always on. It would probably accommodate my entire collection free, but how many weeks would it take me to get it there? And how would it stay in sync? It simply doesn t offer me any compelling advantage over streaming direct from my PC, so I don t see much need to bother with it. Others I briefly considered Streeme (poor mobile support), Audiogalaxy (no Linux support), Jinzora (appears dead), and Sockso (no mobile support). I also briefly considered other streaming services such as Pandora, Slacker, rdio, etc., but based on reviews didn t think that they were worth bothering with. Conclusions I m left mainly considering Spotify and Subsonic. For right now, I m going to try Spotify and see where that leads me, but may wind up getting back to Subsonic if I can t find tracks I want with Spotify.

The Debian Edu / Skolelinux project have users all over the globe, but until recently we have not known about any users in Norway's neighbour country Sweden. This changed when George Bredberg showed up in March this year on the mailing list, asking interesting questions about how to adjust and scale the just released Debian Edu Wheezy setup to his liking. He granted me an interview, and I am happy to share his answers with you here. Who are you, and how do you spend your days? I'm a 44 year old country guy that have been working 12 years at the same school as 50% IT-manager and 50% Teacher. My educational background is fil.kand in history and religious beliefs, an exam as a "folkhighschool" teacher, that is, for teaching grownups. In Norwegian I believe it's called "Vuxenupplaring". I also have a master in "Technology and social change". So I'm not really a tech guy, I just like to study how humans and technology interact and that is my perspective when working with IT. How did you get in contact with the Skolelinux/Debian Edu project? I have followed the Skolelinux project for quite some time by now. Earlier I tested out the K12-LTSP project, which we used for some time, but I really like the idea of having a distribution aimed to be a complete solution for schools with necessary tools integrated. When K12-LTSP abandoned that idea some years ago, I started to look more seriously into Skolelinux instead. What do you see as the advantages of Skolelinux/Debian Edu? The big point of Skolelinux to me is that it is a complete distribution, ready to install. It has LDAP-support, MS Windows integration tools and so forth already configured, saving an administrator a lot of time and headache. We were using another Linux based thin-client system called Thinlinc, that has served us very well. But that Skolelinux is based on VNC and LTSP, to me, is better when it comes to the kind of multimedia used in schools. That is showing videos from Youtube or educational TV. It is also easier to mix thin clients with workstations, since the user settings will be the same. In our VNC-based solution you had to "beat around the bush" by setting up a second, hidden, home-directory for user settings for the workstations, because they will be different from the ones used on the thin clients. Skolelinux support for diskless workstations are very convenient since a school today often need to use a class room projector showing videos in full screen. That is easily done with a small integrated media computer running as a diskless workstation. You have only two installs to update and configure. One for the thin clients and one for the workstations. Also saving a lot of time. Our old system was also based on Redhat and CentOS. They are both very nice distributions, but they are sometimes painfully slow when it comes to updating multimedia support and multimedia programs (even such as Gimp), leaving us with a bit "oldish" applications. Debian is quicker to update. What do you see as the disadvantages of Skolelinux/Debian Edu? Debian is a bit too quick when it comes to updating. As an example we use old HP terminals as thinclients, and two times already this year (2012) the updates you get from the repositories has stopped sound from working with them. It's a kernel/ALSA issue. So you have to be more careful properly testing the updates before you run them in a production environment. This has never happened with CentOS. I also would like to be able to set my own domain-settings at install time. In Skolelinux they are kind of hard coded into the distribution, when it comes to LDAP and at least samba integration. That is more a cosmetic/translation issue, and not a real problem. Running MS Windows applications within the Skolelinux environment needs to be better supported. That is, running them seamlessly via RDP, and support for single-sign on. That will make the transition to free software easier, because you can keep the applications you really need. No support will make it impossible if you work in a school where some applications can't be open source. As for us we really need to run Adobe InDesign in our journalist classes. We run a journalist education, and is one of the very few non university ones that is ok:d by Svenska journalistf rbundet (Swedish journalist association). Our education gives the pupils the right of membership there, once they are done. This is important if you want to get a job. Adobe InDesign is the program most commonly used in newspapers and magazines. We used Quark Express before, but they seem to loose there market to Adobe. The only "equivalent" to InDesign in the opensource world is Scribus, and its not advanced enough. At least not according to the teacher. I think it would be possible to use it, because they are not supposed to learn a program, they are supposed to learn how to edit and compile a newspaper. But politically at our school we are not there yet. And Scribus lacks a lot of things you find i InDesign. We used even a windows program for sound editing when it comes to the radio-journalist part. The year to come we are going to try Audacity. That software has the same kind of limitations compared to Adobe Audition, but that teacher is a bit more open minded. We have tried Ardour also, but that instead is more like a music studio program, not intended for the kind of editing taking place in a radio studio. Its way to complex and the GUI is to scattered when you only want to cut, make pass-overs, add extra channels and normalise. Those things you can do in Audacity, but its not as easy as in Audition. You have to do more things manually with envelopes, and that is a bit old fashion and timewasting. Its also harder to cut and move sound from one channel to another, which is a thing that you do frequently because you often find yourself needing to rearrange parts of the sound file. So, I am not sure we will succeed in replacing even Audition, but we will try. The problem is the students have certain expectations when they start an education towards a profession. So the programs has to look and feel professional. Good thing with radio, there are many programs out there, that radio studios use, so its not as standardised as Newspaper editing. That means, it does not really matter what program they learn, because once they start working they still have to learn the program the studio uses, so instead focus has to be to learn the editing part without to much focus on a specific software. Which free software do you use daily? Myself I'm running Linux Mint, or Ubuntu these days. I use almost only open source software, and preferably Linux based. When it comes to most used applications its OpenOffice, and Firefox (of course ;) ) Which strategy do you believe is the right one to use to get schools to use free software? To get schools to use free software there has to be good open source software that are windows based, to ease the transition. But it's also very important that the multimedia support is working flawlessly. The problems with Youtube, Twitter, Facebook and whatever will create problems when it comes to both teachers and students. Economy are also important for schools, so using thin clients, as long as they have good multimedia support, is a very good idea. It's also important that the open source software works even for the administration. It's hard to convince the teachers to stick with open source, if the principal has to run Windows. It also creates a problem if some classes has to use Windows for there tasks, since that will create a difference in "status" between classes, so a good support for running windows applications via the thin client (Linux) desktop is essential. At least at our school, where we have mixed level of educations, from high-school to journalist-school. Update 2012-07-09 08:30: Paul Wise tipped me on IRC about three useful sources related to Free Software for radio stations: the LWN article Radio station management with Airtime, Airtime which claim to be a Free open source radio automation software and Rivendell which claim to be complete radio broadcast automation solution. All of them seem useful to the aspiring radio producer.

It's been a week with quite some changes to http.debian.net, the Debian mirrors redirector, and it keeps coping very well with the continuously increasing traffic: over 1.5 million requests from APT clients in the last seven days! half-million more from the week before.

The good news is, those rare Hash Sum mismatch errors should be mostly gone. Ditto for some other sort of errors. There is now a second server that takes care of monitoring the mirrors and is ready to handle some of the traffic if there's any need. With this new server, http.debian.net will soon be available over IPv6 too.

Those who are at Debconf12 and followed my advice to use http.debian.net will have noticed that it is redirecting users to the local mirror. So, once again, forget about switching mirrors!

Thanks are also due to J rg Jaspert and Philipp Kern, for the new server and for the work needed to allow http.debian.net access to Debconf12's local mirror, respectively.

Many thanks again to those who keep providing feedback and have helped the project along the way.

What's next? even more improvements and fixing some issues, some of which involve further collaboration and cooperation with the mirror administrators.

Some parts of the Debian mirrors network are fragile and may bite every once in a while, but those are being worked on. If you administer a mirror, please use ftpsync and submit your Debian mirror.
You might be happy to know that I'm working on restricting mirrors to specific Autonomous System, or countries. More on this later.

So, I was trying to write a script for myself to sanitise the EXIF metadata in images produced by my camera. I had a text file for a while with some tags that I should remove (like the camera serial number, etc.), but this became cumbersome and I wanted a fully scripted workflow. I thought initially it's a simple exercise: look at the current tags, decide either which one I wanted to remove (black-list) or alternatively keep (white-list; preferred solution) and then call exiftool with the appropriate options. Boy oh boy EXIF is just part of the equation. There are of course also the MakerNotes, which are manufacturer-specific and most of the time editable only as a block. Furthermore, there's no way to easily just duplicate each individual entry in this group to a standard EXIF or even XMP section, so basically you either keep to much or just give up a lot. Some of them can be copied to XMP, for example lens information, flash, etc. but white-balance (with the exiftool version that I have) is not, for example. Ideally I could just copy all maker notes to XMP, and then drop that entire group. For the record, the way to drop all and just copy some is to run:

exiftool -all= -tagsfromfile @ -Orientation -AllDates  other tags  file.jpg

But do not add "-makernotes", because that will bring the entire block back. The other fun thing is the number of thumbnails/preview images embedded. Looking at a random jpeg file, I found:

• a thumbnail image in the EXIF section, indeed small, around ~8KiB in size (160 120 pixels)
• a preview image in the MakerNotes section, somewhat bigger, around ~24KiB in size (570 375 pixels)
• a large thumbnail (full HD equivalent) in the MPF section, with an incredible size of ~756KiB (1632 1080)

Now, I doubt even the use of one thumbnail, but 3? And one being more than half a megabyte? Someone's going overboard at Nikon On the plus side, removing all thumbnails and sanitising the meta-data makes the jpeg files quite a bit smaller, which also means easier to share, at least; not sure if I want to actually clean the master copies (even though I keep the NEFs).

A few days ago I had blogged about getting NumPy data in R by using a simple converter script. That works fine, but it is a little annoying to have to write an entire file only to read from it again. So I kept looking around for a better solution---and soon found the cnpy library by Carl Rogers which provides simple C++ functions to read and write NumPy files. Bringing such a C++ library to R is done very easily via Rcpp modules. The resulting package contains a single R file with a single line: loadModule("cnpy", TRUE). And it relies on the following module declarations in the a C++ file:

RCPP_MODULE(cnpy) 
    using namespace Rcpp;
    function("npyLoad",                         // name of the identifier at the R level
             &npyLoad,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("type") = "numeric"),
             "read an npy file into a numeric or integer vector or matrix");
    function("npySave",                         // name of the identifier at the R level
             &npySave,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("object"), 
                           Named("mode") = "w"),
             "save an R object (vector or matrix of type integer or numeric) to an npy file");
 

which give us at the R prompt

R> library(RcppCNPy)
Loading required package: Rcpp
R> npyLoad
internal C++ function <0x243af70>
    docstring : read an npy file into a numeric or integer vector or matrix
    signature : Rcpp::RObject npyLoad(std::string, std::string)
R> npySave
internal C++ function <0x23033e0>
    docstring : save an R object (vector or matrix of type integer or numeric) to an npy file
    signature : void npySave(std::string, Rcpp::RObject, std::string)
R> 

these two functions (and their docstrings) defined above. That's all! Well there are about one hundred more lines dealing with whether we have integer or numeric data, and whether we use a vector or a matrix. But all in all pretty simple... So version 0.1.0 of this new package RcppCNPy completes the initial release 0.0.1 from earlier in the week by adding

• the ability to load compressing NumPy files ending in .npy.gz
• a simple regression test suite loading some data sets
• a demo script with a timing example comparing ascii reads to reading npy and compressed npy
• a short pdf vignette describing the package

The NEWS entry for this release (as well as the initial one) follow:

News for Package RcppCNPy Changes in version 0.1.0 (2012-07-07)

• Added automatic use of transpose to automagically account for Fortran-vs-C major storage defaults between Python and R.
• Support for integer types in dependent on the int64_t type which is available only when the -std=c++0x switch is used at build-time (and CRAN still discourages use of it)
• Added support for reading gzip'ed files ending in ".npy.gz"
• Added regression tests in directory tests/
• Added a vignette describing the package
• Added a timing benchmark in demo/timings.R

Changes in version 0.0.1 (2012-07-04)

• Initial version, as a straightforward Rcpp modules wrap around the cpny library by Carl Rogers (on github under a MIT license).
• At present, npy files can be read and written for vectors and matrices of either numeric or integer type. Note however that matrices are currently transposed because of the default Fortran ordering done by numpy.

I will follow up with a little usage example later. CRANberries also provides a diffstat report for 0.1.0 relative to 0.0.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

I got a few emails about the status panel I'd both toyed with and posted. The end result is that the live load graphs now have documentation, look prettier, and contain a link to the source code. Apart from that this week has mostly involved photographing cute cats, hairy dogs, and women in corsets. In Debian-related news njam: Insecure usage of environmental variable was closed after about 7 months, and I reported a failure of omega-rpg to drop group(games) privileges prior to saving game-state. That leads to things like this:

skx@precious:~$ ls -l   grep games
-rw-r--r--   1 skx games   14506 Jul  8 15:20 Omega1000

Not the end of the world, but it does mean you can write to directories owned by root.games, and potentially over-write level/high-score files in other packages leading to compromises. ObQuote: "Your suffering will be legendary, even in hell! " - Hellraiser II (Did you know there were eight HellRaiser sequels?)

I keep using a Neo Freerunner with Qtmoko in a Debian Squeeze (+ some Wheezy) chroot as my daily phone. It also works well for SMS, GPS, MUA, IM, IRC and some Web browsing with Aurora (using both Wifi and GPRS). Last weekend I managed to play a little with VoIP possibilities in Qtmoko and ended with a functional SIP client in my hands. I want to share some points which may be useful for other brave freerunners:

• Qtmoko has a native VoIP client but it seems to be broken. Its sipagent uses port 5060. I couldn't find a way to disable it at qpe startup, so I had to setup my client to use another port.
• Choosing the application: Ekiga takes 93.4MB of additional disk space. Qutecom takes 212MB. Too much for a 256MB rootfs. Twinkle is ok on this point, it uses about 40MB of additional space, but it fails in the capture (mic) quality. I've tried many setups in my voip-handset state file with no success. Perhaps it's something related to the codecs and/or default parameters. I didn't go deeper on this and gave Linphone a try. This package also uses about 40MB of additional space and finally made my day.
• Linphone gui is called by linphone-3, which has a character not supported by /opt/qtmoko/etc/qx/profiles.conf and /opt/qtmoko/etc/qx/favourites.conf. So if you want to have some setup here you need to create a link/wrapper/whatever without this dash.
• You can edit your preferences directly in your ~/.linphonerc . Remember to change the sip_port entry to any other than 5060.
• I use a basic VoIP service from diamondcard.us. It's cheap and works properly. This company gives financial support to Ekiga and Twinkle projects.
• I was not able to have default Freerunner headset working on an acceptable quality. Help will be very welcome.
• Here is my current config:
• /opt/qtmoko/etc/qx/profiles.conf
• /opt/qtmoko/etc/qx/favourites.conf
• .linphonerc
• /usr/share/openmoko/scenarios/voip-handset.state (please share your improvements!)

UPDATE: I didn't make clear that I get a very nice audio quality using internal mic/speaker with the above voip-handset state file. The issue is only when using the headset, which demands understanding and setting up a proper state file. Enjoy some shots:

Forward Error Correction with the TI CC1120 Sub-GHz transceiver We re building a new, fancier flight computer and decided to give the new, fancier TI CC1120 chip a try. It has significantly more transmit power (+16dBm) and receive sensitivity (-109dBm) than the transciever built in to the TI CC1111 RF SoC that we ve built our other boards with. Given that we d already decided to use a fancier STM32L processor, using the nicest stand-alone radio chip we could find seemed to make good sense. One of the nice features found in the CC1111 is support for FEC using a 1/2 rate constraint length 4 convolutional code. Both the encoder and the soft-decision Viterbi decoder are built right into the chip; all we had to do was flip a couple of bits and we got error correction, interleaving, data whitening and CRC checking all for free. We assumed that, as the CC1120 was advertised as a better replacement for our CC1111 radio, that it would also include all of these fine features. After we got the chip all soldered down to our first prototype boards, we read through the reference manual looking for the right register values to flip for forward error correction. We found support for CRC computation and data whitening, but no mention of interleaving or forward error correction at all. Thanks, TI. Of course, without the FEC pieces, the other packet handling hardware is worthless you can t exactly do a CRC on the encoded data and expect it to be useful on the other end of the link, and data whitening happens after the CRC. Basic CC1120 driver I assumed I d eventually figure out something to do about the FEC pieces and started writing a basic CC1120 driver AltOS, the operating system used in all of our projects. I started with the transmit side, as that seemed easier as I could use the existing packet hardware by uploading the desired bit sequence using whatever fancy encoding desired and the hardware would happily transmit it. I first tested that using low data-rate bits (2kbps), transmitting an alternating sequence of zeros and ones to generate a 1kHz tone on a regular 70cm receiver. That seemed to work just fine, demonstrating that I had some grasp of the basic operation of the chip. Convolutional Encoding in Software Not to be daunted by a small challenge in software, I set out to replicate the encoding scheme used in the CC1111 chips for use in our CC1120-based design. We obviously want our CC1120 boards to interoperate with the CC1111 boards. Fortunately, TI fully documents the CRC, data whitening, convolutional encoding and interleaving done in the hardware. I created my own implementation of the encoder (licensed, as usual, under the GPLv2) which you can see in the file ao_fec_tx.c It s not quite as flexible as the hardware, it always whitens the data and always appends the CRC bytes to the end of the packet, along with the necessary trellis termination bytes. The core of that code is surprisingly simple:

static const uint8_t ao_fec_encode_table[16] =  
/* next 0  1      state */
    0, 3,   /* 000 */
    1, 2,   /* 001 */
    3, 0,   /* 010 */
    2, 1,   /* 011 */
    3, 0,   /* 100 */
    2, 1,   /* 101 */
    0, 3,   /* 110 */
    1, 2    /* 111 */
 ;
uint8_t
ao_fec_encode(const uint8_t *in, uint8_t len, uint8_t *out)
 
    uint8_t     extra[AO_FEC_PREPARE_EXTRA];
    uint8_t     extra_len;
    uint32_t    encode, interleave;
    uint8_t     pair, byte, bit;
    uint16_t    fec = 0;
    const uint8_t   *whiten = ao_fec_whiten_table;
    extra_len = ao_fec_prepare(in, len, extra);
    for (pair = 0; pair < len + extra_len; pair += 2)  
        encode = 0;
        for (byte = 0; byte < 2; byte++)  
            if (pair + byte == len)
                in = extra;
            fec  = *in++ ^ *whiten++;
            for (bit = 0; bit < 8; bit++)  
                encode = encode << 2   ao_fec_encode_table[fec >> 7];
                fec = (fec << 1) & 0x7ff;
             
         
        interleave = 0;
        for (bit = 0; bit < 4 * 4; bit++)  
            uint8_t byte_shift = (bit & 0x3) << 3;
            uint8_t bit_shift = (bit & 0xc) >> 1;
            interleave = (interleave << 2)   ((encode >> (byte_shift + bit_shift)) & 0x3);
         
        *out++ = interleave >> 24;
        *out++ = interleave >> 16;
        *out++ = interleave >> 8;
        *out++ = interleave >> 0;
     
    return (len + extra_len) * 2;
 

ao_fec_encode_table takes care of the convolutional piece, ao_fec_whiten_table (not shown) contains the data whitening codes while the little loop at the bottom does the interleaving. I didn t spend a lot of time optimizing this as it seemed to run plenty fast on the 32MHz processor. With the bits all nicely encoded, I passed them to my simple CC1120 driver and let it send them along to our CC1111 receiver. Much to my delight, with only a few bugs fixed, it worked just fine! I d managed to get one direction working, validating the hardware and making it clear that we d at least be able to use the board in flight for telemetry. Getting Soft-decision data out of the CC1120 In packet mode, the CC1120 receiver converts the incoming GFSK stream directly into bits, making its best guess as to whether the detected value was a one or zero. Of course, like any FM modulation, the receiver will be producing intermediate values, especially when the signal is weak or masked by interference. A Viterbi decoder can make good use of this information;these noisy intermediate values are less influential in the final resulting path cost which allows the strong 0/1 values to pull the solution to the right one. Soft decision decoding yields a theoretical 2dB gain over hard decision decoders, and given our desire for maximum packet reception over long distances, it seemed important to make this work. The CC1120 has a way to recover the soft decision values, but only one 8-bit value at a time. It does, however, helpfully provide a GPIO wire which can be programmed to interrupt the CPU when the soft decision data has arrived. So, I hooked up a GPIO line on the processor to the GPIO line on the CC1120 and let it interrupt once per bit time, or 38400 times per second, to read this valuable soft decision data out of the radio part. According to TI support, this is the only way to make this work, and that they generally recommend that developers use the packet mode stuff and throw away this valueable source of data. I was happily surprised to learn that the STM32L can in fact survive this interrupt rate, and by setting interrupt priorities appropriately, I manage to reliably capture an entire packets worth of data without dropping a single bit. I set up a simple packet dumping function to run on the prototype board and captured raw encoded bytes from our CC1111-based flight computer so that I d have some real data to work from. Soft-decision Viterbi Decoding The sample encoding implementation from TI made that piece fairly straightforward. Lacking a similar sample for the decoding piece meant that I d be starting from scratch. And, implementing something that seemed like magic to me when I started. I started by reading a few articles on Viterbi decoding: I found Chip Fleming s tutorial very useful as it included a bunch of worked examples using a convolutional encoders with constraint length (k) of 3 instead of 4; this makes all of the trellis diagrams half the size. I also sent my friend, Phil Karn, a query about this process, and he responded with a bunch of helpful suggestions, while also recommending that I just steal his DSP and FEC library implementation. Of course, I don t like using code that I can t understand, and his code only implements much longer codes than I needed, so I d have to understand it anyways to make it work for me. Phil s code is a marvel, but it s not exactly comprehensible to someone who doesn t even know what a Viterbi decoder is supposed to do. So, with Chip s tutorial on my screen, I wrote a primitive Viterbi decoder. This did only the Viterbi step, and split that into two phases. The first computed the cost for each state in the system for every received pair of symbols received. It also tracked the chain of states through the entire decode process:

    for (state = 0; state < 8; state++)  
        struct ao_soft_sym zero = ao_soft_sym(ao_fec_encode_table[state * 2 + 0]);
        struct ao_soft_sym one = ao_soft_sym(ao_fec_encode_table[state * 2 + 1]);
        uint8_t zero_state = ao_next_state(state, 0);
        uint8_t one_state = ao_next_state(state, 1);
        int zero_cost = ao_cost(s, zero);
        int one_cost = ao_cost(s, one);
        zero_cost += cost[b][state];
        one_cost += cost[b][state];
        if (zero_cost < cost[b+1][zero_state])  
            prev[b+1][zero_state] = state;
            cost[b+1][zero_state] = zero_cost;
         
        if (one_cost < cost[b+1][one_state])  
            prev[b+1][one_state] = state;
            cost[b+1][one_state] = one_cost;
         
     

At the end of the packet, it found the least costly path and walked that from back to front, generating output bits (yes, one bit per element of the bits array):

for (state = 1; state < 8; state++)  
    if (cost[b][state] < c)  
        c = cost[b][state];
        min_state = state;
     
 
for (b = len/2; b > 0; b--)  
    bits[b-1] = min_state & 1;
    min_state = prev[b][min_state];
 

Completely separate from this code were the interleaving and data whitening stages. Keeping those separate allowed them to be tested independently, which made debugging them far easier. None of this code ever ran on the STM32L processor, it was instead run in a test framework on my laptop. I first got this working with data generated by the software convolutional encoder, then I managed to get it to decode the raw packets that I had recorded directly from the receiver it self. This marked a significant milestone in my mind I d managed to replace, in software, the CC1111 FEC encoder and decoder. However, the decoder performance was not fast enough to manage back-to-back packets yet. Having something in hand that worked meant that further development would be directly comparable to something that did work, making optimization much easier. Yet another lesson in making it work, and then making it work fast. Optimizing the Viterbi Decoder The first step was to reduce memory usage within the decoder. While it s best to defer computing the path until the very last bit has been received, it s also true that the chances of a bit affecting old data reduces as the distance between the new and old data increases. With the rule of thumb being a distance of k 6 or k 7, I saved 32 output bits for each state and wrote the oldest of 8 each time the buffer filled up, making a distance of 24 bits between the newest saved data and the current decoding position. That meant a fixed amount of storage was needed for an arbitrarily long packet. I tested a variety of different history sizes from 8 to 56 bits (cooresponding to 16, 32 and 64 bits of saved data for each state). With 100000 random packets generated that had gaussian noise added to each bit, here are the error rates for the different history lengths:

History Length	Total Packets	Correct	Incorrect
8 bits	100000	90520 (90.52%)	9480 (9.48%)
24 bits	100000	93614 (93.61%)	6386 (6.39%)
56 bits	100000	93620 (93.62%)	6380 (6.38%)

The performance of the 8-bit history and 16-bit history versions turned out to be essentially identical (given the 32-bit nature of the STM32L, that shouldn t be too surprising). The performance of the 56-bit version, which manipulated uint64_t data types was sigificantly slower, and given the very modest benefit, deemed not worth it. If we move to a larger constraint length (k > 4), or started using a punctured convolutional encoding, we would want to re-measure this. The next step was to combine the de-interleaving, decoding, de-whitening and CRC computation into a single loop. This would eliminate a pile of data copying and extra memory usage. Nothing fancy here, but it made enough of a difference that I could decode most of the incoming packets now, dropping only about 25% of them. To get to being able to decode every packet, I needed to start decoding the packet while it was being received. Because of interleaving, I had to have 32 bits of data to be able to decode anything, so it was a simple matter of having my per-soft-value interrupt handler start the decoder going after each interleave block of data was received. With this in place, I was able to decode a 76-byte payload (160 transmitted bits) in 14.7 ms, or just about 50% of the time it took to transmit the packet. Unrolling one of the inner decoding loops eliminated a bunch of computation and sped this up even more, decoding packets in 9.5ms. Success at last, a solid 100% of packets received were being decoded, as long as the CPU was otherwise idle. Finally, I went and re-read Phil Karn s code and pondered over how it works. It was opaque until I wrote a message back to Phil asking how it worked and pointing out sections that were confusing. Fortunately, simply phrasing my confusion in words made me understand how the code works. Here s my original code for computing the per-state cost metric:

/* Metric for a zero bit */
uint32_t    bitcost = ((uint32_t) (s0 ^ ao_fec_decode_table[(state<<1)]) +
               (uint32_t) (s1 ^ ao_fec_decode_table[(state<<1)+1]));
 
    /* Total path metric and state for a zero bit */
    uint32_t    cost0 = cost[p][state] + bitcost;
    uint8_t     state0 = ao_next_state(state, 0);
    /* Compare the total path metric against all existing
     * metrics heading into the same new state, choosing
     * the least expensive and killing the others.  Record
     * the new lowest cost and output bit.
     */
    if (cost0 < cost[n][state0])  
        cost[n][state0] = cost0;
        bits[n][state0] = (bits[p][state] << 1)   (state & 1);
     
 
 
    /* Total path metric and state for a one bit.  Because
     * encoding a one is always exactly the opposite of
     * encoding a zero from any state, this cost is
     * 255*2 - bitcost
     */
    uint32_t    cost1 = cost[p][state] + 510 - bitcost;
    uint8_t     state1 = ao_next_state(state, 1);
    if (cost1 < cost[n][state1])  
        cost[n][state1] = cost1;
        bits[n][state1] = (bits[p][state] << 1)   (state & 1);
     
 

Before this loop is run, I have to initialize cost[n] to a large value so that the comparisons will be true for the first case to reach each state. Phil s code, in contrast, doesn t have to initialize cost[n] as we know which two states can reach any possible new state. His code looks like this:

/* Metric for a zero bit */
bitcost = ((uint32_t) (s0 ^ ao_fec_decode_table[(state<<1)]) +
       (uint32_t) (s1 ^ ao_fec_decode_table[(state<<1) 1]));
/* Only state and state+4 reach state<<1 with a zero bit */
/* Cost from state to state<<1 for a zero bit*/
m0 = cost[p][state] + bitcost;
/* Cost from state+4 to state<<1 for a zero bit */
m1 = cost[p][state+4] + (510 - bitcost);
/* Which source state is cheaper? */
bit = m0 > m1;
/* Record the cheaper cost and the add to the path of bits */
cost[n][state<<1] = bit ? m1 : m0;
bits[n][state<<1] = (bits[p][state + (bit<<2)] << 1)   (state&1);
/* State and state+4 reach (state<<1)+1 with a one bit */
/* Cost from state to (state<<1)+1 for a one bit */
m0 -= (bitcost+bitcost-510);
/* Cost from state+4 to (state<<1)+1 for a one bit */
m1 += (bitcost+bitcost-510);
/* Which source state is cheaper? */
bit = m0 > m1;
/* Record cheaper cost and add to the path of bits */
cost[n][(state<<1)+1] = bit ? m1 : m0;
bits[n][(state<<1)+1] = (bits[p][state + (bit<<2)] << 1)   (state&1);

This code does two states at a time, and so while it s slightly longer than the above, it only needs to run half as many times. The resulting code now decodes packets in 5.7ms. The final version of the code can be see in ao_fec_rx.c The whole AltOS operating system, with many drivers and support for CC1111, AVR and STM32L processirs is available, under the GPLv2 from git://git.gag.com/scm/git/fw/altos

[...] read also says how many bytes of the file were returned, so end of file is assumed when a read says "zero bytes are being returned." [...] When a program reads from your terminal, each input line is given to the program by the kernel only when you type its newline (i.e, press RETURN). [...] [...] Now try something different: type some characters and then a *ctl*-d rather than a RETURN: $ cat -u
123ctl-d123 cat prints the characters out immediately. *ctl*-d says, "immediately send the characters I have typed to the program that is reading from my terminal." The *ctl*-d itself is not sent to the program, unlike a newline. Now type a second ctl-d, with no other chracters: $ cat -u
123ctl-d123ctl-d$ The shell responds with a prompt, because cat read no characters, decided that meant end of file, and stopped. ctl-d sends whatever you have typed to the program that is reading from the terminal. If you haven't typed anything, the program will therefore read no characters, and that looks like the end of the file. That is why *ctl*-d logs you out --- the shell sees no more input. Of course, *ctl*-d is usually used to signal an end-of-file but it is interesting that is has a more general function.

Brian W. Kernighan, Rob Pike. The UNIX Programming Environment. Prentice-Haskell Software Series, 1984, section 2.1, pages 44-45.

This seems like a simple question: how do you listen to music? But in a way, I think I ve fallen behind the times. I ve enjoyed listening to digital music for years, since well before MP3 even existed (MIDI files could be comfortably downloaded over dialup; some of you may even remember MOD files). Anyhow, a few years ago, I digitized my entire CD collection, ripping it to FLAC and encoding to high-bitrate MP3 for storage on my 160GB iPod. Since, I ve purchased some music from DRM-free MP3 retailers, and even FLAC files from Magnatune, but at this point I basically say that I have enough music. I have only rarely bought anything new in the last couple of years. I have used Musicbrainz Picard to enforce consistent tagging across almost my entire collection. I am usually in front of a Linux machine. I run mt-daapd on my fileserver, so I can stream music from it to any other machine in the house. However, this has a lot of drawbacks. It is generally impossible to create or modify persistent playlists with this protocol. I can t modify metadata. I can t assign ratings. Though perhaps I don t need to. There are a lot of online services I could think of: Pandora, Spotify, Rdio, Slacker. Plus, of course, the upload your music to the cloud services such as Google and Amazon. Have people tried these? I m hesitant to use a service where I pay a monthly fee, unless I wind up owning the music in a DRM-free fashion. And I m a little nervous about the privacy implications of uploading a vast amount of music to the cloud. There are some items in my collection that I am certain are not available on any of those others, but for the most part I m willing to take what is out there if it meets my tastes. My tastes, by the way, typically run to classical and opera but also can go towards middle eastern or certain new age music (though I m very picky about that, since I don t care for most new age stuff I hear.) But these are huge categories; sometimes I d like to hear some load and noisy classical (say, Beethoven s Ninth conclusion, some Wagner, Verdi arias, etc.) And other times, something more contemplative (a Mozert concerto) or even quiet (a nice sonata) meet my mood. I have playlists for these on my iPod Classic a bit of a problem, since I don t use that device much anymore except in the car, and it takes a LOT of time to categorize things into these playlists. (An opera or a symphony could have parts in various styles and various levels of energy, for instance.) I keep a small manually-managed subset on my Android phone, which works fine. A recommendation engine for things like this could be useful, if it works well. My requirements, basically: I want something that will play the same music on my Linux machines. I want to be able to have playlists kept in sync across my machines. And I don t want a monthly fee. Ideally, the software will work on Android as well, and provide similar features there. Anything that requires Windows is a non-starter. I ve considered NFS mounts, but that s a bit annoying when the laptop disconnects from the network or when it s not at home. What are your thoughts? Ideas?

As Weblate 1.1 is out, it's time to schedule next release. This release will bring some small improvements to the workflow, especially it will allow to tune it to different use cases than I originally had in mind. I've already started tagging bugs and feature requests to be fixed in 1.2 in [issue tracker][4]. Basically no big features on the list, just smaller things like notifications and locking. And of course dozen of bugfixes as the bugs will appear :-). As you can see there as well, 1.2 should be ready about month from now. This will be possible only thanks to Hackweek VIII (the wiki is still showing last one), which will happen from July 23rd to July 27th. The rest is time for testing the features and polishing the features. Anyway I might have time for some other features as well, so 1.2 might bring more things than I currently plan. But that depends on how fast I will be in implementing planned stuff.

Filed under: English Suse Weblate 0 comments Flattr this!

For my GSoC project this year I need to be able to enumerate all elementary circuits of a directed graph. My code is written in Ocaml but neither the ocamlgraph library nor graph libraries for other languages seem to implement a well tested algorithm for this task. In lack of such a well tested solution to the problem, I decided to implement a couple of different algorithms. Since it is unlikely that different algorithms yield the same wrong result, I can be certain enough that each individual algorithm is working correctly in case they all agree on a single solution. As a result I wrote a testsuite, containing an unholy mixture of Python, Ocaml, D and Java code which implements algorithms by D. B. Johnson, R. Tarjan, K. A. Hawick and H. A. James.

Algorithm by R. Tarjan The earliest algorithm that I included was published by R. Tarjan in 1973.

Enumeration of the elementary circuits of a directed graph
R. Tarjan, SIAM Journal on Computing, 2 (1973), pp. 211-216
http://dx.doi.org/10.1137/0202017

I implemented the pseudocode given in the paper using Python. The git repository can be found here: https://github.com/josch/cycles_tarjan

Algorithm by D. B. Johnson The algorithm by D. B. Johnson from 1975 improves on Tarjan's algorithm by its complexity.

Finding all the elementary circuits of a directed graph.
D. B. Johnson, SIAM Journal on Computing 4, no. 1, 77-84, 1975.
http://dx.doi.org/10.1137/0204007

In the worst case, Tarjan's algorithm has a time complexity of O(n e(c+1)) whereas Johnson's algorithm supposedly manages to stay in O((n+e)(c+1)) where n is the number of vertices, e is the number of edges and c is the number of cycles in the graph. I found two implementations of Johnson's algorithm. One was done by Frank Meyer and can be downloaded as a zip archive. The other was done by Pietro Abate and the code can be found in a blog entry which also points to a git repository. The implementation by Frank Meyer seemed to work flawlessly. I only had to add code so that a graph could be given via commandline. The git repository of my additions can be found here: https://github.com/josch/cycles_johnson_meyer Pietro Abate implemented an iterative and a functional version of Johnson's algorithm. It turned out that both yielded incorrect results as some cycles were missing from the output. A fixed version can be found in this git repository: https://github.com/josch/cycles_johnson_abate

Algorithm by K. A. Hawick and H. A. James The algorithm by K. A. Hawick and H. A. James from 2008 improves further on Johnson's algorithm and does away with its limitations.

Enumerating Circuits and Loops in Graphs with Self-Arcs and Multiple-Arcs.
Hawick and H.A. James, In Proceedings of FCS. 2008, 14-20
www.massey.ac.nz/~kahawick/cstn/013/cstn-013.pdf

In contrast to Johnson's algorithm, the algorithm by K. A. Hawick and H. A. James is able to handle graphs containing edges that start and end at the same vertex as well as multiple edges connecting the same two vertices. I do not need this functionality but add the code as additional verification. The paper posts extensive code snippets written in the D programming language. A full, working version with all pieces connected together can be found here: https://github.com/josch/cycles_hawick_james The algorithm was verified using example output given in the paper. The project README states how to reproduce it.

Input format All four codebases have been modified to produce executables that take the same commandline arguments which describes the graphs to investigate. The first argument is the number of vertices of the graph. Subsequent arguments are ordered pairs of comma separated vertices that make up the directed edges of the graph. Lets look at the following graph as an example: The DOT source for this graph is:

digraph G  
  0;
  1;
  2;
  0 -> 1;
  0 -> 2;
  1 -> 0;
  2 -> 0;
  2 -> 1;
   

To generate the list of elementary circuits using Tarjan's algorithm for the graph above, use:

$ python cycles.py 3 0,1 0,2 1,0 2,0 2,1
0 1
0 2
0 2 1

The commandline arguments are the exact same for the other three methods and yield the same result in the same order. If the DOT graph is in a format as simple as above, the following sed construct can be used to generate the commandline argument that represents the graph:

$ echo  sed -n -e '/^\s*[0-9]\+;$/p' graph.dot   wc -l   sed -n -e 's/^\s*\([0-9]\) -> \([0-9]\);$/\1,\2/p' graph.dot 

Testsuite As all four codebases take the same input format and have the same output format, it is now trivial to write a testsuite that compares the individual output of each algorithm for the same input and checks for differences. The code of the testsuite is available via this git repository: https://github.com/josch/cycle_test The other four repositories exist as submodules of the testsuite repository.

$ git clone git://github.com/josch/cycle_test.git
$ cd cycle_test
$ git submodule update --init

A testrun is done via calling:

$ ./test.sh 11

The argument to the shell script is an integer denoting the maximum number N of vertices for which graphs will be generated. The script will compile the Ocaml, Java and D sourcecode of the submodules as well as an ocaml script called rand_graph.ml which generates random graphs from v = 1..N vertices where N is given as a commandline argument. For each number of vertices n, e = 1..M number of edges are chosen where M is maximum number of edges given the number of vertices. For every combination of number of vertices v and number of edges e, a graph is randomly generated using Pack.Digraph.Rand.graph from the ocamlgraph library. Each of those generated graphs is checked for cycles and written to a file graph-v-e.dot if the graph contains a cycle. test.sh then loops over all generated dot files. It uses the above sed expression to convert each dot file to a commandline argument for each of the algorithms. The outputs of each algorithm are then compared with each other and only if they do not differ, does the loop continue. Otherwise the script returns with an exit code. The tested algorithms are the Python implementation of Tarjan's algorithm, the iterative and functional Ocaml implementation as well as the Java implementation of Johnson's algorithm and the D implementation of the algorithm by Hawick and James.

Future developments Running the testsuite with a maximum of 12 vertices takes about 33 minutes on a 2.53GHz Core2Duo and produces graphs with as much as 1.1 million cycles. It seems that all five implementations agree on the output for all 504 generated graphs that were used as input. If there should be another implementation of an algorithm that enumerates all elementary circuits of a directed graph, I would like to add it. There are some more papers that I would like to read but I lack access to epubs.siam.org and ieeexplore.ieee.org and would have to buy them. Benchmarks seem a bit pointless as not only the algorithms are very different from each other (and there are many ways to tweak each of them) but also the programming languages differ. Though for the curious kind, it follows the time each algorithm takes to enumerate all cycles for all generated graphs up to 11 vertices.

algorithm	time (s)
Johnson, Abate, Ocaml, iterative	137
Johnson, Abate, Ocaml, functional	139
Tarjan, Python	153
Hawick, D	175
Johnson, Meyer, Java	357

The iterative Ocaml code performs as well as the functional one. In practice, the iterative code should probably be preferred as the functional code is not tail recursive. On the other hand it is also unlikely that cycles ever grow big enough to make a difference in the used stack space. The Python implementation executes surprisingly fast, given that Tarjan's algorithm is supposedly inferior to Johnson's and given that Python is interpreted but the Python implementation is also the most simple one with the least amount of required datastructures. The D code potentially suffers from the bigger datastructures and other bookkeeping that is required to support multi and self arcs. The java code implements a whole graph library which might explain some of its slowness.

Update on computational chemistry packages in Debian

For many years, me and the Debichem team maintained two computational chemistry packages, first MPQC and later PSI3. In recent years however, two respected quantum chemistry codes have been open sourced and are now available in Debian: ACESIII and NWChem. Make no mistake, those are not recently started codes or some Ph.D.'s pet-project but state-of-the-art projects targeted at massively parallel supercomputers and developed at the University of Florida's Quantum Chemistry Project (QTP) in the case of ACESIII and at Pacific Northwest National Laboratory (PNNL) for NWChem.

While ACESIII has a somewhat narrow focus on Coupled-Cluster (unfortunately, the current version does not yet implement analytic gradients for the "golden standard" CCSD(T) method) and Multi-Body Perturbation Theory methods, NWChem is a very versatile package supporting a variety of molecular quantum chemistry methods (including Density Functional Theory (DFT)), as well as periodic plane-wave electronic structure and (ab-initio and classical) molecular dynamics. At this point, NWChem probably covers 90% of all use cases for routine computational chemistry compared to the popular non-free codes Gaussian and GAMESS, while likely exhibiting superior parallel scaling at least compared to Gaussian.

Over the last year, I have packaged and integrated NWChem and ACESIII for Debian, and they are now both in testing. Furthermore, I have recently overhauled the packaging of MPQC and enabled parallel execution via MPI. Further, both MPQC and PSI will see updated versions (MPQC3 and PSI4) in the near future with significant increased functionality, including some unique (at least among open source codes) features like Symmetry-Adapted Perturbation Theory (SAPT) for PSI4 and general explicitly correlated corrections ([2]_f12) for MPQC3. Unfortunately, they both did not get released in time for the wheezy freeze.

On the periodic/plane-wave front, I have also packaged and uploaded Quantum ESPRESSO (formerly PwSCF), a set of plane-wave programs, and I have been working with lafur Jens Sigur sson to get the latest Abinit release packaged. The nanoscale-physics team is also targeting further periodic ab-initio packages like BigDFT and Octopus, but those as well did not land in time for the freeze.

Finally, I also packaged and uploaded CP2K, a well-engineered and versatile periodic plane-wave/pseudo-potential code which focuses on state-of-the-art ab-initio Molecular Dynamics (AIMD) using Density Functional Theory. Some of the authors have recently published a paper reporting linear scaling benchmark calculations on a system with a million atoms using almost 50,000 CPU cores.

All of the above mentioned packages are included either in the molecular ab-initio or the periodic ab-initio Debichem tasks.

Pretty much on schedule, Weblate 1.1 has been released today. It comes with translation updates, bug fixes, improvements in working with Git repositories and brings support for offloading indexing. Full list of changes for 1.1:

• Improved several translations.
• Better validation while creating subproject.
• Added support for shared git repositories across subprojects.
• Do not necessary commit on every attempt to pull remote repo.
• Added support for offloading indexing.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances can be found in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you (this will be decided case by case as my hosting space is limited). Update: Weblate appliace has now also been updated to 1.1.

Filed under: English Phpmyadmin Suse Weblate 0 comments Flattr this!

Monthly DPL bits, fresh from the oven^W^W^W hot from DebConf12, and just posted to d-d-a.

---

Howdy from DebConf12. It's hot, but it's also time to bother you again with a (not so) brief DPL activity report, this time for June 2012. Time-based freeze: DONE, short freeze: TODO Two highlights for this month. First, you've probably noticed Wheezy is now frozen, YAY. This is huge achievement for the release, but also for the project. It's the first time we do a time-based freeze, and it took some quite heated discussion at the beginning of the release cycle to decide to do this. And we did it properly: respecting the planned month and narrowing down the period later. This exercise has hopefully helped both DDs in their package planning and our upstreams in targeting Wheezy with stable releases of their software. Kudos to the release team for their coordination work! Now we've the second part still TODO: releasing Wheezy, without RC bugs, with a freeze period as short as possible. See the beginning of my last "bits from the DPL" mail for my usual song and dance on how to deliver that, together. DebConf12 A lot of us will attend DebConf12. Enjoy it! ... and take the chance to both have fun and make great plans for Debian's future. But remember that "if it didn't happen on a mailing list, it didn't happen". Not all of us will be lucky enough to attend DebConf (in person or remotely). Make sure that those who don't can take part in your team decisions and get informed of what is going to happen here. Politics

• After discussion (and consensus) on debian-boot, I've asked to join FSF campaign on secure boot.
• Since OIN was seeking comments on their Linux System definition (not really openly though: I learned about it last minute from people directly in touch with OIN representatives), I've contacted them proposing to include all "Debian main" packages in their definition. I haven't yet hear back from them, but I doubt they'll accept the idea. It'd be nice to have an official answer about why, though.

Zack's spring tour I spent a significant part of June doing Debian talks ins some sort of "spring tour" between Italy and France. In particular:

• I've taught a lenghty class about the Debian Project at the Insubria International Open Source Summer School in Como, Italy (see slides)
• I've delivered the opening keynote at the yearly Free Software conference in Ancona, Italy, ConfSL'12 (see slides)
• I've delivered the opening talk at the Debian for Scientific Facilities Days event organized by the European Synchrotron Radiation Facility (ESRF) in Grenoble, France (see slides)

Many thanks to the organizers of these events for inviting and sponsoring me (as well as other Debian people, in the ESRF case) and for their interest in Debian. Sprints

• I approved (for a budget of ~2'000 EUR) and helped hosting the i18n sprint in Paris. As a (great!) result of it, the i18n infrastructure has now been moved on DSA administered machines. A more detailed report is available. Thanks to all attendees and in particular to Christian Perrier for the organization work.
• as part of the "Debian for Scientific Facilities" event, a Debian Science sprint also took place at ESRF. Report is pending
• a BSP took place in Salzburg in June, see the report posted by Bernd Zeimetz. I've approved using some Debian money (~350 EUR) to pay for lodgement of some of the attendees

Assets

• DSA have been working on seting up the first machines we bought as part of the yearly revamping plan of Debian hardware. As one of the first user visible changes, UDD has been moved to a new, faster, machine. I single this out because, if you're doing RC bug squashing (as you should! :-)), you'll notice that the bugs search interface is now much faster, hopefully making your release work less frustrating. Similar improvements are forthcoming for other machines, while DSA (thanks!) migrate other services to newer hardware.
• Last April I mentioned that we're aiming at publishing comprehensive Debian budget reports for the past couple of years and that one of the blocker is access to our transaction information at SPI. This is still a blocker, unfortunately. And we still don't have an ETA on when it'll be fixed.
• The current owner of debian.eu has let us know that he is willing to hand over the domain to us. Finally.
• The European part of our Madrid protocol application for the Debian trademark has been vetted by the European trademark office. Meaning: Debian trademark is now officially protected in Europe.
• DuckDuckGo sent me a first report of the "donation" they're ready to make, based on our agreement with them. The aument is still unclaimed, as we're aiming at invoicing twice per year (to keep the papwerwork low). It isn't a stellar amount: 32.55 USD for the month of May.

Discussions Some relevant discussions for project evolution has been going on in June and I took part into them. You might want to have a look at them:

• proposal to change the way DM permissions are handled
• proposal to change the policy ruling the debian.net domain
• we've been discussing debconf12 budget at length, including how to deal with travel sponsorship requests in the future:
• in the "debian account on github" thread, I've proposed a general policy for presence of accounts named "debian" on non-free services
• the long quest on how to choose Wheezy artwork has come to an end, and we have a very nice theme now, thanks to contributions by many artist and integration work by Paul Tagliamonte. Not all went well in the process, but there are useful lessons for the future.

Misc

• I got quite some feedback about the debate with debian-multimedia.org I mentioned last month. My take away message from that feedback is that many users have no idea about the multimedia capabilities (and in particular of codecs availability) of recent Debian releases. We should probably invest some communication energies into that.
• as planned, another tech-ctte IRC meeting has happened. I took part into it, mostly asking to prioritize long standing issues, such as the Python maintenance one. Minutes of the meeting are available.

Cheers.

---

PS the boring day-to-day activity log for June is available at master:/srv/leader/news/bits-from-the-DPL.txt.201206

In the NUUG FiksGataMi project (Norwegian version of FixMyStreet from mySociety), we have discovered a problem with the municipalities using Zimbra. When FiksGataMi send a problem report to the government, the email From: address is set to the address of the person reporting the problem, while envelope sender is set to the FiksGataMi contact address. The intention is to make sure the municipality send any replies to the person reporting the problem, while any email delivery problems are sent to us in NUUG. This work well in most cases, but not for Karm y municipality using Zimbra. Karm y is using the vacation message function in Zimbra to send an automatic reply to report that the message has been received, and this message is sent to the envelope sender and not the address in the From: header. This causes the automatic message from Karm y to go to NUUGs request-tracker instance instead of to the person reporting the problem. We can not really change the envelope sender address, as this would make it impossible for us to discover when there are problems with the MTAs receiving problem reports. We have been in contact with the people at Karm y municipality, and they are willing to adjust Zimbra if something can be changed there to get a better behaviour. The default behaviour of Zimbra is as far as I can tell according to the specification in RFC 3834, which recommend that vacation messages are sent to the envelope sender and not to the From: address. But I wonder if it is possible to adjust or configure Zimbra to behave differently. Anyone know? Please let us know at fiksgatami (at) nuug.no.

Every now and then I have to kill mutt, which if done at the wrong time, can leave the header cache in an open state. Then attempting to restart mutt will cause it to hang at the "Selecting INBOX" stage. If mutt is compiled with support for the Tokyo Cabinet header cache backend (as it appears to be in Debian), a quick fix for this issue is to repair the header cache database using the Tokyo Cabinet utility tcbmgr from the tokyocabinet-bin package. Simply running tcbmgr optimize -nl ~/.mutt/cache/headers does the trick, and is much nicer of a fix than deleting the header cache.

Jan Dejemyr reported Debian bug #680000 on Monday July 2nd, against update-manager-core. Bug #670000 was reported as of April 22nd: 2 months and 10 days for 10,000 bugs. About a constant bug report rate (2 months and 7 days last time). How will the wheezy freeze affect this? We'll see in two months!

Hello A while ago, Peter Knowles wrote a configuration model for Kerberos. Unfortunately, his priorities shifted before he completed this work. He was kind enough to send me the prototype in hope someone could take over. Although I do not know Kerberos, I believe the model is rather complete. Just click on the image below to judge for yourself: This work was released under a LGPL-2+ license. There are some tasks left to get a Kerberos configuration editor. These are mentioned in config-model-kerberos readme file. To look at the code, just clone config-model-kerberos with

git://github.com/dod38fr/config-model-kerberos.git

If you need more information on Config::Model, see:

• https://github.com/dod38fr/config-model/wiki
• http://search.cpan.org/dist/Config-Model/lib/Config/Model/Manual/ModelCreationIntroduction.pod

If you are interested in adopting Kerberos model, please keep us informed on config-model-users at lists.sourceforge.net. This will avoid duplicated work. On my side, I ll gladly answer questions related to Config::Model to help you get started. All the best
Tagged: Config::Model, Kerberos, Perl

I'm a heavy user of Debian's Bug Tracking System (especially during this period of the year ;)) and I quite like it. And as a Debian developer, I have to check multiple sources when looking at some source package. Namely, I mainly check the following sources:

• Packages Tracking System gives almost all information concerning a source package (maintainer, available versions, number of bugs, etc )
• Buildd Status Pages build status for the package on each architecture
• and sometimes Popularity Contest numbers

All this is great since you can move from the first one to the second one (and vice-versa) just by clicking with your mouse. It becomes less handy when you are looking at any bug report. In fact, the BTS doesn't link back to the previously mentioned services [1]. So, you have to copy/paste the name of the package or start a new tab and write down the address of the desired service. All this is not very nice for a lazy person like me :)

A few days ago, I remembered that I was already using GreaseMonkey and thought : "Hey, it would be nice to hack quickly something using GM to add the missing links". After a few tests, I've come up with a (very simple, yet extremely useful to me) user script that I called Debian BTS extra links [2]. It adds what I've been missing for years. If you are lazy and don't want to install right now, you can check the following pages to see what it adds:

• A regular bug report page
• A modified bug report page

The userscript will add links to PTS, Buildd status pages and Popcon near every pkgreport.cgi?src= link. Simple and useful. I hope you'll enjoy it as much as I do.

Of course, I've submitted a bug (#679282) against the pseudo package bugs.debian.org kindly asking its maintainer to add a link to (for now) the PTS. But I know that they are busy so I'm not expecting a result soon. Fow now, my simple and dumb userscript WFM.

[1] This is partly true since it links back to the PTS on main pages (pages that list all bug reports of a given set of packages).

[2] Only tested with a recent Iceweasel (from experimental) and Chromium (from unstable).

P.S.: If you do wonder, #4300 is the least bug number known to the BTS. Well, or at least, the result of running ls -1 /srv/bugs.debian.org/spool/archive/00/ sort -h head -n1 on bugs.debian.org.