I'm at DebConf 12 and I've decided to use my time to clear out some minor bits and pieces I've been planning for a while. One of these was to do some graphing of the Debian keyrings over time. The bzr repository goes back to March 2008, but I've also got copies of keyrings for releases back to slink (February 1999). I've been a long time user of GD::Graph under Perl, but recently discovered SVG::TT::Graph and have been meaning to play with it. So I did. First up, number of keys in each keyring: Most of the interesting data is towards the right, but we can also see the point where our v4 keyring overtook v3 keys back in 2001. More recently there's the end of our v3 support in 2010, and the steady increase of Debian Maintainers. The tiny green line is the Debian non-upload keyring. Next I looked at key size (limiting myself to the DDv4 keyring to make things simpler): Here we can see the steady increase of 4096 bit keys since 2009, and to a lesser extent 2048 bit keys. There are a few other sizes - 1 10k key, 1 8k key and 2 3k keys (I suspect these are tied to OpenPGPv2 cards). We're up to 28% of the keyring being stronger keys, but there's still a long way to go. (Interestingly the Debian Maintainer keyring is much better with 65% of keys being 2k or larger. The non-upload keyring is all 2k or greater.) Finally I graphed key type, again limiting myself to the DDv4 keyring: No real surprises here; DSA far and away the most common with RSA usage increasing as part of the move to larger key sizes. In the past we had a few Elgamal signing keys, but these were shown to be compromised thus disappeared entirely. What do these graphs show me? At least the following:

• Debian has a steady rate of growth, for both DDs and DMs. As Zack mentioned in his keynote yesterday it would be nice to see more non-packaging contributors.
• We've made a noticeable effort towards transitioning to stronger keys, but there's still a lot of people who need to make the switch.
• Our rate of growth has slowed over the years (not really surprising).

(You should be able to click on the graphs for larger version.)

A year ago, I bought a Logitech Wireless Solar Keyboard K750. I'm particularly picky on keyboards, but this one is good. It has an incredible useful feature: while being wireless, it has no need for disposable or rechargeable batteries, it uses solar power! My problem is that there's obviously no way to know the battery status from Linux, the provided application only working on Windows. And one dark night, while fragging on QuakeLive, my keyboard stopped working: it had no battery left. This activity being quite energy consuming, it emptied the whole battery. Someone should write code to get the battery status and light meter from Linux: challenge accepted! How the keyboard works This keyboard, like many of the new wireless devices from Logitech, uses the Unifying interface. It's an USB receiver that can be attached up to 6 differents devices (mouse, keyboards ). On old Linux kernel, the Unifying receiver is recognized as only one keyboard and/or one mouse device. Recently, a driver called hid-logitech-dj has been added to the Linux kernel. With this driver, each device attached to the receiver is recognized as one different device. What the Logitech application does The Logitech application under Windows works that way: you launch it, and it displays the battery charge level. On the keyboard, there's a special "light" button (up right). When pressed, a LED will light up on the keyboard: green if the keyboard is receiving enough light and is charging, red if the keyboard does not receive enough light and is therefore discharging. Pushing this same button while the application is running will makes the light meter activated: the application will tell you how much lux your keyboard is receiving. Let's reverse engineer this As far as I know, there's nothing in the USB HID protocol that handles this kind of functionality (battery status, light meter ) in a standard way. So the first task to accomplish is, unfortunately, to reverse engineer the program. I discovered a bit too late that Drew Fisher did a good presentation on USB reverse engineering at 28c3. You might want to take a look at it if you want to reverse engineer on USB. I did not need it, but I learned a few things. Anyway, my plan was the following: run the Logitech application inside a virtual machine running Windows, give it direct access to the USB keyboard, and sniff what happens on the USB wire. To achieve that, you need a virtual machine emulator that can do USB pass-through. Both KVM and VirtualBox can do that, but VirtualBox works much better with USB and allow hot(un)plugging of devices, so I used it. To sniff what happens on the USB, you need to load the usbmon Linux kernel module. Simply doing modprobe usbmon will work. You can then use Wireshark which know how to use usbmon devices and understand the USB protocol. USB stuff you need to know You don't need to know much about USB to understand what I'll write about below, but for the sake of comprehensibility I'll write a couple of things here before jumping in. To communicate with an USB device, we communicate with one of its endpoints. Endpoints are regrouped into an interface. Interfaces are regrouped into a configuration. A device might contains one or several configurations. There's also several types of packets in the USB wire protocol, and at least two of them interest us there, they are:

• Interrupt packets, a packet send spontaneously;
• Controls packets, used for command and status operations.

All of this and more is well (and better) explained in the chapter 13 of Linux Device Drivers, Third Edition. Sniffed data Once everything was set-up, I ran my beloved Wireshark. There's a an URB of type interrupt sent each time you press any key with some data in it. Therefore I advise you to plug another keyboard (or use the laptop keyboard if you're doing this on a laptop), otherwise you'll get crazy trying to sniff the keyboard you're typing on. At this point, just launching the application does a bunch of USB traffic. Pressing the "light" button on the keyboard makes even more USB packets coming in and out. Here's the interesting packets that I noticed once I excluded the noise:

• When pressing the "light" button, an URB of type interrupt is sent by the keyboard to the computer;
• An URB control packet is sent by the computer to the keyboard in response;
• Regularly URB interrupt packets are sent just after.

With all this, the next step was clear: understand the packets and reproduce that exchange under Linux. What the packets mean The "go for the light meter" packet The packet sent from the computer to the keyboard is the following. <figure>

Frame 17: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
    Frame Length: 71 bytes (568 bits)
    Capture Length: 71 bytes (568 bits)
USB URB
    URB id: 0xffff880161997240
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x00, Direction: OUT
        0... .... = Direction: OUT (0)
        .000 0000 = Endpoint value: 0
    Device: 6
    URB bus id: 1
    Device setup request: relevant (0)
    Data: present (0)
    URB sec: 1340124450
    URB usec: 495643
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 7
    Data length [bytes]: 7
    [Response in: 18]
    [bInterfaceClass: HID (0x03)]
    URB setup
        bmRequestType: 0x21
            0... .... = Direction: Host-to-device
            .01. .... = Type: Class (0x01)
            ...0 0001 = Recipient: Interface (0x01)
    bRequest: SET_REPORT (0x09)
    wValue: 0x0210
        ReportID: 16
        ReportType: Output (2)
    wIndex: 2
    wLength: 7
0000  40 72 99 61 01 88 ff ff 53 02 00 06 01 00 00 00   @r.a....S.......
0010  22 ad e0 4f 00 00 00 00 1b 90 07 00 8d ff ff ff   "..O............
0020  07 00 00 00 07 00 00 00 21 09 10 02 02 00 07 00   ........!.......
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040  10 01 09 03 78 01 00                              ....x..

</figure> What's here interesting is the last part representing the data. wLength says that the length of the data is 7 bytes, so let's take a look at those 7 bytes: 10 01 09 03 78 01 00. Well, actually, you can't decode them like that, unless you're a freak or a Logitech engineer. And I have actually no idea what they mean. But sending this to the keyboard will trigger an interesting thing: the keyboard will start sending URB interrupt with some data without you pressing any more key. The "light meter and battery values" packet This is most interesting packet. This is the one sent by the keyboard to the host and that contains the data we want to retrieve. <figure>

Frame 1467: 84 bytes on wire (672 bits), 84 bytes captured (672 bits)
    Frame Length: 84 bytes (672 bits)
    Capture Length: 84 bytes (672 bits)
USB URB
    URB id: 0xffff88010c43c380
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_INTERRUPT (0x01)
    Endpoint: 0x83, Direction: IN
        1... .... = Direction: IN (1)
        .000 0011 = Endpoint value: 3
    Device: 2
    URB bus id: 6
    Device setup request: not relevant ('-')
    Data: present (0)
    URB sec: 1334953309
    URB usec: 728740
    URB status: Success (0)
    URB length [bytes]: 20
    Data length [bytes]: 20
    [Request in: 1466]
    [Time from request: 0.992374000 seconds]
    [bInterfaceClass: Unknown (0xffff)]
Leftover Capture Data: 1102091039000c061d474f4f4400000000000000
0000  80 c3 43 0c 01 88 ff ff 43 01 83 02 06 00 2d 00   ..C.....C.....-.
0010  5d c5 91 4f 00 00 00 00 a4 1e 0b 00 00 00 00 00   ]..O............
0020  14 00 00 00 14 00 00 00 00 00 00 00 00 00 00 00   ................
0030  02 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00   ................
0040  11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00   ....9....GOOD...
0050  00 00 00 00                                       ....

</figure> This packets come in regularly (1 per second) on the wire for some time once you sent the "go for the light meter" packet. At one point they are emitted less often and do not contain the value for the light meter anymore, suggesting that the control packet sent earlier triggers the activation of the light meter for a defined period. Now you probably wonder where the data are in this. They're in the 20 bytes leftover in the capture data part, indicated by Wireshark, at the end of the packet: 11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00. Fortunately, it was easy to decode. Knowing we're looking for 2 values (battery charge and light meter), we just need to observe and compare the packet emitted on the wire with the values displayed by the Logitech Solar App. To achieve this, I looked both at the Logitech Solar App and Wireshark while bringing more and more light near the keyboard, increasing the lux value received by the meter on the Solar App, and saw that the fields represented in blue (see below) where changing in Wireshark. Since 2 bytes were changing, I guessed that it was coded on 16 bits, and therefore it was easy to correlate the value with the Solar App. <figure>

[ ....9....GOOD....... ]
11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00
4 bytes - 1 byte for battery charge - 2 bytes for light meter - 2 bytes - 4 bytes for GOOD - 7 bytes

</figure> In this example, the battery has a charge of 0x39 = 57 % and the light meter receives 0x0c = 12 lux of light. It's basically dark, and that makes sense: it was night and the light was off in my office, the only light being the one coming from my screen. I've no idea what the GOOD part of the packet is about, but it's present in every packet and it's actually very handy to recognize such a packet. Therefore I'm considering this as some sort of useful mark for now. For the other bytes, they were always the same (0x11 0x2 0x9 0x10 at the beginning, 7 times 0x00 at the end). The 2 bytes between the light meter and GOOD probably mean something, but I've no idea what for now. Building our solar app Now we've enough information to build our own very basic solar application. We know how to triggers the light meter, and we know how to decode the packets. We're going to write a small application using libusb. Here's a quick example. It's not perfect and does not check for error codes, be careful. <figure>

/* Written by Julien Danjou <julien> in 2012 */
#include <linux/hid.h>
#include <libusb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
 
    libusb_context *ctx;
    libusb_init(&ctx);
    libusb_set_debug(ctx, 3);
    /* Look at the keyboard based on vendor and device id */
    libusb_device_handle *device_handle = libusb_open_device_with_vid_pid(ctx, 0x046d, 0xc52b);
    fprintf(stderr, "Found keyboard 0x%p\n", device_handle);
    libusb_device *device = libusb_get_device(device_handle);
    struct libusb_device_descriptor desc;
    libusb_get_device_descriptor(device, &desc);
    for(uint8_t config_index = 0; config_index < desc.bNumConfigurations; config_index++)
     
        struct libusb_config_descriptor *config;
        libusb_get_config_descriptor(device, config_index, &config);
        /* We know we want interface 2 */
        int iface_index = 2;
        const struct libusb_interface *iface = &config->interface[iface_index];
        for (int altsetting_index = 0; altsetting_index < iface->num_altsetting; altsetting_index++)
         
            const struct libusb_interface_descriptor *iface_desc = &iface->altsetting[al>tsetting_index];
            if (iface_desc->bInterfaceClass == LIBUSB_CLASS_HID)
             
                libusb_detach_kernel_driver(device_handle, iface_index);
                libusb_claim_interface(device_handle, iface_index);
                unsigned char ret[65535];
                unsigned char payload[] = "\x10\x02\x09\x03\x78\x01\x00";
                if(libusb_control_transfer(device_handle,
                                           LIBUSB_REQUEST_TYPE_CLASS   LIBUSB_RECIPIENT_INTERFACE,
                                           HID_REQ_SET_REPORT,
                                           0x0210, iface_index, payload, sizeof(payload) - 1, 10000))
                 
                    int actual_length = 0;
                    while(actual_length != 20   strncmp((const char *) &ret[9], "GOOD", 4))
                        libusb_interrupt_transfer(device_handle,
                                                  iface_desc->endpoint[0].bEndpointAddress,
                                                  ret, sizeof(ret), &actual_length, 100000);
                    uint16_t lux = ret[5] << 8   ret[6];
                    fprintf(stderr, "Charge: %d %%\nLight: %d lux\n", ret[4], lux);
                 
                libusb_release_interface(device_handle, iface_index);
                libusb_attach_kernel_driver(device_handle, iface_index);
             
         
     
    libusb_close(device_handle);
    libusb_exit(ctx);
 

</figure> What the program is doing is the following:

• Request for the Unifying Receiver device based on vendor and product ID
• Get the HID interface
• Detach the HID interface from the kernel driver
• Claim the interface
• Send a control packets, were parameters are defined using the same data we captured earlier
• Read interrupt packets coming in until we receive one we recognize (length 20 containing the "GOOD" string)
• Decode the content (battery charge & light meter)
• Release the interface
• Reattach the kernel driver to the interface

This gives the following:

Found keyboard 0x0x24ec8e0
Charge: 64 %
Light: 21 lux

Challenge accomplished! To be continued Unfortunately, this approach has at least one major drawback. We have to disconnect the Logitech Unifying Receiver from the kernel. That means that while we're waiting for the packet, we're dropping packets corresponding to other events from every connected device (key presses, pointer motions ). In order to solve that, I sent a request for help on the linux-input mailing list. That way, I learned that Logitech is actually using the HID++ protocol to communicate with the devices using the Unifying Receiver. Lars-Dominik Braun managed to get the HID++ specifications from Logitech and published them with their authorization. This opens a whole new world. With that document, I may be able to understand the part I reverse engineered and convert this to a more useful and generic library using the hidraw interface (so we don't have to disconnect the devices from the kernel driver).

Thanks to the awesome work of a lot of people (some not even here), this time we did the setup in one day less then previous years, and IMHO it was mostly rather very relaxed and efficient as well!
As you probably will know already, the streams are linked on the DebConf12 webpage and the recordings will be linked there, too, when they are ready. The scheduled release for that is tomorrow evening localtime, we ll see if we can deliver :-)
To make this happen, there will be a video training session tomorrow morning at 11:00 in talkroom 2, Roberto Ter n. The training will cover both the live session setup and the reviewing workflow and is definitly still open for new members!
So once again (one cannot say it enough, so please express your thanks as well) muchisimas gracias to all members of the dc12 videoteam! As Biella once said, you re ninjas!
Last and definitly not least, many thanks to our equipment sponsors too (HP, IRILL, Debian and individuals)!
cheers, a very happy h01ger!

I am in Berlin for the Wikipedia Academy, a very cool hybrid free culture community plus refereed academic conference organized, in part, by Wikimedia Deutschland. On Friday, I was very excited to have been invited to give the conference's opening keynote based on my own hybrid take on learning from failures in peer production and incorporating a bunch of my own research. Today, I was on a panel at the conference about free culture and sharing practices. I'll post talks materials and videos when the conference puts them online. I will be in Berlin for the next week or so before I head to directly to Washington, DC for Wikimania between the 11th and 15th. I'll be giving three talks there:

• Almost Wikipedia: A What Eight Collaborative Encyclopedia Projects Reveal About Mechanisms of Collective Action
• Can Social Awards Create Better Wikis? (with Aaron Shaw)
• The State of Wikimedia Scholarship (2011-2012) (with any poor souls I can recruit into helping me in the next week or so)

Between then and now, I'm taking the next week in Berlin to catch up on work, and with friends. If you're in either place and want to meet up, please get in touch and lets try to arrange something.

Since last November, I've been participating in and coordinating Iron Blogger: a drinking club where you pay $5 to a "beer" pool if you fail to blog weekly. The revival of Iron Blogger in Boston has been a big success. Even more exciting, however, is that Iron Blogger concept has spread. There are now two other Iron Blogger instances: in San Francisco coordinated by Parker Higgens, and in Berlin run by Nicole Ebber and Michelle Thorne. Yesterday, we convened a virtual meeting of the Global Iron Blogger Council (i.e., an email thread) and we all agreed a new on iron blogger rule that might sweeten the deal for jet-setting prospective Iron Bloggers: any paid-up member of any Iron Blogger club can attend meet-ups in any other Iron Blogger cities if they happen to be in town for one. Because We Are One. If you want to join us in Boston, we have some room through attrition. Rust bloggers, perhaps? If you'd like to join, you should contact me. And if you'd like to set up your own in a different city, the code is in git. One warning, however. As those of us that have set it up have figured out, the documentation for the software to run Iron Blogger is between poor and non-existent. If you do want to set up your own instance, please get in touch. I'm happy to give you some pointers that you'll probably need but, more importantly, I'd like to work with the next brave soul to put together documentation of the setup process along the way.

I ve just broken my second SATA connector. This isn t a lot considering the number of hard drives I ve worked with, but it s still really annoying as I generally don t break things. The problem is that unplugging a SATA cable requires pushing a little clip, this isn t overly difficult but it unfortunately doesn t fit well with habits formed from previous hardware. The power cables used for hard drives based on the ST-506 interface which was copied for the IDE interface was large and had a fairly tight fit. Removing such a cable requires a significant amount of force which is about the same as the amount of force required to break a SATA connector. When I first started using PCs a reasonably configured AT system cost over $5,000 (maybe something like $10,000 in today s money). With that sort of price hardly anyone had a set of test PCs. When hardware prices dropped such that hard drives of reasonable size became reasonably affordable on the second-hand market I bought more disks and used some for extra storage and some for testing software. As there was nothing like VMWare for testing OS images the way to test a new OS was to plug in a different hard drive and boot it. So I got a lot of practice at removing IDE power cables with as much force as was necessary. Now I own a pile of test PCs, SATA disks less than 100G are free, I use Xen for a lot of my testing, and generally I have much less need to swap hard drives around. In most situations in which I would swap hard drives in the 90 s I will now swap PCs and I have piles of PCs ready for this purpose. So I haven t had enough practice with SATA disks to develop habits for safely removing them. So far this lack of habit development has resulted in damaging two disks due to changing drives while not concentrating enough. Fortunately duct-tape works well for holding a SATA connector in place when the plastic that attaches to the clip is broken. Related posts:

1. Strange SATA Disk Performance Below is a GNUPlot graph of ZCAV output from a...
2. Vibration and Strange SATA Performance Almost two years ago I blogged about a strange performance...
3. Dell PowerEdge T105 Today I received a Dell PowerEDGE T105 for use by...

Over a year ago when I was considering my first Android phone purchase I setup a test account on my mail server so that I could test email clients on phones and tablets. I used a short password because I didn t want to type a lot on small screens and because typing a password into a random system owned by someone else isn t particularly secure anyway. Then I forgot about the account until I noticed that my mail server was sending out spam. Next time I setup such a test account I ll put rules similar to the following in my Postfwd [1] configuration to stop Postfix from sending such messages. That will prevent the test account from receiving mail from outside or sending mail out of the server. The former is optional (getting a few thousand spam messages in an unused test account is no big deal) but the latter is needed to prevent getting my server blacklisted. id=R_test_recipient ; recipient==test@coker.com.au ; sender!~.*@coker.com.au ; action=REJECT
id=R_test_sender ; sender==test@coker.com.au ; recipient!~.*@coker.com.au ; action=REJECT

• [1] http://postfwd.org/

Related posts:

1. email disclaimers Andre Pang blogs about the annoyance of email disclaimers. For...
2. Email Passwords I was doing some routine sysadmin work for a client...
3. Some Postfix Scripts for dealing with Outbound Spamming I ve just written some small scripts to help me manage...

I've been working the past two days on debian-cd, which was the main thing (besides git-annex assistant) I planned to work on at DebCamp. Yesterday, Steve McIntyre and I cleaned up some cruft in debian-cd's package lists. This freed somewhere in the area of 30 mb. I also took a grep through d-i and made sure the CDs include packages that d-i can optionally install in some situations. Today, I investigated how Recommends are ordered on the CD, and concluded it's as close to optimal as can be easily achieved. So was not able to save space there, but I did find a way to reorganize the desktop task that avoids needing to include a lot of printing stuff and some other stuff on the first CD. While that helped some, it still didn't get either Gnome or Kde to entirely fit, so getting there will probably involve rebuilding 100 or so packages with xz, if someone decides to do that. So it's stil TBD whether Gnome or Kde will fit on a single CD in Debian wheezy. At this point I think most of us are getting tired of fighting this increasingly losing battle every release, and so other options like only having a desktop DVD are looking more appealing, as they solve the problem long-term. This would also free up the first CD for other interesting use cases, perhaps xfce, or perhaps a CD targeted at server users, and/or containing high-popularity non-desktop packages.

A couple of days ago, I commented about my thoughts on finding a better way to listen to music, and asked for suggestions. I ve checked out the avenues suggested, and here are my thoughts so far. Streaming service: Spotify Of the streaming service offerings, Spotify seems the most compelling. It has both Linux and Android clients. The Linux client is unofficial and a bit buggy, but serviceable. The service is fairly nice, and the serendipity of listening to radio of tracks like any track or artist I may pick is an incredible feature. It will mean a different mindset; rather than carefully choosing what I buy, I have a vast catalog at my fingertips. I d have to give up some control, but might be better off for it. Spotify can play local files, but sadly lacks support for FLAC that most of my collection is in. A little work with the shell, and I had a quick multi-core FLAC to MP3 conversion done. It can also identify local files and match them up with its catalog, but it does a somewhat poor job of it, despite the fact that my files are tagged by Musicbrainz. There are also a fair number of albums in my collection (from Magnatune and from local artists) that aren t in Spotify s catalog and probably never will be. I may wind up using git-annex or something like it to sync them between PCs. Surprisingly, it doesn t have any kind of web-based player available. It has a lot of features, but many of them are under-documented. Subsonic / Supersonic The idea of being able to stream music anywhere certainly has appeal, and the idea of streaming that music from my own machine my own collection is quite the appealing one. Subsonic seems to be the standard that people recommend for this. It has a web-based player that works well, as well as mobile clients for various platforms. The Android mobile client is particularly well-regarded and is probably the best one here, aside from Spotify. The playlists work well, as expected, and generally it has the appearance of a well-rounded system. There is a lot of flexibility in bitrates; maximum bitrates can be set per-user and per-client. It, of course, transcodes on the fly. My chief gripe about Subsonic is that it has no true album/artist/genre browser. Its main browser is mostly a filesystem browser. I have used clients that show album/artist/genre panes or sortable lists, based on tags, for so long that my filesystem organization is rather different centered around bitrate of rip, origin, etc. It is not very easy to navigate my collection in that way. The search feature can search artist and album, and this can partially make up for it. But there is simply no way to see a list of all artists in the collection. The server is written in Java, and takes up 362MB at startup. That s, I guess, decent for a Java server, but doesn t make me pleased as I d be running it on lower-end hardware. Supersonic is a git-maintained actively synced subsonic fork which removes the license cost (the source is GPL3 anyhow so that wasn t a big deal). Ampache Often mentioned in the same breath as Subsonic, this is yet another frequent suggestion for streaming your own collection. Its primary player is web-based, like Subsonic. It has full artist/genre browsers, and adds tag cloud and various other options as well by default, it loaded the tag cloud with my genre tags, which was a nice touch. The search feature is a little less robust, as it simply searches anywhere and returns a list of all matching tracks, rather than listing matching albums and artists before going into the list of all tracks. But that is somewhat of a minor nit. Ampache s web-based player, liks Subsonic s, uses Flash, but the implementation is really quite awkward. You can t simply click a play button and have playback start. Rather, you must add things to the playlist (queue), a frame on the web interface. Then, you must click play in THAT frame, which launches a new window for the player and transfers its contents to it. You thus have two separate running playlists. And if you think you can just add to the playlist of the player window, think again; you can only replace it. It is simply not possible to add a track to the end of that list without losing your existing place. This makes the usefulness of the web-based player rather, well, low. Playlist functionality in Ampache is rather odd. You can have playlists, but if you want to add a track to a saved playlist, first you have to clear you current playlist, then add the track to it, then save it to the existing playlist. Frontends for Ampache exist in banshee and in amarok. Banshee s is broken, and simply silently fails to do anything. Unless you look in the console, in which case you ll see a .NET backtrace. Banshee has had these mysterious errors for me for years. The Amarok plugin is so basic that you can browse by artist but not by album, and is almost unusable. There is also a rhythmbox plugin in Debian, but it is apparently for an old version of Rhythmbox and no longer works. The best frontend is a dedicated Ampache frontend called Viridian. It has a nice interface, provides features such as taskbar integration, and generally works pretty well. The only thing it s bad at is playlists, and here it s even worse than the web interface. It doesn t support modifying the server-side playlists at all. And accessing one is an annoying load playlist operation, that replaces your current play queue (also called the playlist ) with its contents by default. Several Android clients exist for Ampache, though it seems only one has seen any updates since 2010. mpd This is an interesting program, as it is designed more to provide multiple points of control for a single player than it is to stream a single collection to multiple devices. Nevertheless, I tested its HTTP and icecast stream modules. (It can also stream with PulseAudio, but PulseAudio causes devious breakage on my workstation, so I refuse to bother.) This could be made to work reasonably well on my LAN using the gmpc client. There are several Android clients as well, though none of them integrate as well as Subsonic or Spotify. Most seem focused on controlling a remote player. The ones that do stream do so with some hiccups, and don t integrate controls with the lock screen. gmpc is a reasonable control program and, combined with mplayer, makes a decent player. It can browse the database every way I d want to. It doesn t let me right-click on tracks to add them to playlists, but I can use copy/paste to do so, or add them to the play queue and from there to a playlist. It s annoying, but not completely unworkable. But here s the real concern. mpd has no provision whatsoever for any sort of encryption such as SSL. If I want to be listening to my music collection from on the road, I don t really wish to open up a streaming music service for the entire Internet. Nor do I really want to mess with SSH tunnels and the like, which are cumbersome on Android (and can be in general). This is pretty much a fatal flaw to me. Google Music An interesting service, but ultimately one I don t see the point for in my situation where I already have my music on a PC that is always on. It would probably accommodate my entire collection free, but how many weeks would it take me to get it there? And how would it stay in sync? It simply doesn t offer me any compelling advantage over streaming direct from my PC, so I don t see much need to bother with it. Others I briefly considered Streeme (poor mobile support), Audiogalaxy (no Linux support), Jinzora (appears dead), and Sockso (no mobile support). I also briefly considered other streaming services such as Pandora, Slacker, rdio, etc., but based on reviews didn t think that they were worth bothering with. Conclusions I m left mainly considering Spotify and Subsonic. For right now, I m going to try Spotify and see where that leads me, but may wind up getting back to Subsonic if I can t find tracks I want with Spotify.

I've been lazy today: I just went to the same place than yesterday..:-) I actually waslate and missed the meeting with No l who indeed went running in about the same neighbourhood, as well as Martin Bagge who went running on the road to the "Intermezzo" restaurant I described earlier..:-) So, no real fancy description today. And, as usual, the full GPS trace is here. The interesting challenge is to see if I can manage to achieve a run every day..:-). We'll see!

When I first undertook the tracking of minor security fixes in point releases, I quickly out-scaled flat text files and a good memory. A Python library and sqlite database helped automate sending notifications and keeping tabs, but the manual work associated with tracking incoming bugs from the security team, applications to and responses from the release team, and the action or inaction of maintainers was still too time-consuming to be useful. This weekend I deployed pyprsc2, with a public view at http://prsc.debian.net/tracker/<bug>. I had planned to do this at Debconf12, but given the circumstances still, it needed doing anyway and what better time? Result: my work now involves adding tracks where required; keeping an eye on the notified list for manual prods; and after a point release, archiving the included bugs and updating the suite version numbers. Bliss. Features:

• automatic sync of metadata for new tracks
• automatic detection of bugs closed from unstable
• automatic notification (and unarchive if required)
• automatic, though manually triggered, prods
• automatic detection of stable/oldstable upload and RT acceptance
• public view of bug status, e.g. http://prsc.debian.net/tracker/660650

Todo:

• finish the views so that bug lists work, as well as detail
• add process documentation pages
• iron out the mailing so bugs.debian.org doesn t throttle back submissions
• add ways to permit more contributors
• refactor where required; this was a learning exercise too
• publish the code

Technical: prsc.debian.net leverages large parts of the Django MVC framework in fact, this was really a learning exercise in disguise since I want to use Django on some more complex projects later. BTS synchronisation is handled by python-debianbts, and synchronisation with proposed-updates is through XML and lxml/objectify (thanks to the release team s awesome XML queue viewer and Adam adding bug numbers to it). Since this was a learning exercise, some of the Python is probably questionable at best and downright wrong at worst, so it probably needs some work still.

---

Point Release Security, Reloaded is a post from: jwiltshire.org.uk Flattr

The Debian Edu / Skolelinux project have users all over the globe, but until recently we have not known about any users in Norway's neighbour country Sweden. This changed when George Bredberg showed up in March this year on the mailing list, asking interesting questions about how to adjust and scale the just released Debian Edu Wheezy setup to his liking. He granted me an interview, and I am happy to share his answers with you here. Who are you, and how do you spend your days? I'm a 44 year old country guy that have been working 12 years at the same school as 50% IT-manager and 50% Teacher. My educational background is fil.kand in history and religious beliefs, an exam as a "folkhighschool" teacher, that is, for teaching grownups. In Norwegian I believe it's called "Vuxenupplaring". I also have a master in "Technology and social change". So I'm not really a tech guy, I just like to study how humans and technology interact and that is my perspective when working with IT. How did you get in contact with the Skolelinux/Debian Edu project? I have followed the Skolelinux project for quite some time by now. Earlier I tested out the K12-LTSP project, which we used for some time, but I really like the idea of having a distribution aimed to be a complete solution for schools with necessary tools integrated. When K12-LTSP abandoned that idea some years ago, I started to look more seriously into Skolelinux instead. What do you see as the advantages of Skolelinux/Debian Edu? The big point of Skolelinux to me is that it is a complete distribution, ready to install. It has LDAP-support, MS Windows integration tools and so forth already configured, saving an administrator a lot of time and headache. We were using another Linux based thin-client system called Thinlinc, that has served us very well. But that Skolelinux is based on VNC and LTSP, to me, is better when it comes to the kind of multimedia used in schools. That is showing videos from Youtube or educational TV. It is also easier to mix thin clients with workstations, since the user settings will be the same. In our VNC-based solution you had to "beat around the bush" by setting up a second, hidden, home-directory for user settings for the workstations, because they will be different from the ones used on the thin clients. Skolelinux support for diskless workstations are very convenient since a school today often need to use a class room projector showing videos in full screen. That is easily done with a small integrated media computer running as a diskless workstation. You have only two installs to update and configure. One for the thin clients and one for the workstations. Also saving a lot of time. Our old system was also based on Redhat and CentOS. They are both very nice distributions, but they are sometimes painfully slow when it comes to updating multimedia support and multimedia programs (even such as Gimp), leaving us with a bit "oldish" applications. Debian is quicker to update. What do you see as the disadvantages of Skolelinux/Debian Edu? Debian is a bit too quick when it comes to updating. As an example we use old HP terminals as thinclients, and two times already this year (2012) the updates you get from the repositories has stopped sound from working with them. It's a kernel/ALSA issue. So you have to be more careful properly testing the updates before you run them in a production environment. This has never happened with CentOS. I also would like to be able to set my own domain-settings at install time. In Skolelinux they are kind of hard coded into the distribution, when it comes to LDAP and at least samba integration. That is more a cosmetic/translation issue, and not a real problem. Running MS Windows applications within the Skolelinux environment needs to be better supported. That is, running them seamlessly via RDP, and support for single-sign on. That will make the transition to free software easier, because you can keep the applications you really need. No support will make it impossible if you work in a school where some applications can't be open source. As for us we really need to run Adobe InDesign in our journalist classes. We run a journalist education, and is one of the very few non university ones that is ok:d by Svenska journalistf rbundet (Swedish journalist association). Our education gives the pupils the right of membership there, once they are done. This is important if you want to get a job. Adobe InDesign is the program most commonly used in newspapers and magazines. We used Quark Express before, but they seem to loose there market to Adobe. The only "equivalent" to InDesign in the opensource world is Scribus, and its not advanced enough. At least not according to the teacher. I think it would be possible to use it, because they are not supposed to learn a program, they are supposed to learn how to edit and compile a newspaper. But politically at our school we are not there yet. And Scribus lacks a lot of things you find i InDesign. We used even a windows program for sound editing when it comes to the radio-journalist part. The year to come we are going to try Audacity. That software has the same kind of limitations compared to Adobe Audition, but that teacher is a bit more open minded. We have tried Ardour also, but that instead is more like a music studio program, not intended for the kind of editing taking place in a radio studio. Its way to complex and the GUI is to scattered when you only want to cut, make pass-overs, add extra channels and normalise. Those things you can do in Audacity, but its not as easy as in Audition. You have to do more things manually with envelopes, and that is a bit old fashion and timewasting. Its also harder to cut and move sound from one channel to another, which is a thing that you do frequently because you often find yourself needing to rearrange parts of the sound file. So, I am not sure we will succeed in replacing even Audition, but we will try. The problem is the students have certain expectations when they start an education towards a profession. So the programs has to look and feel professional. Good thing with radio, there are many programs out there, that radio studios use, so its not as standardised as Newspaper editing. That means, it does not really matter what program they learn, because once they start working they still have to learn the program the studio uses, so instead focus has to be to learn the editing part without to much focus on a specific software. Which free software do you use daily? Myself I'm running Linux Mint, or Ubuntu these days. I use almost only open source software, and preferably Linux based. When it comes to most used applications its OpenOffice, and Firefox (of course ;) ) Which strategy do you believe is the right one to use to get schools to use free software? To get schools to use free software there has to be good open source software that are windows based, to ease the transition. But it's also very important that the multimedia support is working flawlessly. The problems with Youtube, Twitter, Facebook and whatever will create problems when it comes to both teachers and students. Economy are also important for schools, so using thin clients, as long as they have good multimedia support, is a very good idea. It's also important that the open source software works even for the administration. It's hard to convince the teachers to stick with open source, if the principal has to run Windows. It also creates a problem if some classes has to use Windows for there tasks, since that will create a difference in "status" between classes, so a good support for running windows applications via the thin client (Linux) desktop is essential. At least at our school, where we have mixed level of educations, from high-school to journalist-school. Update 2012-07-09 08:30: Paul Wise tipped me on IRC about three useful sources related to Free Software for radio stations: the LWN article Radio station management with Airtime, Airtime which claim to be a Free open source radio automation software and Rivendell which claim to be complete radio broadcast automation solution. All of them seem useful to the aspiring radio producer.

It's been a week with quite some changes to http.debian.net, the Debian mirrors redirector, and it keeps coping very well with the continuously increasing traffic: over 1.5 million requests from APT clients in the last seven days! half-million more from the week before.

The good news is, those rare Hash Sum mismatch errors should be mostly gone. Ditto for some other sort of errors. There is now a second server that takes care of monitoring the mirrors and is ready to handle some of the traffic if there's any need. With this new server, http.debian.net will soon be available over IPv6 too.

Those who are at Debconf12 and followed my advice to use http.debian.net will have noticed that it is redirecting users to the local mirror. So, once again, forget about switching mirrors!

Thanks are also due to J rg Jaspert and Philipp Kern, for the new server and for the work needed to allow http.debian.net access to Debconf12's local mirror, respectively.

Many thanks again to those who keep providing feedback and have helped the project along the way.

What's next? even more improvements and fixing some issues, some of which involve further collaboration and cooperation with the mirror administrators.

Some parts of the Debian mirrors network are fragile and may bite every once in a while, but those are being worked on. If you administer a mirror, please use ftpsync and submit your Debian mirror.
You might be happy to know that I'm working on restricting mirrors to specific Autonomous System, or countries. More on this later.

So, I was trying to write a script for myself to sanitise the EXIF metadata in images produced by my camera. I had a text file for a while with some tags that I should remove (like the camera serial number, etc.), but this became cumbersome and I wanted a fully scripted workflow. I thought initially it's a simple exercise: look at the current tags, decide either which one I wanted to remove (black-list) or alternatively keep (white-list; preferred solution) and then call exiftool with the appropriate options. Boy oh boy EXIF is just part of the equation. There are of course also the MakerNotes, which are manufacturer-specific and most of the time editable only as a block. Furthermore, there's no way to easily just duplicate each individual entry in this group to a standard EXIF or even XMP section, so basically you either keep to much or just give up a lot. Some of them can be copied to XMP, for example lens information, flash, etc. but white-balance (with the exiftool version that I have) is not, for example. Ideally I could just copy all maker notes to XMP, and then drop that entire group. For the record, the way to drop all and just copy some is to run:

exiftool -all= -tagsfromfile @ -Orientation -AllDates  other tags  file.jpg

But do not add "-makernotes", because that will bring the entire block back. The other fun thing is the number of thumbnails/preview images embedded. Looking at a random jpeg file, I found:

• a thumbnail image in the EXIF section, indeed small, around ~8KiB in size (160 120 pixels)
• a preview image in the MakerNotes section, somewhat bigger, around ~24KiB in size (570 375 pixels)
• a large thumbnail (full HD equivalent) in the MPF section, with an incredible size of ~756KiB (1632 1080)

Now, I doubt even the use of one thumbnail, but 3? And one being more than half a megabyte? Someone's going overboard at Nikon On the plus side, removing all thumbnails and sanitising the meta-data makes the jpeg files quite a bit smaller, which also means easier to share, at least; not sure if I want to actually clean the master copies (even though I keep the NEFs).

A few days ago I had blogged about getting NumPy data in R by using a simple converter script. That works fine, but it is a little annoying to have to write an entire file only to read from it again. So I kept looking around for a better solution---and soon found the cnpy library by Carl Rogers which provides simple C++ functions to read and write NumPy files. Bringing such a C++ library to R is done very easily via Rcpp modules. The resulting package contains a single R file with a single line: loadModule("cnpy", TRUE). And it relies on the following module declarations in the a C++ file:

RCPP_MODULE(cnpy) 
    using namespace Rcpp;
    function("npyLoad",                         // name of the identifier at the R level
             &npyLoad,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("type") = "numeric"),
             "read an npy file into a numeric or integer vector or matrix");
    function("npySave",                         // name of the identifier at the R level
             &npySave,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("object"), 
                           Named("mode") = "w"),
             "save an R object (vector or matrix of type integer or numeric) to an npy file");
 

which give us at the R prompt

R> library(RcppCNPy)
Loading required package: Rcpp
R> npyLoad
internal C++ function <0x243af70>
    docstring : read an npy file into a numeric or integer vector or matrix
    signature : Rcpp::RObject npyLoad(std::string, std::string)
R> npySave
internal C++ function <0x23033e0>
    docstring : save an R object (vector or matrix of type integer or numeric) to an npy file
    signature : void npySave(std::string, Rcpp::RObject, std::string)
R> 

these two functions (and their docstrings) defined above. That's all! Well there are about one hundred more lines dealing with whether we have integer or numeric data, and whether we use a vector or a matrix. But all in all pretty simple... So version 0.1.0 of this new package RcppCNPy completes the initial release 0.0.1 from earlier in the week by adding

• the ability to load compressing NumPy files ending in .npy.gz
• a simple regression test suite loading some data sets
• a demo script with a timing example comparing ascii reads to reading npy and compressed npy
• a short pdf vignette describing the package

The NEWS entry for this release (as well as the initial one) follow:

News for Package RcppCNPy Changes in version 0.1.0 (2012-07-07)

• Added automatic use of transpose to automagically account for Fortran-vs-C major storage defaults between Python and R.
• Support for integer types in dependent on the int64_t type which is available only when the -std=c++0x switch is used at build-time (and CRAN still discourages use of it)
• Added support for reading gzip'ed files ending in ".npy.gz"
• Added regression tests in directory tests/
• Added a vignette describing the package
• Added a timing benchmark in demo/timings.R

Changes in version 0.0.1 (2012-07-04)

• Initial version, as a straightforward Rcpp modules wrap around the cpny library by Carl Rogers (on github under a MIT license).
• At present, npy files can be read and written for vectors and matrices of either numeric or integer type. Note however that matrices are currently transposed because of the default Fortran ordering done by numpy.

I will follow up with a little usage example later. CRANberries also provides a diffstat report for 0.1.0 relative to 0.0.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

I've released Obnam version 1.1, released 2012-06-30, but only announced now since I had trouble building the packages for code.liw.fi.

• Mark the --small-files-in-btree settings as deprecated.
• Obnam now correctly checks that --repository is set.
• Options in --help output are now grouped in random senseless ways rather than being in one randomly ordered group.
• Manual page clarification for --root and verify. Thanks, Saint Germain.
• Remove outdated section from manual page explaining that there is not format conversion. Thanks, Elrond of Samba-TNG.
• Added missing information about specifying a user in sftp URLs. Thanks, Joey Hess, for pointing it out.
• Manual page clarification on --keep from Damien Courouss .
• Make obnam forget report which generations it would remove without --pretend. Thanks, Neal Becker, for the suggestion.

I got a few emails about the status panel I'd both toyed with and posted. The end result is that the live load graphs now have documentation, look prettier, and contain a link to the source code. Apart from that this week has mostly involved photographing cute cats, hairy dogs, and women in corsets. In Debian-related news njam: Insecure usage of environmental variable was closed after about 7 months, and I reported a failure of omega-rpg to drop group(games) privileges prior to saving game-state. That leads to things like this:

skx@precious:~$ ls -l   grep games
-rw-r--r--   1 skx games   14506 Jul  8 15:20 Omega1000

Not the end of the world, but it does mean you can write to directories owned by root.games, and potentially over-write level/high-score files in other packages leading to compromises. ObQuote: "Your suffering will be legendary, even in hell! " - Hellraiser II (Did you know there were eight HellRaiser sequels?)

In the Debian Edu / Skolelinux project, we have realised that one of the major blockers for the project success is the purchasing skills in schools and municipalities. We provide what the happy users of Debian Edu / Skolelinux say they need and to a lower cost than the alternatives, and yet so few schools decide to use our solution. I was pleased to discover the same observation done by mySociety and Tom Steinberg in his blog post "Can you recognize the million pound chair?". Read it and weep for the spending of your tax money. Of course there are other factors involved as well, like our projects bad marketing skills and the Linux community fragmentation causing worry with the people on the outside, so we as a project need to keep working hard to gain users, but it is a up-hill battle when public decision makers are unable to understand computer system purchases.

In DebConf9 i worked on some photo editing and ended with a sort of over-saturated album which is still available online. For the C&W party in DebConf11 i tried to delivery a black and white set of pictures by just doing some adjustments using ufraw. Pictures were taken with a Nikon D50 camera + Nikkor 50mm 1/1.4 in raw mode. Here are some examples of the result: You can enjoy these and other DebConf11 shots by following the links in DC11 wikipage. (update: sorry for this old post being re-published in the planet, i've migrated my blog and this post got a new timestamp)