Search Results: "zed"

10 July 2012

Keith Packard: hotplug-displaylink

Getting Hotplugging working with a DisplayLink USB to DVI adapter I merged Dave Airlie s randr provider patches to the server and pushed them out in preparation for freezing the X server for the version 1.13 release. Before freezing, I figured I should at least test hotplugging my DisplayLink adapter. Well, that took all day Sources For the impatient, here s where all of the bits are. These work on my machine. Upstream bits:
  1. randrproto. git://anongit.freedesktop.org/git/xorg/proto/randrproto master
Bits from Dave Airlie s trees:
  1. libdrm. git://people.freedesktop.org/~airlied/drm.git prime
  2. libXrandr. git://people.freedesktop.org/~airlied/libXrandr.git prime
  3. xrandr. git://people.freedesktop.org/~airlied/xrandr.git prime
  4. xf86-video-intel. git://people.freedesktop.org/~airlied/xf86-video-intel prime
Bits from my trees:
  1. xf86-video-modesetting. git://people.freedesktop.org/~keithp/xf86-video-modesetting prime
  2. kernel. git://people.freedesktop.org/~keithp/linux prime
  3. X server. git://people.freedesktop.org/~keithp/xserver master
Kernel adventures The 3.5-rc6 bits have all of the necessary driver changes to support DisplayLink hot-plug. At least, they do if you re not running 32-bit user space atop a 64-bit kernel. If you are, then most of the DRM drivers don t work at all they re missing the .compat_ioctl entry in the file_operations structure, which makes all ioctl calls return -ENOTTY. In particular, the udl driver is missing this entry, so when you plug in the device, the server tries to talk to it and nothing works. X server adventures Once I got the kernel working, I discovered some nice crashing behaviour in the X server. If I started the server with the DisplayLink device plugged in, as soon as I tried to enable the output, the server would segfault. Turns out this cases requires special code in the server to ensure that the DisplayLink screen privates are adjust correctly while the intel driver loads. Modesetting driver fixup The modesetting driver was missing a rename that happened to a structure member in the last couple of days. I d bet Dave has the same patch sitting on a disk somewhere. xrandr changes The command line additions to xrandr are sufficient to get this stuff working, but I think it could use some improvements to make the interface a bit friendlier. In particular, requiring you to use XIDs to identify providers is a bit harsh. Schedules I ll be pushing the X server changes out this evening; review would be appreciated, but I don t think any of the patches I made there are scary. I sent the kernel patches necessary to lkml, but with Dave in transit, I m not sure who is minding the store. As for the rest of the bits, they re sitting in Dave s repositories, presumably they ll get pushed upstream soon. But, does it actually work? Almost. Everything says it s working, but I m not getting any signal to my DVI monitor. My DisplayLink device is ancient, and the kernel complains about it, saying
[drm:udl_parse_vendor_descriptor] *ERROR* Unrecognized vendor firmware descriptor".
So close, and yet

9 July 2012

Julien Danjou: Logitech K750 keyboard and Unifying Receiver Linux support

A year ago, I bought a Logitech Wireless Solar Keyboard K750. I'm particularly picky on keyboards, but this one is good. It has an incredible useful feature: while being wireless, it has no need for disposable or rechargeable batteries, it uses solar power! Logitech Wireless Solar Keyboard K750 My problem is that there's obviously no way to know the battery status from Linux, the provided application only working on Windows. And one dark night, while fragging on QuakeLive, my keyboard stopped working: it had no battery left. This activity being quite energy consuming, it emptied the whole battery. Someone should write code to get the battery status and light meter from Linux: challenge accepted! How the keyboard works Logitech Unifying Receiver This keyboard, like many of the new wireless devices from Logitech, uses the Unifying interface. It's an USB receiver that can be attached up to 6 differents devices (mouse, keyboards ). On old Linux kernel, the Unifying receiver is recognized as only one keyboard and/or one mouse device. Recently, a driver called hid-logitech-dj has been added to the Linux kernel. With this driver, each device attached to the receiver is recognized as one different device. What the Logitech application does Logitech Solar App The Logitech application under Windows works that way: you launch it, and it displays the battery charge level. On the keyboard, there's a special "light" button (up right). When pressed, a LED will light up on the keyboard: green if the keyboard is receiving enough light and is charging, red if the keyboard does not receive enough light and is therefore discharging. Pushing this same button while the application is running will makes the light meter activated: the application will tell you how much lux your keyboard is receiving. Let's reverse engineer this As far as I know, there's nothing in the USB HID protocol that handles this kind of functionality (battery status, light meter ) in a standard way. So the first task to accomplish is, unfortunately, to reverse engineer the program. I discovered a bit too late that Drew Fisher did a good presentation on USB reverse engineering at 28c3. You might want to take a look at it if you want to reverse engineer on USB. I did not need it, but I learned a few things. Anyway, my plan was the following: run the Logitech application inside a virtual machine running Windows, give it direct access to the USB keyboard, and sniff what happens on the USB wire. To achieve that, you need a virtual machine emulator that can do USB pass-through. Both KVM and VirtualBox can do that, but VirtualBox works much better with USB and allow hot(un)plugging of devices, so I used it. To sniff what happens on the USB, you need to load the usbmon Linux kernel module. Simply doing modprobe usbmon will work. You can then use Wireshark which know how to use usbmon devices and understand the USB protocol. USB stuff you need to know You don't need to know much about USB to understand what I'll write about below, but for the sake of comprehensibility I'll write a couple of things here before jumping in. To communicate with an USB device, we communicate with one of its endpoints. Endpoints are regrouped into an interface. Interfaces are regrouped into a configuration. A device might contains one or several configurations. There's also several types of packets in the USB wire protocol, and at least two of them interest us there, they are: All of this and more is well (and better) explained in the chapter 13 of Linux Device Drivers, Third Edition. Sniffed data Once everything was set-up, I ran my beloved Wireshark. There's a an URB of type interrupt sent each time you press any key with some data in it. Therefore I advise you to plug another keyboard (or use the laptop keyboard if you're doing this on a laptop), otherwise you'll get crazy trying to sniff the keyboard you're typing on. At this point, just launching the application does a bunch of USB traffic. Pressing the "light" button on the keyboard makes even more USB packets coming in and out. Here's the interesting packets that I noticed once I excluded the noise: With all this, the next step was clear: understand the packets and reproduce that exchange under Linux. What the packets mean The "go for the light meter" packet The packet sent from the computer to the keyboard is the following. <figure>
Frame 17: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
    Frame Length: 71 bytes (568 bits)
    Capture Length: 71 bytes (568 bits)
USB URB
    URB id: 0xffff880161997240
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x00, Direction: OUT
        0... .... = Direction: OUT (0)
        .000 0000 = Endpoint value: 0
    Device: 6
    URB bus id: 1
    Device setup request: relevant (0)
    Data: present (0)
    URB sec: 1340124450
    URB usec: 495643
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 7
    Data length [bytes]: 7
    [Response in: 18]
    [bInterfaceClass: HID (0x03)]
    URB setup
        bmRequestType: 0x21
            0... .... = Direction: Host-to-device
            .01. .... = Type: Class (0x01)
            ...0 0001 = Recipient: Interface (0x01)
    bRequest: SET_REPORT (0x09)
    wValue: 0x0210
        ReportID: 16
        ReportType: Output (2)
    wIndex: 2
    wLength: 7
0000  40 72 99 61 01 88 ff ff 53 02 00 06 01 00 00 00   @r.a....S.......
0010  22 ad e0 4f 00 00 00 00 1b 90 07 00 8d ff ff ff   "..O............
0020  07 00 00 00 07 00 00 00 21 09 10 02 02 00 07 00   ........!.......
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040  10 01 09 03 78 01 00                              ....x..
</figure> What's here interesting is the last part representing the data. wLength says that the length of the data is 7 bytes, so let's take a look at those 7 bytes: 10 01 09 03 78 01 00. Well, actually, you can't decode them like that, unless you're a freak or a Logitech engineer. And I have actually no idea what they mean. But sending this to the keyboard will trigger an interesting thing: the keyboard will start sending URB interrupt with some data without you pressing any more key. The "light meter and battery values" packet This is most interesting packet. This is the one sent by the keyboard to the host and that contains the data we want to retrieve. <figure>
Frame 1467: 84 bytes on wire (672 bits), 84 bytes captured (672 bits)
    Frame Length: 84 bytes (672 bits)
    Capture Length: 84 bytes (672 bits)
USB URB
    URB id: 0xffff88010c43c380
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_INTERRUPT (0x01)
    Endpoint: 0x83, Direction: IN
        1... .... = Direction: IN (1)
        .000 0011 = Endpoint value: 3
    Device: 2
    URB bus id: 6
    Device setup request: not relevant ('-')
    Data: present (0)
    URB sec: 1334953309
    URB usec: 728740
    URB status: Success (0)
    URB length [bytes]: 20
    Data length [bytes]: 20
    [Request in: 1466]
    [Time from request: 0.992374000 seconds]
    [bInterfaceClass: Unknown (0xffff)]
Leftover Capture Data: 1102091039000c061d474f4f4400000000000000
0000  80 c3 43 0c 01 88 ff ff 43 01 83 02 06 00 2d 00   ..C.....C.....-.
0010  5d c5 91 4f 00 00 00 00 a4 1e 0b 00 00 00 00 00   ]..O............
0020  14 00 00 00 14 00 00 00 00 00 00 00 00 00 00 00   ................
0030  02 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00   ................
0040  11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00   ....9....GOOD...
0050  00 00 00 00                                       ....
</figure> This packets come in regularly (1 per second) on the wire for some time once you sent the "go for the light meter" packet. At one point they are emitted less often and do not contain the value for the light meter anymore, suggesting that the control packet sent earlier triggers the activation of the light meter for a defined period. Now you probably wonder where the data are in this. They're in the 20 bytes leftover in the capture data part, indicated by Wireshark, at the end of the packet: 11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00. Fortunately, it was easy to decode. Knowing we're looking for 2 values (battery charge and light meter), we just need to observe and compare the packet emitted on the wire with the values displayed by the Logitech Solar App. To achieve this, I looked both at the Logitech Solar App and Wireshark while bringing more and more light near the keyboard, increasing the lux value received by the meter on the Solar App, and saw that the fields represented in blue (see below) where changing in Wireshark. Since 2 bytes were changing, I guessed that it was coded on 16 bits, and therefore it was easy to correlate the value with the Solar App. <figure>
[ ....9....GOOD....... ]
11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00
4 bytes - 1 byte for battery charge - 2 bytes for light meter - 2 bytes - 4 bytes for GOOD - 7 bytes
</figure> In this example, the battery has a charge of 0x39 = 57 % and the light meter receives 0x0c = 12 lux of light. It's basically dark, and that makes sense: it was night and the light was off in my office, the only light being the one coming from my screen. I've no idea what the GOOD part of the packet is about, but it's present in every packet and it's actually very handy to recognize such a packet. Therefore I'm considering this as some sort of useful mark for now. For the other bytes, they were always the same (0x11 0x2 0x9 0x10 at the beginning, 7 times 0x00 at the end). The 2 bytes between the light meter and GOOD probably mean something, but I've no idea what for now. Building our solar app Now we've enough information to build our own very basic solar application. We know how to triggers the light meter, and we know how to decode the packets. We're going to write a small application using libusb. Here's a quick example. It's not perfect and does not check for error codes, be careful. <figure>
/* Written by Julien Danjou <julien> in 2012 */
#include <linux/hid.h>
#include <libusb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
 
    libusb_context *ctx;
    libusb_init(&ctx);
    libusb_set_debug(ctx, 3);
    /* Look at the keyboard based on vendor and device id */
    libusb_device_handle *device_handle = libusb_open_device_with_vid_pid(ctx, 0x046d, 0xc52b);
    fprintf(stderr, "Found keyboard 0x%p\n", device_handle);
    libusb_device *device = libusb_get_device(device_handle);
    struct libusb_device_descriptor desc;
    libusb_get_device_descriptor(device, &desc);
    for(uint8_t config_index = 0; config_index < desc.bNumConfigurations; config_index++)
     
        struct libusb_config_descriptor *config;
        libusb_get_config_descriptor(device, config_index, &config);
        /* We know we want interface 2 */
        int iface_index = 2;
        const struct libusb_interface *iface = &config->interface[iface_index];
        for (int altsetting_index = 0; altsetting_index < iface->num_altsetting; altsetting_index++)
         
            const struct libusb_interface_descriptor *iface_desc = &iface->altsetting[al>tsetting_index];
            if (iface_desc->bInterfaceClass == LIBUSB_CLASS_HID)
             
                libusb_detach_kernel_driver(device_handle, iface_index);
                libusb_claim_interface(device_handle, iface_index);
                unsigned char ret[65535];
                unsigned char payload[] = "\x10\x02\x09\x03\x78\x01\x00";
                if(libusb_control_transfer(device_handle,
                                           LIBUSB_REQUEST_TYPE_CLASS   LIBUSB_RECIPIENT_INTERFACE,
                                           HID_REQ_SET_REPORT,
                                           0x0210, iface_index, payload, sizeof(payload) - 1, 10000))
                 
                    int actual_length = 0;
                    while(actual_length != 20   strncmp((const char *) &ret[9], "GOOD", 4))
                        libusb_interrupt_transfer(device_handle,
                                                  iface_desc->endpoint[0].bEndpointAddress,
                                                  ret, sizeof(ret), &actual_length, 100000);
                    uint16_t lux = ret[5] << 8   ret[6];
                    fprintf(stderr, "Charge: %d %%\nLight: %d lux\n", ret[4], lux);
                 
                libusb_release_interface(device_handle, iface_index);
                libusb_attach_kernel_driver(device_handle, iface_index);
             
         
     
    libusb_close(device_handle);
    libusb_exit(ctx);
 
</figure> What the program is doing is the following: This gives the following:
Found keyboard 0x0x24ec8e0
Charge: 64 %
Light: 21 lux
Challenge accomplished! To be continued Unfortunately, this approach has at least one major drawback. We have to disconnect the Logitech Unifying Receiver from the kernel. That means that while we're waiting for the packet, we're dropping packets corresponding to other events from every connected device (key presses, pointer motions ). In order to solve that, I sent a request for help on the linux-input mailing list. That way, I learned that Logitech is actually using the HID++ protocol to communicate with the devices using the Unifying Receiver. Lars-Dominik Braun managed to get the HID++ specifications from Logitech and published them with their authorization. This opens a whole new world. With that document, I may be able to understand the part I reverse engineered and convert this to a more useful and generic library using the hidraw interface (so we don't have to disconnect the devices from the kernel driver).

Benjamin Mako Hill: Wiki Conferencing

I am in Berlin for the Wikipedia Academy, a very cool hybrid free culture community plus refereed academic conference organized, in part, by Wikimedia Deutschland. On Friday, I was very excited to have been invited to give the conference's opening keynote based on my own hybrid take on learning from failures in peer production and incorporating a bunch of my own research. Today, I was on a panel at the conference about free culture and sharing practices. I'll post talks materials and videos when the conference puts them online. I will be in Berlin for the next week or so before I head to directly to Washington, DC for Wikimania between the 11th and 15th. I'll be giving three talks there: Between then and now, I'm taking the next week in Berlin to catch up on work, and with friends. If you're in either place and want to meet up, please get in touch and lets try to arrange something.

7 July 2012

Christian Perrier: DebConf running: take 5 (South-West suburbs of Managua)

The challenge is now more and more to find good places to run without too much cars and noises....and not too far away. Today, I think I made it well..:-) After a look on the map, and some wild guess from what I now know of the way the city is organized, I decided to go south-west of Managa, from the hotel Seminole. This is again a hilly run because, anyway, wherever you go south in Managua, you're going up. Ralf, who arrived just yesterday, joined me and, I guess, enjoyed the run. After going through an obviously quite rich neighbourhood with nice villas, gardeners, huge US truck-style cars, we went close to the Mormonschurch...and a mosque, then headed westward close to "Colegio Americano", with fences, walls, guards in the corner, etc. Seems that any tiny bit of USA in the world needs an army to protect it. Crossing la Pista Suburbana, we then went on a very quiet roads towards the hill, in a very green and peaceful area. House there were really not as fancy as in other places, even seeming quite "poor" in some way. However, we never felt any problem and people we meet on the way are always friendly and smiling : "Hola", "Buenos Dias", etc. At the end of the road, after about half an hour, we decided to head back. However, as going back the same way is boring, I suggested we head up west as my phone's map (it is very helpful to have a smartphone with GPS and Google Maps for wandering through unknwon places) was mentioning another road going north-west a little bit westward. So we entered a small trail between house farms...with dogs! These were a bit "scary" as they were barking at us and of course running entirely freely. On the other hand, none was really aggressive and we had no problem. I however saw a few people really staring at us as I guess there are not so many runners in this place..:-). But still, we never felt unsafe: just in a quite uncommon place. After abotu 500m we found the "road" that was on my map: indeed a path going down slowly along the hill. And there was the marvel. An incredible panoramic view going going the volcanoes that are East and North of Leon : San Cristobal, Telica Rota, Pilas elHoj and last but definitely not least: EL MOMOTOMBO. A nearly perfect pyramid-style volcano that lies about 50km north-west of Managua, on the eastern shore of Managua Lake. The view there is...just fantastic with also a 180 panorama to West and North-West of the lake and the volcanoes area. After this, all we had to do was heading back to the hotel and share this with you...:-) As usual, the full GPS trace is here.

Petter Reinholdtsen: Free Timetabling Software - nice free software

Included in Debian Edu / Skolelinux is a large collection of end user and school specific software. It is one of the packages not installed by default but provided in the Debian archive for schools to install if they want to, is a system to automatically plan the school time table using information about available teachers, classes and rooms, combined with the list of required courses and how many hours each topic should receive. The software is named FET, and it provide a graphical user interface to input the required information, save the result in a fairly simple XML format, and generate time tables for both teachers and students. It is available both for Linux, MacOSX and Windows. This is the feature list, liftet from the project web site: I have not used it myself, as I am not involved in time table planning at a school, but it seem to work fine when I test it. If you need to set up your schools time table, and is tired of doing it manually, check it out. A quick summary on how to use it can be found in a blog post from MarvelSoft. If you find FET useful, please provide a recipe for the Debian Edu project in the Debian Edu HowTo section.

6 July 2012

Christian Perrier: DebConf running: take 4 (Masaya volcano)

Yet another Grand Plan today: run in Masaya Volcan National Park. Masaya Volcano is one of the most active volcanoes in Nicaragua. It is located only 20km south-east of Managua, so going there is as easy as renting a taxi. The original plan was to drive to the entrance of the National park (on the road to Masaya city), then run up to the volcan, so from 250m altitude up to about 600m. I therefore rented a taxi with the kind help of Norman, and we agreed to meet up at 08:30 at the hotel, along with No l and Daniel (who was planning to walk up, not running). After discussing here or there, Kurt and Gaudenz joined, both with the intent of running up. Others would also have liked to join, but I wasn't in the mood of organizing a full bus of Debconfers..:-). My plan was to enjoy pyself the hike...not to manage several taxis, etc. We thus packed in Denis blue taxi...a fancy and very visible customized blue car that looked like it had been the topic of on episode of the "Pimp My Ride" TV show..:-) Sitting 4 people at the back of a regular car without hitting some blue neon lights, or 7" TV screens is kinda tricky, but we made it. When arriving at the park's entrance, we however learned that it is forbidden to walk or hike up the road by foot, because of potential high concentrations of Carbon Dioxide. Sadly, we then had to pack again in the car and go up with it. After reaching the parking close to the volcano's active crater (that is said to have a small lava lake at its bottom, one of the very few in the world), we decided to head up for going round the Nindira crater, an inactive crater located slightly above. Unfortunately, again, the path around the Masaya crater is forbidden to walk on, because of landslides. We indeed still had great fun by running around this crater (which is about a 2.5 kilometers round trip, very very hilly and sometimes hard to run on such as a mountain path. After about 50 minutes and two laps (after all, we had a giant stadium!), we headed back to the parking, packed 5 sweating geeks in the car and went back to the hotel. Full GPS trace is here. I also put a few pictures on Facebook, supposedly visible by anybody : Masaya crater and Nindiri crater.

John Goerzen: How to listen to music?

This seems like a simple question: how do you listen to music? But in a way, I think I ve fallen behind the times. I ve enjoyed listening to digital music for years, since well before MP3 even existed (MIDI files could be comfortably downloaded over dialup; some of you may even remember MOD files). Anyhow, a few years ago, I digitized my entire CD collection, ripping it to FLAC and encoding to high-bitrate MP3 for storage on my 160GB iPod. Since, I ve purchased some music from DRM-free MP3 retailers, and even FLAC files from Magnatune, but at this point I basically say that I have enough music. I have only rarely bought anything new in the last couple of years. I have used Musicbrainz Picard to enforce consistent tagging across almost my entire collection. I am usually in front of a Linux machine. I run mt-daapd on my fileserver, so I can stream music from it to any other machine in the house. However, this has a lot of drawbacks. It is generally impossible to create or modify persistent playlists with this protocol. I can t modify metadata. I can t assign ratings. Though perhaps I don t need to. There are a lot of online services I could think of: Pandora, Spotify, Rdio, Slacker. Plus, of course, the upload your music to the cloud services such as Google and Amazon. Have people tried these? I m hesitant to use a service where I pay a monthly fee, unless I wind up owning the music in a DRM-free fashion. And I m a little nervous about the privacy implications of uploading a vast amount of music to the cloud. There are some items in my collection that I am certain are not available on any of those others, but for the most part I m willing to take what is out there if it meets my tastes. My tastes, by the way, typically run to classical and opera but also can go towards middle eastern or certain new age music (though I m very picky about that, since I don t care for most new age stuff I hear.) But these are huge categories; sometimes I d like to hear some load and noisy classical (say, Beethoven s Ninth conclusion, some Wagner, Verdi arias, etc.) And other times, something more contemplative (a Mozert concerto) or even quiet (a nice sonata) meet my mood. I have playlists for these on my iPod Classic a bit of a problem, since I don t use that device much anymore except in the car, and it takes a LOT of time to categorize things into these playlists. (An opera or a symphony could have parts in various styles and various levels of energy, for instance.) I keep a small manually-managed subset on my Android phone, which works fine. A recommendation engine for things like this could be useful, if it works well. My requirements, basically: I want something that will play the same music on my Linux machines. I want to be able to have playlists kept in sync across my machines. And I don t want a monthly fee. Ideally, the software will work on Android as well, and provide similar features there. Anything that requires Windows is a non-starter. I ve considered NFS mounts, but that s a bit annoying when the laptop disconnects from the network or when it s not at home. What are your thoughts? Ideas?

4 July 2012

Stefano Zacchiroli: bits from the DPL for June 2012

Monthly DPL bits, fresh from the oven^W^W^W hot from DebConf12, and just posted to d-d-a.
Howdy from DebConf12. It's hot, but it's also time to bother you again with a (not so) brief DPL activity report, this time for June 2012. Time-based freeze: DONE, short freeze: TODO Two highlights for this month. First, you've probably noticed Wheezy is now frozen, YAY. This is huge achievement for the release, but also for the project. It's the first time we do a time-based freeze, and it took some quite heated discussion at the beginning of the release cycle to decide to do this. And we did it properly: respecting the planned month and narrowing down the period later. This exercise has hopefully helped both DDs in their package planning and our upstreams in targeting Wheezy with stable releases of their software. Kudos to the release team for their coordination work! Now we've the second part still TODO: releasing Wheezy, without RC bugs, with a freeze period as short as possible. See the beginning of my last "bits from the DPL" mail for my usual song and dance :-P on how to deliver that, together. DebConf12 A lot of us will attend DebConf12. Enjoy it! ... and take the chance to both have fun and make great plans for Debian's future. But remember that "if it didn't happen on a mailing list, it didn't happen". Not all of us will be lucky enough to attend DebConf (in person or remotely). Make sure that those who don't can take part in your team decisions and get informed of what is going to happen here. Politics Zack's spring tour I spent a significant part of June doing Debian talks ins some sort of "spring tour" between Italy and France. In particular: Many thanks to the organizers of these events for inviting and sponsoring me (as well as other Debian people, in the ESRF case) and for their interest in Debian. Sprints Assets Discussions Some relevant discussions for project evolution has been going on in June and I took part into them. You might want to have a look at them: Misc Cheers.
PS the boring day-to-day activity log for June is available at master:/srv/leader/news/bits-from-the-DPL.txt.201206

3 July 2012

C.J. Adams-Collier: Currency platform superior to US Federal Reserve Bank

Should I recommend that Software in the Public Interest implement a monetary stack with the stated purpose of replacing the federal reserve system? We would have to publish the source code, of course, and the transaction db would need to be de-centralized so as to avoid private interests using it for their own benefit at the expense of their fellow humans. Not that that would ever happen, mittens. We d probably experience a period of instability due to scaling issues during the learning state, say the first two to five years. The IRS would need to be convinced that it would be a good idea, though. And I think they only listen to voters. And I don t want to be the one responsible for convincing them. I m not so good at exchanging paperwork with them, and they would likely get confused. Anyway, the idea would be that after the learning state, sessions would be migrated from the Fed db at a sustainable rate. After moving the sessions off of the Fed s current, compromised stack, the increase in tax revenues caused by removal of the current implementation s filters should be more than enough to fund the maintenance of the implementation undergoing strict peer review. http://www.wapublicbankproject.org/

2 July 2012

Julien Danjou: Making the jump: working freelance

For the last 10 years, I've been working on many Free Software projects. From Debian to OpenStack, through awesome, Emacs, XCB and many more. This obviously allowed me to enhance my technical skills, but it also taught me about Free Software and Open Source development processes, and how to work with and close to the community. Working for almost 6 years at Easter-eggs taught me how to work in an autonomous manner, how to lead and manage a project. And how to run a company, thanks to the cooperative status of this great one. These are the reasons why I decided to leave my latest job and run my own company to work as a freelance consultant & developer specialized in Free Software, starting today. Therefore, I am now able and available to provide expertise and development on Free Software, including upstream contribution. Especially on projects I already worked on recently, like OpenStack.

Benjamin Mako Hill: Wiki Conferencing

I am in Berlin for the Wikipedia Academy, a very cool hybrid free culture community plus refereed academic conference organized, in part, by Wikimedia Deutschland. On Friday, I was very excited to have been invited to give the conference's opening keynote based on my own hybrid take on learning from failures in peer production and incorporating a bunch of my own research. Today, I was on a panel at the conference about free culture and sharing practices. I'll post talks materials and videos when the conference puts them online. I will be in Berlin for the next week or so before I head to directly to Washington, DC for Wikimania between the 11th and 15th. I'll be giving three talks there: Between then and now, I'm taking the next week in Berlin to catch up on work, and with friends. If you're in either place and want to meet up, please get in touch and lets try to arrange something.

28 June 2012

DebConf team: DebConf12 schedule ready! (Posted by Gunnar Wolf)

The DebConf12 schedule is finally available! As usual, the DebConf conference can only be finalized a couple of days before DebConf itself starts. But hey Take a look at it, and start planning your activities! Not final, never final Remember that, even now that the conference schedule is announced, you can still submit talks to DebConf, following what we call ad-hoc talks. What are the steps to do it?
  1. Check the schedule and find the timeslot (day, hour and room) you want to talk in
  2. Register your event in Pentabarf, and note the generated event ID (the last number in the URL after the submission is accepted)
  3. Send a mail to schedule@debconf.org detailing the event ID, title (just to confirm it), and the requested timeslot
  4. All requests _must_ be received at least 24hr before the requested event for the scheduling team to act accordingly and for the video team to be able to plan on resources to cover the requested session
  5. Video team coverage (event recording plus streaming on a best-effort basis) will be given to activities scheduled on the Aula Magna and Roberto Ter n auditoriums.
  6. We will try to grant requests on a first-come-first-serve basis. If two events are requested for the same timeslot, the first will have precedence, and we will schedule the second as close as possible to it. We will notify the submitters if this happens, of course.
Conference tracks In DebConf12, many talks are organized around the following tracks:
DebianDay
This track is meant for the local (Nicaraguan/Central American) community to be attracted to Debian. It does not usually have technically minded talks; the DebConf general audience is, of course, invited to take part in it, as there will surely be interesting angles on how Debian is used in different use-cases, and there will be interesting Q&A session with the audience.
Getting involved in Debian
Central America is a very special region. It has vibrant Free Software user groups and communities, among which there is an important interest to get involved in Debian.
This track will have some talks in Spanish, some talks in English, explaining different Debian processes and tools, aiming at attracting locals to contribute to Debian.
Debian for the cloud
All about virtualization. Management infrastructure, details on virtual machine engines, insight on how to tweak your kernel for optimum virtualized performance, and of course how to get it all humming perfectly with your Debian packages.
Building and porting
With an every day stronger transit to mobile computing, we are seeing a renaissance of porting. At the same time, we finally have multiarch as part of Debian. And were that not to be enough, we finally have more than one compiler that can be used for most of the software in the archive. Put all this together, and you will have a very interesting track.
Social activities
DebConf is as much about technical stuff as it is about building closer ties within the members of its community. Eating cheese+wine, going on a trip to the guts of a volcano, having a nice banquet Do you want to add an activity to this pseudo-track?
Skills exchange
Knowledge can also be built by all of us putting our grain of sand. We first had explicit skills exchange sessions in Bosnia last year, and they were very well received. Feel free to propose similar sessions!
And Well, three days to DebCamp and counting. See you Real Soon Now in Managua!

Sorina Sandu: Status update

Hello!

First of all: I'm going to DebConf :D and I really want to say a big "Thank you!" to every one who made this possible.


So, here's what I've done recently:

I spent some time putting everything in order for the departure to Nicaragua, things like finding a plane ticket through a route that does not require a Visa, making a new Passport, now all is in order.

I've integrated the changes for listing available ESSIDs, in order to close #637128 [0] into a separate branch , which can be found here [1].

The plan is to have only one new template which lists the available networks and which has a option for entering the ESSID manualy. In order to do so, a simple scan is performed on the wireless interface, and the configuration for a particular network resulted after the scan is kept if that one is chosen. The enter manually situation is treated exactly as it was before.

Furthermore, last week (or the week before) I've done some coding style fixes again in a separate branch [2], nothing very fancy, just trimming trailing white spaces and changing some tabs to spaces, where spaces were mostly used. I've also noticed some inconsistencies, regarding white spaces between functions names and the parenthesis. However I think I'm overreacting a little, those matters are not really that relevant, but when I'll have some free time I'll get to this as well.


Next in line: Issues

Testing is still giving me some troubles, the show essids part is not yet fully tested, so I couldn't say for sure if everything works as planned.

Apart from the classical: write the image to an USB and reboot method, I'm using Virtual Box for testing. I've managed to create a passthrough for the wireless interface, but now I have troubles with some firmware. Hopefully I'll solve this as soon as possible since it would really make my life easier.


Future plans:

I've realized that I've underestimated a little what the project implies and the amount of information that is new to me. I feel that I have learned a lot and that there is still a lot to learn and I'm very happy that it is so. I've now started documenting regarding IPv6, beginning with this [3]. We haven't decided yet what the next task will be exactly.


[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637128
[1] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/show_essids
[2] http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/sorina/coding_style
[3] http://tldp.org/HOWTO/Linux+IPv6-HOWTO/

26 June 2012

Christian Perrier: 2012 update 28 for Debian Installer localization

D-I beta 1 is in preparation. Commits to level 1 are still allowed but no guarantee they'll make it to beta1 and therefore wheezy. Status for D-I level 1 (core D-I files): Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.): Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader) Full 100% completeness (hall of fame) for 32 languages: Asturian, Belarusian, Bulgarian, Catalan, Czech, Welsh, Danish, German, Esperanto, Spanish, Persian, French, Galician, Hebrew, Indonesian, Italian, Japanese, Kazakh, Khmer, Latvian, Norwegian Bokm l, Dutch, Polish, Portuguese, Russian, Slovak, Slovenian, Swedish, Thai, Turkish, Simplified Chinese, Traditional Chinese

24 June 2012

Joey Hess: trying obnam

Obnam 1.0 was released during several months when I had no well-connected server to use for backups. Yesterday I installed a terabyte disk in a basement with a fiber optic network connection, so my backupless time is over. Now, granted, I have a very multi-layered approach to backups; all my data is stored in git, most of it with dozens of copies automatically maintained, and with archival data managed by git-annex. But I still like to have a "real" backup system underneath, to catch anything else. And to back up those parts of my user's data that I have not given them tools to put into git yet... My backup server is not my basement, so I need to securely encrypt the backups stored there. Encrypting your offsite backups is such a good idea that I've always been surprised at the paucity of tools to do it. I got by with duplicity for years, but it's increasingly creaky, and the few times I've needed to restore, it's been a terrific pain. So I'm excited to be trying Obnam today. So far I quite like it. The only real problem is that it can be slow, when there's a transatlantic link between the client and the server. Each file backed up requires several TCP round-trips, and the latency kills the bandwidth. Large files are still sent fast, and obnam uses little resources on either the client or server while running. And this mostly only affects the initial, full backup. But the encryption and ease of use more than make up for this. The real killer feature with Obnam's encryption isn't that it's industry-standard encryption with gpg, that can be trivially enabled with a single option (--encrypt-with=DEADBEEF). No, the great thing about it is its key management. I generate a new gpg key for each system I back up. This prevents systems reading each other's backups. But that means you have to backup the backup keys.. or when a system is lost, the backup would be inaccessible. With Obnam, I can instead just grant my personal gpg key access to the repository: obnam add-key --keyid 2512E3C7. Now both the machine's key and my gpg key can access the data. Great system; can't revoke access, but otherwise perfect. I liked this so much I stole the design and used it in git-annex too. :) I'm also pleased I can lock down .ssh/authorized_keys on my backup server, to prevent clients running arbitrary commands. Duplicity runs ad-hoc commands over ssh, which defeated me from ever locking it down. Obnam can be easily locked down, like this: command="/usr/lib/openssh/sftp-server" This could still be improved, since clients can still read the whole filesystem with sftp. I'd like to have something like git-annex's git-annex-shell, which can limit access to only a specific repository. Hmm, if Obnam had its own server-side program like this, it could stream backup data to it using a protocol that avoids the roundtrips needed by the SFTP protocol, and fix the latency issue too. Lars, I know you've been looking for a Haskell starter project ... perhaps this is it? :)

23 June 2012

Bernd Zeimetz: Report from the Bug Squashing Party in Salzburg

bsp_2012_salzburg photo from salzburg-cityguide.com, Copyright (C) Uwe Brandl

Participation and Results From June 15-17th we held a Debian BugSquashingParty in Salzburg, hosted and sponsored by conova communications GmbH. It was a fun and busy weekend, with 15-17 people from 5 countries being around, mainly working on RC bugs in Testing/Unstable. Gerfried Fuchs (rhonda) also worked on triaging the impact of RC bugs on the version in Squeeze, while Peter Palfrader (weasel) took care of Tor related things and Debian sysadmin work, including starting on the new bugs and udd hosts. Phillip Hug (hug) worked on the debian.ch infrastructure. Together with Miroslav Such from Red Hat Bernd Zeimetz (bzed) worked on the packaging of the necessary libraries and daemons to add (basic) Spacewalk client support to Debian. As soon as the packages passed NEW and #677871 was applied (thanks to the APT guys for working on that already), managing Debian clients with Spacewalk should work out of the box. Of course we also had a little keysigning party :)

Statistics
  • about 68 bugs in unstable/testing were triaged/patched/fixed or at least pinged
  • 54 bugs were tagged to show if they affect Squeeze, several other bugs were pinged to retrieve necessary information or to trigger an update in the next stable pointrelease.
  • 5 packages were introduced into Debian (still in NEW, though) - the Spacewalk client related packages and libapache2-mod-auth-memcookie.

Accomodation Thanks to Debian funds we were able to provide accomodation for four participants in the JUFA youth hostel in Salzburg. We had paid in advance for eight, but changing to rooms with a higher category for only 4 people would have been equally or more expensive.

Press/Media coverage Additionally to being mentioned in the calendars on ProLinux and similar pages, we had some press coverage by the local newspaper and online magazines:

Fun facts We consumed 2kg of Leberkas, a big plate of "Buchteln mit Vanillesosse", about 16000cm^2 of Pizza, about 80 litres of coke, juice, beer and wine and I guess we drank at least the same amount of water. We had coffee made of 1.5kg coffee beans and managed to empty the (formerly well filled) icemaker in the fridge. Also we had successful training sessions of a standard Debconf game (rules won't be explained here obviously). Maybe we even successfully spread the game to the employees of a commercial linux distribution ;)

22 June 2012

Christian Perrier: 2012 update 27 for Debian Installer localization

The D-I beta release is in preparation so we have many last minute updates and I'm uploading gazillion of packages. Each time, I upload one....a translator pops up and sends another update, etc. :-) Status for D-I level 1 (core D-I files): Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.): Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader) Full 100% completeness (hall of fame) for 28 languages: Asturian, Belarusian, Bulgarian, Catalan, Czech, Welsh, Danish, German, Esperanto, Spanish, Persian, French, Hebrew, Indonesian, Italian, Japanese, Kazakh, Khmer, Latvian, Dutch, Polish, Portuguese, Russian, Slovak, Slovenian, Swedish, Turkish, Simplified Chinese

17 June 2012

Johannes Schauer: port bootstrap build-ordering tool report 2

A copy of this post is sent to soc-coordination@lists.alioth.debian.org as well as to debian-bootstrap@lists.mister-muffin.de. Diary June 4 I added the first version of basenocycles.ml to git. Given an initial set of cross built packages, it tries to compile as much as possible on the resulting system in multiple rounds. June 5 During June 3, I discovered and error in my program that would only come up when using the Debian Sid package lists as the input:
Fatal error: exception Assert_failure("common/edosSolver.ml", 610, 11)
On this day, June 5, I wrote a minimal test case for this problem. The same day, Pietro figured out that this is a bug in dose which will be fixed in the next release. Begin writing code to figure out how important a binary package is for the further build process. Try to use Depsolver.edos_install to find out what packages are needed to make debhelper available. Restructure basenocycles.ml, exclude source packages that already have been built, still trouble with already existing binary packages and Cudf.mem_installed, comment stuff better. June 6 I wrote some crude code (only estimate, not correct, fixed later) that would give a rough overview of how often a given binary package is directly required as a build dependency. Debhelper came out as the most needed package. It is architecture:all, so it does not have to be built but it has unfulfilled runtime dependencies. To make those packages available, 13 (actually 11, fixed later) packages have to be compiled on Ubuntu Natty. But those packages all (except gettext) require debhelper itself to be built. The first dependency cycle. This dependency cycle (actually, the 12 cycles) can be broken by either cross compiling those source packages or by making them build without debhelper. One goal of the program is to help decide what the easier option is, but this is not yet implemented. To play around a bit, I created the possibility to specify a list of packages that are additionally to the minimal set of cross compiled packages also cross compiled. I added the 13 packages found above to the list, thus making the binary packages they build available. This made debhelper available in the system. As a result, 1625 out of 3339 source packages can be built with just a minimal build system (priority:essential packages plus build-essential) and debhelper available. The next package that blocks the most other source packages from being built is cdbs. The next nine packages in that list also require cdbs so it seems to be the next important package to be available. Pietro's suggestions make me:
 - do not open BootstrapCommon but ExtLib, Common, Algo, Debian
 - do proper option parsing and logging
 - use Debcudf.ignore_essential = true
 - do Debcudf.init_tables (binlist@srclist)
 - use @ with shorter list first
 - use more List.rev_append instead of @
 - use CudfAdd.who_provides to find if a package is available
June 7 Pietro and Patrick suggest that for solving the debhelper cycles, one could provide a minimal debhelper version so that the above list of 12 packages can be built without debhelper. I try to figure out how to get a list of packages that are missing to make a package installable/buildable. This functionality should be provided in dose but I fail to find it. June 8 Lacking a solution of the problem of June 7, I write a mail to Pietro. I start my first graphs in ocaml using the ocamlgraph library. The graph I generate, starts off at a binary package. For each binary package it connects new vertices as its runtime dependencies. If a binary package is not arch:all and also not yet otherwise compiled, its source package is also added. The result is a graph in which set of source packages in it will make the initial package available, if those source packages would be cross compiled. The graph is extended further than the source packages. June 9 I refine a couple of functions, make univ_get_pkg_by_name return the package with the highest version number. I wrote a rather lengthy (1027 words) email to the list that explains my status as of this day. I can create graphviz dot files with ocaml, can create node and edge types and create the graph by an imperative pattern that I saw a lot in Pietro's code. Disjunctions are not yet handled correctly (see mail from June 8). The graphs generated look like the following: http://mister-muffin.de/p/8nyc.png June 11 I write a test case which shows how CudfAdd.who_provides doesnt find virtual packages. Automate the process of finding the packages that, if cross compiled, would make another package available. Add more predicates (identifying source packages) and improve input file reading code. Move build_compile_rounds which compiles as many source packages as it can in multiple rounds on a given minimal system a toplevel function and thereby abstract it more. Create a rudimentary text based menu to choose different actions to take for an analysis. Start writing an extended version of simple_dependency_graph for deeper analysis. Use xdot to show graphs from the text menu. Allow saving those graphs to a file. June 12 Move functionality from the extended version of simple_dependency_graph over to the normal version and delete the extended version. Add the new Closure vertex type. Create extended_dependency_graph which is supposed to not contain single binary package vertices but handle a package and its installation set as one vertex. The output of extended_dependency_graph is optionally reduced to the biggest (non degenerate) strongly connected component. User gets the option of choosing the exploration depth. June 13 Pietro replies to my mail from June 8 but apparently I failed to express myself well enough in my last mail, so I rephrase my question. Pietro replies to my email from June 11 and explains how the effect I see is due to "a nuisance of the debian to cudf encoding". As a result I change my code accordingly. June 14 Another lengthy (1130 words) email to the list. I explain what was done in the past days, what parts work and how they work. I list some rationales on why I did things the way I did them. The most important observation is, that after improving my code again and again, I ended up representing the dependency cycle problem in the same (very similar) way that Pietro suggested in the beginning. This is probably a good discovery. Lots of data of that email is now of only little use as of June 16, I make lots of improvements in correctness. As I dont have an answer to my other email to Pietro from June 13, I implement a very crude way to get an answer to the question of what packages are missing for a package to be available/compileable. I called it flatten_vpkgformula_best_effort and it suffers from many faults including disjunctions and package conflicts. Patrick spots a problem. As a result, I make sure that at no point, the source package of an arch:all package can be listed. June 15 As a reply to my mail from June 13, Pietro creates a new branch in the git and adds the code I needed to get a proper installation set. June 16 As a result of Pietro's efforts from June 15, I make great advancements on all fronts. Details of the current status follow in the next section. Results A big leap was made on June 16 due to Pietro's great help on making me understand how Depsolver.listcheck can be used for my purposes. My difficulties in finding the solution myself are rooted in many parts of the dose framework being poorly commented but Pietro did already a couple of documentation commits whenever things were unclear for me. Using Depsolver.listcheck makes it possible to be more distribution agnostic and I dont have to handle vpkgs, virtual packages and constraints myself anymore. The code also doesnt suffer anymore by wrongly analyzed dependencies and conflicts. The only thing that is not yet taken care of, is that Depsolver.listcheck only chooses one out of several possible installation set. A final version should be able to take into account that a different installation set could provide a better solution. Overall, in comparison to two weeks ago, I can now properly build, traverse and analyze graphs, can choose an installation set properly, understand more about dependencies, closures, dose and ocaml in general. Finding the importance of binary packages for building When calculating how many source packages are depending on the availability of a binary package I originally flattened the pkg.Cudf.depends list twice for a rough overview. This is of course wrong due to disjunctions and conflicts and also doesnt provide a deep dependency inspection. The new method is to calculate an installation set that is necessary to compile a source package for every source package. The resulting list of binary packages is then used to find out how often a binary package appears in an installation set. I see three drawbacks though: Removing simple graph The simple graph which contained single binary and source packages was removed. I realized it doesnt really serve any purpose to look at it. As a result, Bin vertices and InstallDep edges are also not part of the graph anymore. Since it was responsible for generating the list of source packages that have to be cross built to make a package available, I created a new function get_cross_source_packages which uses an installation to serve the same purpose. Fix extended_dependency_graph extended_dependency_graph now uses installation sets for generating the list of packages that is needed to compile a source package or install a binary package. The list of build dependencies does not include packages that are already installable. The list of runtime dependencies does not include packages that are otherwise available (cross built, arch:all...). Instead of checking for list membership all the time, I created hash tables for the list of installable as well as for the list of available binary packages. Future There are two big tasks for the next two weeks: Task one is to find a way to give hints on which packages to best consider for having reduced build dependencies. This would then probably finally make use of Pietro's cycle algorithms. Task two is to find a way to break cycles and create a build-DAG from a list of packages that already have staged build dependency information. Patrick is currently working on patching dpkg with Build-Depends-StageN dependencies as making perl cross compilable. If he doesnt need the ability to decide which packages to modify to have staged build dependencies in the near future, then task one is probably less urgent and therefor of less importance right now? On the other hand, I can easily generate fake reduced build dependencies so that doing task two right now would not be a problem. Also, having the solution for task two will make it possible to show the user what effect it would have to add reduced build dependencies to a package. For the reasons above (it's not urgent, task one profits from task two being solved) I will go and implement task two first (if there are no objections from my mentors). Another idea, that I discussed with wookey and Patrick yesterday, was that due to multiarch being used for more and more packages, there should exist a set of packages that is cross compilable without any change to the package. We agreed that I make a list of packages that, if cross compiled, would break dependency cycles and make other packages available. I created such a list of about 160 packages for Ubuntu Natty that, if cross compiled, made it possible to have 87% of Natty available (those numbers have to be treated with caution as I did not yet use the proper way of installation sets when generating that list, but the order of magnitude should be correct). Wookey can then try to cross compile those packages. If some packages of those "crucial" source packages are found to be cross compilable, then they should be cross compiled because it means that no work has to be done to break some cycles. Cross compiling all packages that are cross compilable out of the box is no solution, as only natively compiled packages can go into the archive. This is why the list of potentially additionally cross compiled source packages has to be kept as small as possible.

16 June 2012

Jon Dowland: Debian Day #3

Once again Debian day was somewhat compromised: this time by more DIY. I still managed a few bits and pieces, amortized over the rest of the week. I benefited from some train time this weekend. A new version of flactag and catching up the flactag packaging vcs repository. A maintainer endorsement for one of the flactag maintainers and a request to NM for another maintainer to be granted access to the collab-maint repository. More work on the musicbrainz 'transition': First, usertag tracking of packages to update, with upstream bugs where necessary. Tested and confirmed Michael Biebl's sushi patch; found and fixed a sushi bug along the way; wrote patches for sound-juicer ( merged upstream) and goobox.

Christian Perrier: 2012 update 26 for Debian Installer localization

Status for D-I level 1 (core D-I files): Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.): Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader) Full 100% completeness (hall of fame) for 27 languages: Asturian, Belarusian, Bulgarian, Catalan, Czech, Danish, German, Esperanto, Spanish, Persian, French, Hebrew, Indonesian, Italian, Japanese, Kazakh, Khmer, Latvian, Dutch, Polish, Portuguese, Russian, Slovak, Slovenian, Swedish, Turkish, Simplified Chinese

Next.

Previous.