Dog, long time since I blogged about my running activities. Apparently, I didn't since.....I posted a summary for 2013. So, well, that will be a long update as many things happened during the first half of 2014 when it comes at running, for me. January: I was recovering from a fatigue fracture injury inherited from last races in 2013. As a consequence, I resumed running only on Jan 7th. Therefore I cancelled my participation to the "Semi Raid 28", an night orienteering raid of about 50-60km in southern neighbourhood of Paris. Instead, I actually offerred my help to organizers in collecting orienteering signs after the race (the longest one : 120km). So, I ended up spending over 24 hours running in woods and hunting down hidden signs with the same information than runners. My only advantage was that I was able to use my car to go from one point to another. Still, I ended up running over 70km in many small parts, often alone in the dark woods with my headlamp, on very muddy areas...and collecting nearly 80 huge signs. February: Everything was going well and I for instance ran a great half-marathon in Bullion (south of Paris) in 1h3821" (great for a quite hilly race)....until I twisted my left ankle while running back from work. A quite severe twist, though no bone damage, thankfully. I had to stop running, again, for 3 years. Biking to/from work was the replacement activity.... March: I resumed running on March 10th, one week before a quite difficult trail race in my neighbourhood (30km "only" but up to 800 meters positive climb). That race was a preparation (and a test after the injury) for my 3rd participation to "Paris Ecotrail", a 80km trail race in woods of the South-West area of Paris, ending in the Eiffel Tower area. Indeed, both went very well, though I was very careful with my ankle. I finally broke my record at Ecotrail, finishing the race in 9h08 (to be compared to 9h36 last year and 11h15 the year before). April: Paris marathon was scheduled one week after Ecotrail. Everybody will tell you that running a marathon one week after a 80km race is kinda crazy.....which is why I made it..:-). That was my 3rd Paris marathon and my 12th marathon overall. However, this year, no record in sight. The challenge was running the marathon....dressed as SpongeBob (you know me, right?). I actually had great fun doing that and was happy to get zillions of cheering all over the race, from the crowd. I finally completed the race in 4h30, which is, after all, not that far from the time of my very first marathon (4h12). The only drawback was that the succession of quite very long distance runs made my left knee suffer as it never happened before. As a consequence, I (again) had to stop running for nearly one month before we found that I was quite sensitive to pronation, which the succession of long and slow races made worse. May: so finally afterthese (very) long weeks, I could gradually resume running, which finally culminated in mid-May with the 50km race "trail des Cerfs", in the Rambouillet Forest, closed to our place. This quite long but not too difficult trail race ("only" 800 meters positive climb overall) was completed in 5h16, which was completely unexpected, given the low training during the previous weeks. June: no race during that month. The entire month was focused on preparing the Montagn'hard race of July 5th: so several training sessions with a lot of climbing either by running or by fast walking (nordic style) as well as downhill run training (always important for moutain trail). July: the second "big peak" of my 2014 season was scheduled for July 5th: "La Montagn'hard", a moutain trail race close to Les Contamines in the neighbourhood of Chamonix, the french moutaineering Mekkah. "Only" 60 kilometers....but close to 5000 meters positive climb. Montagn'hard is among the thoughest moutain trail races in France and therefore a "must do" for trail runners. This race week-end includes also a 105km ultra-race, which is often said to be as hard, even maybe harder, than the very famous "Ultra Trail du Mont-Blanc" trail in Chamonix. Still, for my second only season in moutain trail running, I decided to be "wise" and stick with the "medium" version (after all, my experience, as of now with moutain trails were only two quite "short" ones). Needless to say, it has indeed been a GREAT race. The environment is wonderful ("Miage" side of the Mont-Blanc range), the race goes through great place (Col de Tricot, noticeably) and I made a great result by finishing80th out of 325+ runners, in 12h18, while my target time was around 13 hours. This is where I am now. Nearly one month after Montagn'hard, I'm deeply training for my next Big Goal: The "Sur la Trace des Ducs de Savoie" or "TDS", one of the 4 races of the Ultra Trail du Mont-Blanc week, in end August (during DebConf): 120km, nearly 7500m positive climp, between Courmayeur and Chamonix, through several passes, up to 2600m height. Yet another challenge: my first "over 24h" race, with a full night out in the moutains. You'll certainly hear again from me about that...:-)

Dog, long time since I blogged about my running activities. Apparently, I didn't since...... So, well, that will be a long update as many things happened during the first half of 2014 when it comes at running, for me. January: I was recovering from a fatigue fracture injury inherited from last races in 2013. As a consequence, I resumed running only on Jan 7th. Therefore I cancelled my participation to the "Semi Raid 28", an night orienteering raid of about 50-60km in southern neighbourhood of Paris. Instead, I actually offerred my help to organizers in collecting orienteering signs after the race (the longest one : 120km). So, I ended up spending over 24 hours running in woods and hunting down hidden signs with the same information than runners. My only advantage was that I was able to use my car to go from one point to another. Still, I ended up running over 70km in many small parts, often alone in the dark woods with my headlamp, on very muddy areas...and collecting nearly 80 huge signs. February: Everything was going well and I for instance ran a great half-marathon in Bullion (south of Paris) in 1h3821" (great for a quite hilly race)....until I twisted my left ankle while running back from work. A quite severe twist, though no bone damage, thankfully. I had to stop running, again, for 3 years. Biking to/from work was the replacement activity.... March: I resumed running on March 10th, one week before a quite difficult trail race in my neighbourhood (30km "only" but up to 800 meters positive climb). That race was a preparation (and a test after the injury) for my 3rd participation to "Paris Ecotrail", a 80km trail race in woods of the South-West area of Paris, ending in the Eiffel Tower area. Indeed, both went very well, though I was very careful with my ankle. I finally broke my record at Ecotrail, finishing the race in 9h08 (to be compared to 9h36 last year and 11h15 the year before). April: Paris marathon was scheduled one week after Ecotrail. Everybody will tell you that running a marathon one week after a 80km race is kinda crazy.....which is why I made it..:-). That was my 3rd Paris marathon and my 12th marathon overall. However, this year, no record in sight. The challenge was running the marathon....dressed as SpongeBob (you know me, right?). I actually had great fun doing that and was happy to get zillions of cheering all over the race, from the crowd. I finally completed the race in 4h30, which is, after all, not that far from the time of my very first marathon (4h12). The only drawback was that the succession of quite very long distance runs made my left knee suffer as it never happened before. As a consequence, I (again) had to stop running for nearly one month before we found that I was quite sensitive to pronation, which the succession of long and slow races made worse. May: so finally afterthese (very) long weeks, I could gradually resume running, which finally culminated in mid-May with the 50km race "trail des Cerfs", in the Rambouillet Forest, closed to our place. This quite long but not too difficult trail race ("only" 800 meters positive climb overall) was completed in 5h16, which was completely unexpected, given the low training during the previous weeks. June: no race during that month. The entire month was focused on preparing the Montagn'hard race of July 5th: so several training sessions with a lot of climbing either by running or by fast walking (nordic style) as well as downhill run training (always important for moutain trail). July: the second "big peak" of my 2014 season was scheduled for July 5th: "La Montagn'hard", a moutain trail race close to Les Contamines in the neighbourhood of Chamonix, the french moutaineering Mekkah. "Only" 60 kilometers....but close to 5000 meters positive climb. Montagn'hard is among the thoughest moutain trail races in France and therefore a "must do" for trail runners. This race week-end includes also a 105km ultra-race, which is often said to be as hard, even maybe harder, than the very famous "Ultra Trail du Mont-Blanc" trail in Chamonix. Still, for my second only season in moutain trail running, I decided to be "wise" and stick with the "medium" version (after all, my experience, as of now with moutain trails were only two quite "short" ones). Needless to say, it has indeed been a GREAT race. The environment is wonderful ("Miage" side of the Mont-Blanc range), the race goes through great place (Col de Tricot, noticeably) and I made a great result by finishing80th out of 3250+ runners, in 12h18, while my target time was around 13 hours. This is where I am now. Nearly one month after Montagn'hard, I'm deeply training for my next Big Goal: The "Sur la Trace des Ducs de Savoie" or "TDS", one of the 4 races of the Ultra Trail du Mont-Blanc week, in end August (during DebConf): 120km, nearly 7500m positive climp, between Courmayeur and Chamonix, through several passes, up to 2600m height. Yet another challenge: my first "over 24h" race, with a full night out in the moutains. You'll certainly hear again from me about that...:-)

Major desktop environments such as Xfce or KDE have a built-in computer suspend feature, but when you use a lighter alternative, things are a bit more complicated, because basically: only root can suspend the computer. There used to be a standard solution to that, using a D-Bus call to a running daemon upowerd. With recent updates, that solution first stopped working for obscure reasons, but it could still be configured back to be usable. With newer updates, it stopped working again, but this time it seems it is gone for good:

$ dbus-send --system --print-reply \
            --dest='org.freedesktop.UPower' \
            /org/freedesktop/UPower org.freedesktop.UPower.Suspend
Error org.freedesktop.DBus.Error.UnknownMethod: Method "Suspend" with
signature "" on interface "org.freedesktop.UPower" doesn't exist

The reason seems to be that upowerd is not running, because it no longer provides an init script, only a systemd service. So, if you do not use systemd, you are left with one simple and stable solution: defining a sudo rule to start the suspend or hibernation process as root. In /etc/sudoers.d/power:

%powerdev ALL=NOPASSWD: /usr/sbin/pm-suspend, \
                        /usr/sbin/pm-suspend-hybrid, \
                        /usr/sbin/pm-hibernate

That allows members of the powderdev group to run sudo pm-suspend, sudo pm-suspend-hybrid and sudo pm-hibernate, which can be used with a key binding manager such as your window manager's or xbindkeys. Simple, efficient, and contrary to all that ever-changing GizmoKit and whatsitd stuff, it has worked and will keep working for years.

You've installed apt-transport-tor to help prevent targeted attacks on your system. Great! Now you want to build Debian packages using cowbuilder, and you notice these are still using plain HTTP. If you're willing to fetch the first few packages without using apt-transport-tor, this is as easy as:

• Add 'EXTRAPACKAGES="apt-transport-tor"' to your pbuilderrc.
• Run 'cowbuilder --update'
• Set 'MIRRORSITE=tor+http://http.debian.net/debian' in pbuilderrc.
• Run 'cowbuilder --update' again.

Now any future builds should fetch build-dependencies over Tor. Unfortunately, creating a base.cow from scratch is more problematic. Neither 'debootstrap' nor 'cdebootstrap' actually rely on apt acquire methods to download files - they look at the URL scheme themselves to work out where to fetch from. I think it's a design point that they shouldn't need apt, anyway, so that you can debootstrap on non-Debian systems. I don't have a good solution beyond using some other means to route these requests over Tor.

Recently I was updating my dotfiles, because I wanted to ensure that media-players were "always on top", when launched, as this suits the way I work. For many years I've used devilspie to script the placement of new windows, and once I googled a recipe I managed to achieve my aim. However during the course of my googling I discovered that devilspie is unmaintained, and has been replaced by something using Lua - something I like. I'm surprised I hadn't realized that the project was dead, although I've always hated the configuration syntax it is something that I've used on a constant basis since I found it. Unfortunately the replacement, despite using Lua, and despite being functional just didn't seem to gell with me. So I figured "How hard could it be?". In the past I've written softare which iterated over all (visible) windows, and obviously I'm no stranger to writing Lua bindings. However I did run into a snag. My initial implementation did two things:

• Find all windows.
• For each window invoke a lua script-file.

This worked. This worked well. This worked too well. The problem I ran into was that if I wrote something like "Move window 'emacs' to desktop 2" that action would be applied, over and over again. So if I launched emacs, and then manually moved the window to desktop3 it would jump back! In short I needed to add a "stop()" function, which would cause further actions against a given window to cease. (By keeping a linked list of windows-to-ignore, and avoiding processing them.) The code did work, but it felt wrong to have an ever-growing linked-list of processed windows. So I figured I'd look at the alternative - the original devilspie used libwnck to operate. That library allows you to nominate a callback to be executed every time a new window is created. If you apply your magic only on a window-create event - well you don't need to bother caching prior-windows. So in conclusion :

• devilspie is dead.
• there is a devils2pie2 project.
• I wrote kpie

I think my code is better than devilspie2 because it is smaller, simpler, and does things more neatly - for example instead of a function to get geometry and another to set it, I use one. (e.g. "xy()" returns the position of a window, but xy(3,3) sets it.). kpie also allows you to run as a one-off job, and using the simple primitives I wrote a file to dump your windows, and their size/placement, which looks like this:

shelob ~/git/kpie $ ./kpie --single ./samples/dump.lua
-- Screen width : 1920
-- Screen height: 1080
..
if ( ( window_title() == "Buddy List" ) and
     ( window_class() == "Pidgin" ) and
     ( window_application() == "Pidgin" ) ) then
     xy(1536,24 )
     size(384,1032 )
     workspace(2)
end
if ( ( window_title() == "feeds" ) and
     ( window_class() == "Pidgin" ) and
     ( window_application() == "Pidgin" ) ) then
     xy(1,24 )
     size(1536,1032 )
     workspace(2)
end
..

As you can see that has dumped all my windows, along with their current state. This allows a simple starting-point - Configure your windows the way you want them, then dump them to a script file. Re-run that script file and your windows will be set back the way they were! (Obviously there might be tweaks required.) I used that starting-point to define a simple recipe for configuring pidgin, which is more flexible than what I ever had with pidgin, and suits my tastes. Bug-reports welcome.

Today we finished the talk selection process. We are very grateful to everyone who decided to submit talks and events for DebConf14. If you have submitted an event, please check your email :). If you have not received any confirmation regarding your talk status, please contact us on talks@debconf.org During the selection process, we bore in mind the number of talk slots during the conference, as well as maintaining a balance among the different submitted topics. We are pleased to announce that we have received a total of 115 events, of which 80 have been approved (69%). Approval means your event will be scheduled during the conference and you will have video coverage. The list of approved talks can be found on the following link: https://summit.debconf.org/debconf14/all/ If you got an email telling your talk have being approved, and your talk is not listed, don t panic. Check the status on summit, and make sure to select a track, if you have some track suggestions please mail us and tell us about it. This year, we expect to also have a sort of unconference schedule. This will take place during the designated hacking time . During that time the talks rooms will be empty, and ad hoc meetings can be scheduled on-site while we are in the Conference. The method for booking a room for your ad hoc meeting will be decided and announced later, but is expected to be flexible (i.e: open scheduling board / 1 day or less in advance booking), Please don t abuse the system: bear in mind the space will be limited, and only book your event if you gather enough people to work on your idea. Please make sure to read the email regarding your talk. :) and prepare yourself. Time is ticking and we will be happy to meet you in Portland.

Reworking Intel Glamor The original Intel driver Glamor support was based on the notion that it would be better to have the Intel driver capture any fall backs and try to make them faster than Glamor could do internally. Now that Glamor has reasonably complete acceleration, and its fall backs aren t terrible, this isn t as useful as it once was, and because this uses Glamor in a weird way, we re making the Glamor code harder to maintain. Fixing the Intel driver to not use Glamor in this way took a bit of effort; the UXA support is all tied into the overall operation of the driver. Separating out UXA functions The first task was to just identify which functions were UXA-specific by adding _uxa to their names. A couple dozen sed runs and now a bunch of the driver is looking better. Next, a pile of UXA-specific functions were actually inside the non-UXA parts of the code. Those got moved out, and a new intel_uxa.h file was created to hold all of the definitions. Finally, a few non UXA-specific functions were actually in the uxa files; those got moved over to the generic code. Removing the Glamor paths in UXA Each one of the UXA functions had a little piece of code at the top like:

if (uxa_screen->info->flags & UXA_USE_GLAMOR)  
    int ok = 0;
    if (uxa_prepare_access(pDrawable, UXA_GLAMOR_ACCESS_RW))  
        ok = glamor_fill_spans_nf(pDrawable,
                      pGC, n, ppt, pwidth, fSorted);
        uxa_finish_access(pDrawable, UXA_GLAMOR_ACCESS_RW);
     
    if (!ok)
        goto fallback;
    return;
 

Pulling those out shrank the UXA code by quite a bit. Selecting Acceleration (or not) The intel driver only supported UXA before; Glamor was really just a slightly different mode for UXA. I switched the driver from using a bit in the UXA flags to having an accel variable which could be one of three options:

• ACCEL_GLAMOR.
• ACCEL_UXA.
• ACCEL_NONE

I added ACCEL_NONE to give us a dumb frame buffer mode. That actually supports DRI3 so that we can bring up Mesa and run it under X before we have any acceleration code ready; avoiding a dependency loop when doing new hardware. All that it requires is a kernel that offers mode setting and buffer allocation. Initializing Glamor With UXA no longer supporting Glamor, it was time to plug the Glamor support into the top of the driver. That meant changing a bunch of the entry points to select appropriate Glamor or UXA functionality, instead of just calling into UXA. So, now we ve got lots of places that look like:

        switch (intel->accel)  
#if USE_GLAMOR
        case ACCEL_GLAMOR:
                if (!intel_glamor_create_screen_resources(screen))
                        return FALSE;
                break;
#endif
#if USE_UXA
        case ACCEL_UXA:
                if (!intel_uxa_create_screen_resources(screen))
                        return FALSE;
        break;
#endif
        case ACCEL_NONE:
                if (!intel_none_create_screen_resources(screen))
                        return FALSE;
                break;
         

Using a switch means that we can easily elide code that isn t wanted in a particular build. Of course accel is an enum, so places which are missing one of the required paths will cause a compiler warning. It s not all perfectly clean yet; there are piles of UXA-only paths still. Making It Build Without UXA The final trick was to make the driver build without UXA turned on; that took several iterations before I had the symbols sorted out appropriately. I built the driver with various acceleration options and then tried to count the lines of source code. What I did was just list the source files named in the driver binary itself. This skips all of the header files and the render program source code, and ignores the fact that there are a bunch of #ifdef s in the uxa directory selecting between uxa, glamor and none.

    Accel                    Lines          Size(B)
    -----------             ------          -------
    none                      7143            73039
    glamor                    7397            76540
    uxa                      25979           283777
    sna                     118832          1303904
    none legacy              14449           152480
    glamor legacy            14703           156125
    uxa legacy               33285           350685
    sna legacy              126138          1395231

The legacy addition supports i810-class hardware, which is needed for a complete driver. Along The Way, Enable Tiling for the Front Buffer While hacking the code, I discovered that the initial frame buffer allocated for the screen was created without tiling because a few parameters that depend on the GTT size were not initialized until after that frame buffer was allocated. I haven t analyzed what effect this has on performance. Page Flipping and Resize Page flipping (or just flipping) means switching the entire display from one frame buffer to another. It s generally the fastest way of updating the screen as you don t have to copy any bits. The trick with flipping is that a client hands you a random pixmap and you need to stuff that into the KMS API. With UXA, that s pretty easy as all pixmaps are managed through the UXA API which knows which underlying kernel BO is tied with each pixmap. Using Glamor, only the underlying GL driver knows the mapping. Fortunately (?), we have the EGL Image extension, which lets us take a random GL texture and turn it into a file descriptor for a DMA-BUF kernel object. So, we have this cute little dance:

fd = glamor_fd_from_pixmap(screen,
                               pixmap,
                               &stride,
                               &size);
bo = drm_intel_bo_gem_create_from_prime(intel->bufmgr, fd, size);
    close(fd);
    intel_glamor_get_pixmap(pixmap)->bo = bo;

That last bit remembers the bo in some local memory so we don t have to do this more than once for each pixmap. glamorfdfrompixmap ends up calling eglCreateImageKHR followed by gbmbo_import and then a kernel ioctl to convert a prime handle into an fd. It s all quite round-about, but it does seem to work just fine. After I d gotten Glamor mostly working, I tried a few OpenGL applications and discovered flipping wasn t working. That turned out to have an unexpected consequence all full-screen applications would run flat-out, and not be limited to frame rate. Present recovers from a failed flip queue operation by immediately performing a CopyArea; not waiting for vblank. This needs to get fixed in Present by having it re-queued the CopyArea for the right time. What I did in the intel driver was to add a bunch more checks for tiling mode, pixmap stride and other things to catch pixmaps that were going to fail before the operation was queued and forcing them to fall back to CopyArea at the right time. The second adventure was with XRandR. Glamor has an API to fix up the screen pixmap for a new frame buffer, but that pulls the size of the frame buffer out of the pixmap instead of out of the screen. XRandR leaves the pixmap size set to the old screen size during this call; fixing that just meant getting the pixmap size set correctly before calling into glamor. I think glamor should get fixed to use the screen size rather than the pixmap size. Painting Root before Mode set The X server has generally done initialization in one order:

1. Create root pixmap
2. Set video modes
3. Paint root window

Recently, we ve added a -background none option to the X server which causes it to set the root window background to none and have the driver fill in that pixmap with whatever contents were on the screen before the X server started. In a pre-Glamor world, that was done by hacking the video driver to copy the frame buffer console contents to the root pixmap as it was created. The trouble here is that the root pixmap is created long before the upper layers of the X server are ready for drawing, so you can t use the core rendering paths. Instead, UXA had kludges to call directly into the acceleration functions. What we really want though is to change the order of operations:

1. Create root pixmap
2. Paint root window
3. Set video mode

That way, the normal root window painting operation will take care of getting the image ready before that pixmap is ever used for scanout. I can use regular core X rendering to get the original frame buffer contents into the root window, and even if we re not using -background none and are instead painting the root with some other pattern (like the root weave), I get that presented without an intervening black flash. That turned out to be really easy just delay the call to I830EnterVT (which sets the modes) until the server is actually running. That required one additional kludge I needed to tell the DIX level RandR functions about the new modes; the mode setting operation used during server init doesn t call up into RandR as RandR lists the current configuration after the screen has been initialized, which is when the modes used to be set. Calling xf86RandR12CreateScreenResources does the trick nicely. Getting the root window bits from fbcon, setting video modes and updating the RandR/Xinerama DIX info is now all done from the BlockHandler the first time it is called. Performance I ran the current glamor version of the intel driver with the master branch of the X server and there were not any huge differences since my last Glamor performance evaluation aside from GetImage. The reason is that UXA/Glamor never called Glamor s image functions, and the UXA GetImage is pretty slow. Using Mesa s image download turns out to have a huge performance benefit:

1. UXA/Glamor from April
2. Glamor from today
       1                 2                 Operation
------------   -------------------------   -------------------------
     50700.0        56300.0 (     1.110)   ShmGetImage 10x10 square 
     12600.0        26200.0 (     2.079)   ShmGetImage 100x100 square 
      1840.0         4250.0 (     2.310)   ShmGetImage 500x500 square 
      3290.0          202.0 (     0.061)   ShmGetImage XY 10x10 square 
        36.5          170.0 (     4.658)   ShmGetImage XY 100x100 square 
         1.5           56.4 (    37.600)   ShmGetImage XY 500x500 square 
     49800.0        50200.0 (     1.008)   GetImage 10x10 square 
      5690.0        19300.0 (     3.392)   GetImage 100x100 square 
       609.0         1360.0 (     2.233)   GetImage 500x500 square 
      3100.0          206.0 (     0.066)   GetImage XY 10x10 square 
        36.4          183.0 (     5.027)   GetImage XY 100x100 square 
         1.5           55.4 (    36.933)   GetImage XY 500x500 square

Running UXA from today the situation is even more dire; I suspect that enabling tiling has made CPU reads through the GTT even worse than before?

1: UXA today
2: Glamor today
       1                 2                 Operation
------------   -------------------------   -------------------------
     43200.0        56300.0 (     1.303)   ShmGetImage 10x10 square 
      2600.0        26200.0 (    10.077)   ShmGetImage 100x100 square 
       130.0         4250.0 (    32.692)   ShmGetImage 500x500 square 
      3260.0          202.0 (     0.062)   ShmGetImage XY 10x10 square 
        36.7          170.0 (     4.632)   ShmGetImage XY 100x100 square 
         1.5           56.4 (    37.600)   ShmGetImage XY 500x500 square 
     41700.0        50200.0 (     1.204)   GetImage 10x10 square 
      2520.0        19300.0 (     7.659)   GetImage 100x100 square 
       125.0         1360.0 (    10.880)   GetImage 500x500 square 
      3150.0          206.0 (     0.065)   GetImage XY 10x10 square 
        36.1          183.0 (     5.069)   GetImage XY 100x100 square 
         1.5           55.4 (    36.933)   GetImage XY 500x500 square

Of course, this is all just x11perf, which doesn t represent real applications at all well. However, there are applications which end up doing more GetImage than would seem reasonable, and it s nice to have this kind of speed up. Status I m running this on my crash box to get some performance numbers and continue testing it. I ll switch my desktop over when I feel a bit more comfortable with how it s working. But, I think it s feature complete at this point. Where s the Code As usual, the code is in my personal repository. It s on the glamor branch.

git://people.freedesktop.org/~keithp/xf86-video-intel  glamor

Some days ago I decided to upgrade my laptop from stable to testing. I had tried Jessie since several months, in my husband s laptop, but that was a fresh install, and a not-so-old laptop, and we have not much software installed there. In my netbook (Compaq Mini 110c), with stable, I already had installed Pumpa, Dianara and how-can-i-help from testing, and since the freeze is coming, I thought that I could full-upgrade and use Jessie from now on, and report my issues and help to diagnose or fix them, if possible, before the freeze. I keep Debian stable at work for my desktop and servers (well, some of them are still in oldstable, thanks LTS team!!), and I have testing in a laptop that I use as clonezilla/drbl server (but I had issues, next week I ll put some time on them and I ll write here my findings, and report bugs, if any). So! let s go. Here I write my experience and the issues that I found (very few! and not sure if they are bugs or configuration problems or what, I ll keep an eye on them). The upgrade I pointed my /etc/apt/sources.list to jessie, then apt-get update, then apt-get dist-upgrade. (With the servers I am much more careful, read the release notes, upgrade guides and so, or directly I go for a fresh install, but with my laptop, I am too lazy). I went to bed (wow, risky LArjona!) and when I got up for going to work, the laptop was waiting for me to accept to block root from ssh access, or restart some services, and so. Ok! the upgrade resumed but I have to go to work and I wanted my laptop! Since all the packages were already downloaded, I closed the lid (double risky LArjona!) unplugged it, put everything in my bag, and catched the bus in time :) At the bus, I opened again the lid of my laptop (crossing fingers!) and perfect, the laptop had suspended and returned back to life, and the upgrade just resumed with no problem. Wow! I love you Debian! After 15 minutes, I had to suspend again, since the bus arrived and I had to take the metro. In the metro, the upgrade resumed, and finished. I shutdown my laptop and arrive to work. Testing testing :) In a break for lunch, I opened my brand new laptop (the hardware is the same, but the software totally renewed, so it s brand new for me). I have to say that use xfce, with some GNOME/GTK apps installed (gedit, cheese, evince, XChat ) and some others that use Qt or are part of the KDE project (Okular, Kile, QtLinguist, Pumpa, Dianara). I don t know/care too much about desktops and tweaking my desktop: I just put the terminal and gedit in black background, Debian wallpaper is enough dark for me so ok, put the font size a bit smaller to better use my low-vertical-resolution, and that s all, I only go to configure something else if there s something that really annoys me. My laptop booted correctly and a nice, more modern LightDM was greeting me. I logged in and everything worked ok, except some issues that follow. Network Manager and WPA2-enterprise wireless connections I had to reconfigure some wireless connections in Network Manager. At the University we use WPA2-enterprise, TTLS + PAP. I had stored my username and password in the connection, and network manager seemed to remember the username but not the password. No problem, I said, and I wrote it when it asked, but the Save or OK button was greyed out. I could not click it. Then I went to edit the connections, and more or less the same, it seems that I could edit, but not save the (new) configuration. Finally, I removed the wireless connection and created it again, and everything worked as a charm. This, I had to do it with the two wireless in my University (both of them are WPA2-enterprise TTLS + PAP). At home, I have WPA2 personal, and I had no issues, everything worked ok. This problem is not appearing in a fresh install, since there are no old configs to keep. Adblock Plus not working any more I opened Iceweasel and I began to see ads in the webpages that I visited. What? I checked and Adblock plus was installed and activated I reinstalled the package xul-ext-adblock-plus and it worked again. Strange display in programs based on Qt When I opened Pumpa I noticed that the edges of the windows where too rough, as if it was not using a desktop theme. I asked to a friend that uses Plasma and he suggested to install qt4-qtconfig, and then, select a theme for my Qt apps. It worked like a charm, but I find strange that I didn t need it before in stable. Maybe the default xfce configuration from stable is setting a theme, and the new one is not setting it, and so, the Qt apps are left barefoot .
With qtconfig I chose a GTK+ Style GUI for my Qt apps and then, they looked similar to what I had in stable (frankly, I cannot say if it was similar or exactly the same , but I didn t find them strange as before, so I m fine). Strange display in programs from GNOME Well, this is not a Jessie problem, it s just that some programs adopted the new GNOME appearance, and since I m on xfce, not on GNOME, they look a bit strange (no menus integration, and so). I am not sure that I can run GNOME (fallback, classic?) in my 1 GB RAM laptop, I have to investigate if I can tweak it to use less memory, or what. I m not very tied to xfce, and in fact it does not look so light (well, on top of it, I don t run light programs, I run Iceweasel, Icedove, Libreoffice, and some others). At work I use GNOME in my desktop, but with GNOME shell, not the fallback or classic modes, so I m thinking about giving a chance to MATE or second chance to LXDE. We ll see. Issues when opening the lid (waking up from suspend) This is the most strange thing I found in the migration, and the most dangerous one, I think. As I said before, I don t tweak too much my desktop, if it works with the default configuration. I m not sure that I know the differences between suspend, hibernate, hard disks disconnections and so. When I was in stable, and I closed the lid of my laptop, it just shutdown the screen, then I heard something like the system going to suspend or whatever, and after some seconds, the harddisk and fans stop, the wireless led turns off, and the power led begins to blink. Ok. When I open the lid, then it was waking up itself (the power led stayed on, the wireless led turns on, and when I tap the touchpad or type anything, the screen was coming, with the xscreensaver asking for my password). Just sometimes, when the screen was turning on, I could see my desktop for less than a second, before xscreensaver turns the background black and asks for the password. Now since I migrated to Jessie, I m experiencing a different behavior. When I close the lid, the laptop behaves the same. When I open the lid, the laptop behaves the same, but when I type or tap the touchpad and xscreensaver comes to ask the password, before than I can type it, the laptop just suspends again (or hibernates, I m not sure), and I have to press the power button in order to bring it back to life (then I see the xscreensaver again asking for the password, I type it, and my desktop is there, the same as I left it when I closed the lid). Strange, isn t it? I have tried to suspend my laptop directly from the menu, and it comes to the same state in which I have to press the power button in order to bring it back to life, but then, no xscreensaver password is required (which is double strange, IMHO). Things I miss in Jessie Well, until now, the only thing I miss in Jessie is the software center. I rarely use it (I love apt) but I think it makes a good job in easing the installation of programs in Debian for people coming from other operative systems (specially after smartphones and their copied software stores became popular). I hope the maintainer can upload a new version before the freeze, and so, it enters in the release. I ll try to contact him. Update 2014/07/20: Julian Andres Klode, maintainer of software-center, just replied (see his comment below) and pointed to GNOME Software (gnome-packagekit) as alternative. I just installed and it looks neat and nice. I m very happy! TODO I have a Debian stable laptop at work (this one with xfce + GNOME), I ll try to upgrade it and see if I see the same problems that I notice in mine. Then, I ll check the corresponding packages to see if there are open bugs about them, and if not, report them to their maintainers. I have to review the wiki pages related to the Jessie Desktop theme selection, I think they wanted the wallpaper to be inside before the freeze. Maybe I can help in publicity about that, handle the votings and so. I like Joy, but it s time to change a bit, new fresh air into the room!
Filed under: My experiences and opinion Tagged: Contributing to libre software, Debian, English, Free Software, Moving into free software

Wallet is the secure credential management infrastructure that we use at Stanford, primarily for keytabs but increasingly for any sort of security keys that have to be stored somewhere and retrieved by specific systems or people. The primary goal of this release is to add Duo support. This is currently somewhat preliminary, with only a single Duo integration object type that creates a UNIX integration. (Well, technically it can create any type of integration, but the integration information is returned in the format expected by the UNIX integration.) I expect a later release to rename all existing "duo" object types to "duo-unix" and add additional object types for the various other types of integrations that one wants to support, but that work will have to wait for another day. Since it's been over a year since the previous release, there are also other accumulated bug fixes and improvements. I also tried to merge or address as many issues or patches that had been sent to me over the past year as I could, although many larger patches or improvements had to be deferred. Highlights:

• The owner and getacl commands now return the name of the ACL instead of its numeric ID, as they probably should have from the beginning.
• The date passed to expires can now be in any format Date::Parse supports. (On a related note, Date::Parse is now required.)
• wallet-rekey now works properly on keytabs containing multiple principals. I had for some reason assumed that one could form a keytab containing multiple principals by just concatenating several together, but that definitely does not work. wallet-rekey now appends new keys to the end of the existing keytab. Unfortunately, I didn't get a chance to implement purging of old keys, for the folks stuck with MIT Kerberos ktutil instead of Heimdal's.

There are also multiple other bug fixes and general improvements, such as using DateTime objects uniformly for all database access that involves date fields, and recording ACL renames in the ACL history table. Both the API and the database layer are still kind of a mess, and I'd love to rewrite them with the benefit of experience and more knowledge, but that's a project for another day. You can get the latest release from the wallet distribution page.

Today I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs. Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what s new in AppStream 0.7: Specification changes Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language. A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications. The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service: (if you are using the cli tool) A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example The KDE Community , GNOME Developers or even the developer s full name. This value can be (optionally) translated and will be displayed in software-centers. An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification. Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs). Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied). As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags) Documentation updates In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution. Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have Quickstart pages in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future. We also have an explicit section dealing with the question How do I translate upstream metadata? now. More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop. AppStream GObject library and tools The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses I like it for tivoization protection, it s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS). On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit s desktop-file-database in the long term). The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format). The appstream-index tool received some changes, making it s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers. Additionally, the testsuite got improved and should now work on systems which do not have metadata installed. Of course, libappstream also implements all features of the new 0.7 specification. With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the pkgname property of AsComponent (which is now a string array and called pkgnames ). API level was bumped to 1. Appstream-Qt A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code! Special thanks to Sune Vuorela for his nice rework of the Qt library! And that s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

Review: Neptune's Brood, by Charles Stross

Series:	Freyaverse #2
Publisher:	Ace
Copyright:	July 2013
ISBN:	1-101-62453-1
Format:	Kindle
Pages:	325

Neptune's Brood is set in the same universe as Saturn's Children, but I wouldn't call it a sequel. It takes place considerably later, after substantial expansion of the robot civilization to the stars, and features entirely different characters (or, if there was overlap, I didn't notice). It also represents a significant shift in tone: while Saturn's Children is clearly a Heinlein pastiche and parody, Neptune's Brood takes its space opera more seriously. There is some situational humor assault auditors, for example but this book is played mostly straight, and I detected little or no Heinlein. This is Stross fleshing out his own space opera concept. This being Stross, that concept is not exactly conventional. This is a space opera about economics. Specifically, it's a space opera about interstellar economics, a debt pyramid, and a very interesting remapping of the continual growth requirements of capitalism to the outward expansion of colonization. The first-person protagonist comes from a "family" (as in Saturn's Children, the concept exists but involves rather more aggressive control of the instantiated "children") of bankers, but she is a forensic accountant and historian who specializes in analysis of financial scams. As you might expect, this is a significant clue about the plot. Neptune's Brood opens with Krina in search of her sister. She is supposed to be an itinerant scholar, moving throughout colonized space to spend some time with various scattered sisters, spreading knowledge and expanding her own. But it's clear from the start of the book that something else is going on, even before an assassin with Krina's face appears on her trail. Unfortunately, it takes roughly a third of the book to learn just what is happening beneath the surface, and most of that time is spent in a pointless interlude on a flying cathedral run by religious fanatics. The religion is a callback to Saturn's Children: robots who are trying to spread original humanity (the Fragiles) to the stars. This mostly doesn't go well, and is going particularly poorly for the ship that Krina works for passage on. But this is a brief gag that I thought went on much too long. The plot happens to Krina for this first section of the book rather than the other way around, little of lasting significance other than some character introductions occurs, and the Church itself, while playing a minor role in the later plot, is not worth the amount of attention that it gets. The best parts of the early book are the interludes in which Krina explains major world concepts to the reader. These are absolutely blatant infodumping, and I'm not sure how Stross gets away with them, but somehow he does, at least for me. They remind me of some of his blog posts, except tighter and fit into an interesting larger structure. Thankfully, once Krina finally arrives on Shin-Tethys, the plot improves considerably. There was a specific moment for me when the book became interesting: when Krina finds her sib's quarters in Shin-Tethys and analyzes what she finds there. It's the first significant thing in the book that she does rather than have done to her or thrust upon her, and she's a much better character when she's making decisions. This is also about the point where Stross starts fully explaining slow money, which is key to both the economics and the plot, and the plot starts to unwind its various mysteries and identify the motives of the players. Even then, Krina suffers from a lack of agency. Only at rare intervals does she get a chance to affect the story. Most of what she did of relevance to this book she did in the past, and while those descriptions of the backstory are interesting, they don't entirely make up for a passive protagonist. Thankfully, the other characters are varied and interesting enough, and the political machinations and cascading revelations captivating enough, that the last part of the book was very satisfying even with Krina on for the ride. This is a Stross novel, so it's full of two-dollar technical words mixed with technobabble. However, it shares with Saturn's Children the recasting of robots as the norm and fleshy humans as the exception, which means much of the technobabble is a straight substitution for our normal babble about meaty bodies and often works as an alienation technique. That makes it a bit more tolerable for me, although I still wished Stross would turn down the manic vocabulary in places. This bothers some people more than others; if you had no trouble with Accelerando, Neptune's Brood will pose no problems. I don't think the first section of this book was successful, but I liked the rest well enough to recommend it. If you like your space opera with a heavy dose of economics, a realistic attitude towards deep space exploration without faster-than-light technology, and a realistic perspective on the hostility of alien planets to Earth life, Neptune's Brood is a good choice. And any book that quotes David Graeber's Debt, and whose author has clearly paid attention to its contents, wins bonus points from me. Rating: 8 out of 10

While I was out at the (immensely impressive and equally enjoyable) useR! 2014 conference at UCLA, Conrad provided a bug-fix release 4.320 of Armadillo, the nifty templated C++ library for linear algebra. I quickly rolled that into RcppArmadillo release 0.4.320.0 which has been on CRAN and in Debian for a good week now. This release fixes some minor things with sparse and dense Eigen solvers (as well as one RNG issue probably of lesser interest to R users deploying the RNGs from R) as shown in the NEWS entry below.

Changes in RcppArmadillo version 0.4.320.0 (2014-07-03)

• Upgraded to Armadillo release Version 4.320 (Daintree Tea Raider)
  • expanded eigs_sym() and eigs_gen() to use an optional tolerance parameter
  • expanded eig_sym() to automatically fall back to standard decomposition method if divide-and-conquer fails
  • automatic installer enables use of C++11 random number generator when using gcc 4.8.3+ in C++11 mode

Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

As my previous post suggested I'd been running a service for a few years, using Redis as a key-value store. Redis is lovely. If your dataset will fit in RAM. Otherwise it dies hard. Inspired by Memcached, which is a simple key=value store, redis allows for more operations: using sets, using hashes, etc, etc. As it transpires I mostly set keys to values, so it crossed my mind last night an alternative to rewriting the service to use a non-RAM-constrained service might be to juggle redis out and replace it with something else. If it were possible to have a redis-compatible API which secretly stored the data in leveldb, sqlite, or even Berkley DB, then that would solve my problem of RAM-constraints, and also be useful. Looking around there are a few projects in this area nds fork of redis, ssdb, etc. I was hoping to find a Perl Redis::Server module, but sadly nothing exists. I should look at the various node.js stub-servers which exist as they might be easy to hack too. Anyway the short version is that this might be a way forward, the real solution might be to use sqlite or postgres, but that would take a few days work. For the moment the service has been moved to a donated guest and has 2Gb of RAM instead of the paltry 512Mb it was running on previously. Happily the server is installed/maintained by my slaughter tool so reinstalling took about ten minutes - the only hard part was migrating the Redis-contents, and that's trivial thanks to the integrated "slave of" support. (I should write that up regardless though.)

printer-driver-postscript-hp Depends: hplip
hplip Depends: policykit-1
policykit-1 Depends: libpam-systemd
libpam-systemd Depends: systemd (= 204-14)

Since the last in the above is a versioned dependency, that means you can't use systemd-shim to satisfy this dependency. I do think we should migrate to systemd. However, it's unfortunate that this change is being rushed like this. I want to migrate my personal laptop to systemd but not before I have the time to deal with any fallout that might result, and to make sure I can properly migrate my configuration. Workaround (for now): hold policykit-1 at 0.105-3 rather than have it upgrade to 0.105-6. That version doesn't have a dependency on libpam-systemd. Off-hand questions:

• Why does one need to log in to an init system? (yes, yes, it's probably a session PAM module, not an auth or password module. Still)
• What does policykit do that can't be solved with proper use of Unix domain sockets and plain old unix groups?

All this feels like another case of overengineering, like most of the *Kit thingies. Update: so the systemd package doesn't actually cause systemd to be run, there are other packages that do that, and systemd-shim can be installed. I misread things. Of course, the package name is somewhat confusing... but that's no excuse.

Once upon a time I setup a centralized service for spam-testing blog/forum-comments in real time, that service is BlogSpam.net. This was created because the Debian Administration site was getting hammered with bogus comments, as was my personal blog. Today the unfortunate thing happened, the virtual machine this service was running on ran out of RAM and died - The redis-store that holds all the state has now exceeded the paltry 512Mb allocated to the guest, so OOM killed it. So I'm at an impasse - I either recode it to use MySQL instead of Redis, or something similar to allow the backing store to exceed the RAM-size, or I shut the thing down. There seems to be virtually no liklihood somebody would sponsor a host to run the service, because people just don't pay for this kind of service. I've temporarily given the guest 1Gb of RAM, but that comes at a cost. I've had to shut down my "builder" host - which is used to build Debian packages via pbuilder. Offering an API, for free, which has become increasingly popular and yet equally gets almost zero feedback or "thanks" is a bit of a double-edged sword. Because it has so many users it provides a better service - but equally costs more to run in terms of time, effort, and attention. (And I just realized over the weekend that my Flattr account is full of money (~50 euro) that I can't withdraw - since I deleted my paypal account last year. Ooops.) Meh. Happy news? I'm avoiding the issue of free service indefinitely with the git-based DNS product which was covering costs and now is .. doing better. (20% off your first months bill with coupon "20PERCENT".)

After dealing with the client side of the DNSSEC puzzle last week, I thought it behooved me to also go about getting DNSSEC going on the domains I run DNS for. Like the resolver configuration, the server side work is straightforward enough once you know how, but boy howdy are there some landmines to be aware of. One thing that made my job a little less ordinary is that I use and love tinydns. It s an amazingly small and simple authoritative DNS server, strong in the Unix tradition of do one thing and do it well . Unfortunately, DNSSEC is anything but small and simple and so tinydns doesn t support DNSSEC out of the box. However, Peter Conrad has produced a patch for tinydns to do DNSSEC, and that does the trick very nicely. A brief aside about tinydns and DNSSEC, if I may Poor key security is probably the single biggest compromise vector for crypto. So you want to keep your keys secure. A great way to keep keys secure is to not put them on machines that run public-facing network services (like DNS servers). So, you want to keep your keys away from your public DNS servers. A really great way of doing that would be to have all of your DNS records somewhere out of the way, and when they change regenerate the zone file, re-sign it, and push it out to all your DNS servers. That happens to be exactly how tinydns works. I happen to think that tinydns fits very nicely into a DNSSEC-enabled world. Anyway, back to the story. Once I d patched the tinydns source and built updated packages, it was time to start DNSSEC-enabling zones. This breaks down into a few simple steps:

1. Generate a key for each zone. This will produce a private key (which, as the name suggests, you should keep to yourself), a public key in a DNSKEY DNS record, and a DS DNS record. More on those in a minute. One thing to be wary of, if you re like me and don t want or need separate Key Signing and Zone Signing keys. You must generate a Key Signing key this is a key with a flags value of 257. Doing this wrong will result in all sorts of odd-ball problems. I wanted to just sign zones, so I generated a Zone Signing key, which has a flags value of 256. Big mistake. Also, the DS record is a hash of everything in the DNSKEY record, so don t just think you can change the 256 to a 257 and everything will still work. It won t.
2. Add the key records to the zone data. For tinydns, this is just a matter of copying the zone records from the generated key into the zone file itself, and adding an extra pseudo record (it s all covered in the tinydnssec howto).
3. Publish the zone data. Reload your BIND config, run tinydns-sign and tinydns-data then rsync, or do whatever it is PowerDNS people do (kick the database until replication starts working again?).
4. Test everything. I found the Verisign Labs DNSSEC Debugger to be very helpful. You want ticks everywhere except for where it s looking for DS records for your zone in the higher-level zone. If there are any other freak-outs, you ll want to fix those because broken DNSSEC will take your domain off the Internet in no time.
5. Tell the world about your DNSSEC keys. This is simply a matter of giving your DS record to your domain registrar, for them to add it to the zone data for your domain s parent. Wherever you d normally go to edit the nameservers or contact details for your domain, you probably want to do to the same place and look for something about DS or Domain Signer records. Copy and paste the details from the DS record in your zone into there, submit, and wait a minute or two for the records to get published.
6. Test again. Before you pat yourself on the back, make sure you ve got a full board of green ticks in the DNSSEC Debugger. if anything s wrong, you want to rollback immediately, because broken DNSSEC means that anyone using a DNSSEC-enabled resolver just lost the ability to see your domain.

That s it! There s a lot of complicated crypto going on behind the scenes, and DNSSEC seems to revel in the number of acronyms and concepts that it introduces, but the actual execution of DNSSEC-enabling your domains is quite straightforward.

The security model on the Google Nexus devices is pretty straightforward. The OS is (nominally) secure and prevents anything from accessing the raw MTD devices. The bootloader will only allow the user to write to partitions if it's unlocked. The recovery image will only permit you to install images that are signed with a trusted key. In combination, these facts mean that it's impossible for an attacker to modify the OS image without unlocking the bootloader[1], and unlocking the bootloader wipes all your data. You'll probably notice that.

The problem comes when you want to run something other than the stock Google images. Step number one for basically all of these is "Unlock your bootloader", which is fair enough. Step number two is "Install a new recovery image", which is also reasonable in that the key database is stored in the recovery image and so there's no way to update it without doing so. Except, unfortunately, basically every third party Android image is either unsigned or is signed with the (publicly available) Android test keys, so this new recovery image will flash anything. Feel free to relock your bootloader - the recovery image will still happily overwrite your OS.

This is unfortunate. Even if you've encrypted your phone, anyone with physical access can simply reboot into recovery and reflash /system with something that'll stash your encryption key and mail your data to the NSA. Surely there's a better way of doing this?

Thankfully, there is. Kind of. It's annoying and involves a bunch of manual processes and you'll need to re-sign every update yourself. But it is possible to configure Nexus devices in such a way that you retain the same level of security you had when you were using the Google keys without losing the freedom to run whatever you want. Here's how.

Note: This is not straightforward. If you're not an experienced developer, you shouldn't attempt this. I'm documenting this so people can create more user-friendly approaches.

First: Unlock your bootloader. /data will be wiped.
Second: Get a copy of the stock recovery.img for your device. You can get it from the factory images available here
Third: Grab mkbootimg from here and build it. Run unpackbootimg against recovery.img.
Fourth: Generate some keys. Get this script and run it.
Fifth: zcat recovery.img-ramdisk.gz cpio -id to extract your recovery image ramdisk. Do this in an otherwise empty directory.
Sixth: Get DumpPublicKey.java from here and run it against the .x509.pem file generated in step 4. Replace /res/keys from the recover image ramdisk with the output. Include the "v2" bit at the beginning.
Seventh: Repack the ramdisk image (find . cpio -o -H newc gzip > ../recovery.img-ramdisk.gz) and rebuild recovery.img with mkbootimg.
Eighth: Write the new recovery image to your device
Ninth: Get signapk from here and build it. Run it against the ROM you want to sign, using the keys you generated earlier. Make sure you use the -w option to sign the whole zip rather than signing individual files.
Tenth: Relock your bootloader
Eleventh: Boot into recovery mode and sideload your newly signed image.

At this point you'll want to set a reasonable security policy on the image (eg, if it grants root access, ensure that it requires a PIN or something), but otherwise you're set - the recovery image can't be overwritten without unlocking the bootloader and wiping all your data, and the recovery image will only write images that are signed with your key. For obvious reasons, keep the key safe.

This, well. It's obviously an excessively convoluted workflow. A *lot* of it could be avoided by providing a standardised mechanism for key management. One approach would be to add a new fastboot command for modifying the key database, and only permit this to be run when the bootloader is unlocked. The workflow would then be something like

• Unlock bootloader
• Generate keys
• Install new key
• Lock bootloader
• Sign image
• Install image

which seems more straightforward. Long term, individual projects could do the signing themselves and distribute their public keys, resulting in the install process becoming as easy as

• Unlock bootloader
• Install ROM key
• Lock bootloader
• Install ROM

which is actually easier than the current requirement to install an entirely new recovery image.

I'd actually previously criticised Google on the grounds that using custom keys wasn't possible on Android devices. I was wrong. It is, it's just that (as far as I can tell) nobody's actually documented it before. It's important that users not be forced into treating security and freedom as mutually exclusive, and it's great that Google have made that possible.

[1] This model fails if it's possible to gain root on the device. Thankfully this would never hold on what's that over there is that a distraction?

comments

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (168.17 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian LTS After having put in place the infrastructure to allow companies to contribute financially to Debian LTS, I spent quite some time to draft the announce of the launch of Debian LTS (on a suggestion of Moritz M hlenhoff who pointed out to me that there was no such announce yet). I m pretty happy about the result because we managed to mention a commercial offer without generating any pushback from the community. The offer is (in my necessarily biased opinion) clearly in the interest of Debian but still the money doesn t go to Debian so we took extra precautions. When I got in touch with the press officers, I included the Debian leader in the discussion and his feedback has been very helpful to improve the announce. He also officially acked the press release to give some confidence to the press officers that they were doing the right thing. Lucas also pushed me to seek public review of the draft press release, which I did. The discussion was constructive and the draft got further improved. The news got widely relayed, but on the flip side, the part with the call for help got almost no attention from the press. Even Linux Weekly News skipped it! On the Freexian side, we just crossed 10% of a full-time position (funded by 6 companies) and we are in contact with a few other companies in discussion. But we re far from our goal yet so we will have to actively reach out to more companies. Do you know companies who are still running Debian 6 servers ? If yes, please send me the details (name + url + contact info if possible) to deblts@freexian.com so that I can get in touch and invite them to contribute to the project. Distro Tracker In the continuation of the Debian France game, I continued to work together with Joseph Herlant and Christophe Siraut on multiple improvements to distro tracker in order to prepare for its deployment on tracker.debian.org (which I just announced \o/). Debian France Since the Debian France game was over, I shipped the rewards. 5 books have been shipped to:

• Joseph Herlant and Christophe Siraut for their distro-tracker work
• Dylan Aissi for his help within the Debian Med team
• Samuel Dorsaz and Thomas Debesse for their work towards better support of Brother printers

Misc Debian work I orphaned sql-ledger and made a last upload to change the maintainer to Debian QA (with a new upstream version). After having been annoyed a few times by dch breaking my name in the changelog, I filed #750855 which got quickly fixed. I disabled a broken patch in quilt to fix RC bug #751109. I filed #751771 when I discovered an incorrect dependency on ruby-uglifier (while doing packaging work for Kali Linux). I tested newer versions of ruby-libv8 on armel/armhf on request of the upstream author. I had reported him those build failures (github ticket here). Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

I have a number of USB hard disks. Like, I suppose, mostly everyone who reads this blog. Unlike many people who do, however, for whatever reason I decided to create LVM volumes on most of my USB hard disks. The unfortunate result is that these now contain a lot of data with a somewhat less than efficient partitioning system. I don't really care much, but it's somewhat annoying, not in the least because disconnecting an LVM device isn't as easy as it used to be; originally you could just run the lvm2 init script with the stop argument, but that isn't the case anymore today. That is, you can run that, but it won't help you because all that does, effectively, is exit 0. So what do you do instead? This:

• First, make sure your devices aren't mounted anymore. Note: do not use lazy umount for a device that you're going to remove from your system! I've seen a few forum posts here and there of people who think it's safe to use umount -l for a device they're about to remove from their system which is still in use. It's not. It's a good way to cause data loss. Instead, make sure your partitions are really unmounted. Use fuser -m if you need to figure out which process is still using the partition.
• Next, use vgchange -a n. This will cause LVM to deactivate any logical volumes and volume groups that aren't open any more. Note that this can't work if you haven't done the above. Also note that this doesn't cause the devices to be gone when you do things like vgs or so. They're still there, they're just not in use anymore. Skipping this step isn't recommended, though; it will make LVM unhappy, mostly because some caches are still in use.
• Remove your device from the computer. That is, disconnect the USB cable, or call nbd-client -d, or do whatever you need to make sure the PV isn't connected to your system anymore.
• Finally, run vgchange --refresh. This will cause the system to rescan all partitions, notice that the volume groups which you've just disconnected aren't there anymore, and remove them from configuration.

Voila, your LVM volume group is no longer available, and you've not suffered data loss. Kewl. Note: I don't know what the lvm2 init script used to do. I suspect there's another way which doesn't require the --refresh step. I don't think it matters all that much, though. This works, and is safe. That being said, comments are welcome...