Search Results: "cas"

21 July 2014

Chris Lamb: Disabling internet for specific processes with libfiu

My primary usecase is to prevent testsuites and build systems from contacting internet-based services. This, at the very least, introduces an element of non-determinism and malicious code at worst. I use Alberto Bertogli's libfiu for this, specifically the fiu-run utility which part of the fiu-utils package on Debian and Ubuntu. Here's a contrived example, where I prevent Curl from talking to the internet:
$ fiu-run -x -c 'enable name=posix/io/net/connect' curl google.com
curl: (6) Couldn't resolve host 'google.com'
... and here's an example of it detecting two possibly internet-connecting tests:
$ fiu-run -x -c 'enable name=posix/io/net/connect' ./manage.py text
[..]
----------------------------------------------------------------------
Ran 892 tests in 2.495s
FAILED (errors=2)
Destroying test database for alias 'default'...

Note that libfiu inherits all the drawbacks of LD_PRELOAD; in particular, we cannot limit the child process that calls setuid binaries such as /bin/ping:
$ fiu-run -x -c 'enable name=posix/io/net/connect' ping google.com
PING google.com (173.194.41.65) 56(84) bytes of data.
64 bytes from lhr08s01.1e100.net (17.194.41.65): icmp_req=1 ttl=57 time=21.7 ms
64 bytes from lhr08s01.1e100.net (17.194.41.65): icmp_req=2 ttl=57 time=18.9 ms
[..]
Whilst it would certainly be more robust and flexible to use iptables such as allowing localhost and other local socket connections but disabling all others I gravitate towards this entirely userspace solution as it requires no setup and I can quickly modify it to block other calls on an ad-hoc basis. The list of other "modules" libfiu supports is viewable here.

Ian Campbell: sunxi-tools now available in Debian

I've recently packaged the sunxi tools for Debian. These are a set of tools produce by the Linux Sunxi project for working with the Allwinner "sunxi" family of processors. See the package page for details. Thanks to Steve McIntyre for sponsoring the initial upload. The most interesting component of the package are the tools for working with the Allwinner processors' FEL mode. This is a low-level processor mode which implements a simple USB protocol allowing for initial programming of the device and recovery which can be entered on boot (usually be pressing a special 'FEL button' somewhere on the device). It is thanks to FEL mode that most sunxi based devices are pretty much unbrickable. The most common use of FEL is to boot over USB. In the Debian package the fel and usb-boot tools are named sunxi-fel and sunxi-usb-boot respectively but otherwise can be used in the normal way described on the sunxi wiki pages. One enhancement I made to the Debian version of usb-boot is to integrate with the u-boot packages to allow you to easily FEL boot any sunxi platform supported by the Debian packaged version of u-boot (currently only Cubietruck, more to come I hope). To make this work we take advantage of Multiarch to install the armhf version of u-boot (unless your host is already armhf of course, in which case just install the u-boot package):
# dpkg --add-architecture armhf
# apt-get update
# apt-get install u-boot:armhf
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  u-boot:armhf
0 upgraded, 1 newly installed, 0 to remove and 1960 not upgraded.
Need to get 0 B/546 kB of archives.
After this operation, 8,676 kB of additional disk space will be used.
Retrieving bug reports... Done
Parsing Found/Fixed information... Done
Selecting previously unselected package u-boot:armhf.
(Reading database ... 309234 files and directories currently installed.)
Preparing to unpack .../u-boot_2014.04+dfsg1-1_armhf.deb ...
Unpacking u-boot:armhf (2014.04+dfsg1-1) ...
Setting up u-boot:armhf (2014.04+dfsg1-1) ...
With that done FEL booting a cubietruck is as simple as starting the board in FEL mode (by holding down the FEL button when powering on) and then:
# sunxi-usb-boot Cubietruck -
fel write 0x2000 /usr/lib/u-boot/Cubietruck_FEL/u-boot-spl.bin
fel exe 0x2000
fel write 0x4a000000 /usr/lib/u-boot/Cubietruck_FEL/u-boot.bin
fel write 0x41000000 /usr/share/sunxi-tools//ramboot.scr
fel exe 0x4a000000
Which should result in something like this on the Cubietruck's serial console:
U-Boot SPL 2014.04 (Jun 16 2014 - 05:31:24)
DRAM: 2048 MiB
U-Boot 2014.04 (Jun 16 2014 - 05:30:47) Allwinner Technology
CPU:   Allwinner A20 (SUN7I)
DRAM:  2 GiB
MMC:   SUNXI SD/MMC: 0
In:    serial
Out:   serial
Err:   serial
SCSI:  SUNXI SCSI INIT
Target spinup took 0 ms.
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst 
Net:   dwmac.1c50000
Hit any key to stop autoboot:  0 
sun7i# 
As more platforms become supported by the u-boot packages you should be able to find them in /usr/lib/u-boot/*_FEL. There is one minor inconvenience which is the need to run sunxi-usb-boot as root in order to access the FEL USB device. This is easily resolved by creating /etc/udev/rules.d/sunxi-fel.rules containing either:
SUBSYSTEMS=="usb", ATTR idVendor =="1f3a", ATTR idProduct =="efe8", OWNER="myuser"
or
SUBSYSTEMS=="usb", ATTR idVendor =="1f3a", ATTR idProduct =="efe8", GROUP="mygroup"
To enable access for myuser or mygroup respectively. Once you have created the rules file then to enable:
# udevadm control --reload-rules
As well as the FEL mode tools the packages also contain a FEX (de)compiler. FEX is Allwinner's own hardware description language and is used with their Android SDK kernels and the fork of that kernel maintained by the linux-sunxi project. Debian's kernels follow mainline and therefore use Device Tree.

Keith Packard: Glamorous Intel

Reworking Intel Glamor The original Intel driver Glamor support was based on the notion that it would be better to have the Intel driver capture any fall backs and try to make them faster than Glamor could do internally. Now that Glamor has reasonably complete acceleration, and its fall backs aren t terrible, this isn t as useful as it once was, and because this uses Glamor in a weird way, we re making the Glamor code harder to maintain. Fixing the Intel driver to not use Glamor in this way took a bit of effort; the UXA support is all tied into the overall operation of the driver. Separating out UXA functions The first task was to just identify which functions were UXA-specific by adding _uxa to their names. A couple dozen sed runs and now a bunch of the driver is looking better. Next, a pile of UXA-specific functions were actually inside the non-UXA parts of the code. Those got moved out, and a new intel_uxa.h file was created to hold all of the definitions. Finally, a few non UXA-specific functions were actually in the uxa files; those got moved over to the generic code. Removing the Glamor paths in UXA Each one of the UXA functions had a little piece of code at the top like:
if (uxa_screen->info->flags & UXA_USE_GLAMOR)  
    int ok = 0;
    if (uxa_prepare_access(pDrawable, UXA_GLAMOR_ACCESS_RW))  
        ok = glamor_fill_spans_nf(pDrawable,
                      pGC, n, ppt, pwidth, fSorted);
        uxa_finish_access(pDrawable, UXA_GLAMOR_ACCESS_RW);
     
    if (!ok)
        goto fallback;
    return;
 
Pulling those out shrank the UXA code by quite a bit. Selecting Acceleration (or not) The intel driver only supported UXA before; Glamor was really just a slightly different mode for UXA. I switched the driver from using a bit in the UXA flags to having an accel variable which could be one of three options: I added ACCEL_NONE to give us a dumb frame buffer mode. That actually supports DRI3 so that we can bring up Mesa and run it under X before we have any acceleration code ready; avoiding a dependency loop when doing new hardware. All that it requires is a kernel that offers mode setting and buffer allocation. Initializing Glamor With UXA no longer supporting Glamor, it was time to plug the Glamor support into the top of the driver. That meant changing a bunch of the entry points to select appropriate Glamor or UXA functionality, instead of just calling into UXA. So, now we ve got lots of places that look like:
        switch (intel->accel)  
#if USE_GLAMOR
        case ACCEL_GLAMOR:
                if (!intel_glamor_create_screen_resources(screen))
                        return FALSE;
                break;
#endif
#if USE_UXA
        case ACCEL_UXA:
                if (!intel_uxa_create_screen_resources(screen))
                        return FALSE;
        break;
#endif
        case ACCEL_NONE:
                if (!intel_none_create_screen_resources(screen))
                        return FALSE;
                break;
         
Using a switch means that we can easily elide code that isn t wanted in a particular build. Of course accel is an enum, so places which are missing one of the required paths will cause a compiler warning. It s not all perfectly clean yet; there are piles of UXA-only paths still. Making It Build Without UXA The final trick was to make the driver build without UXA turned on; that took several iterations before I had the symbols sorted out appropriately. I built the driver with various acceleration options and then tried to count the lines of source code. What I did was just list the source files named in the driver binary itself. This skips all of the header files and the render program source code, and ignores the fact that there are a bunch of #ifdef s in the uxa directory selecting between uxa, glamor and none.
    Accel                    Lines          Size(B)
    -----------             ------          -------
    none                      7143            73039
    glamor                    7397            76540
    uxa                      25979           283777
    sna                     118832          1303904
    none legacy              14449           152480
    glamor legacy            14703           156125
    uxa legacy               33285           350685
    sna legacy              126138          1395231
The legacy addition supports i810-class hardware, which is needed for a complete driver. Along The Way, Enable Tiling for the Front Buffer While hacking the code, I discovered that the initial frame buffer allocated for the screen was created without tiling (!) because a few parameters that depend on the GTT size were not initialized until after that frame buffer was allocated. I haven t analyzed what effect this has on performance. Page Flipping and Resize Page flipping (or just flipping) means switching the entire display from one frame buffer to another. It s generally the fastest way of updating the screen as you don t have to copy any bits. The trick with flipping is that a client hands you a random pixmap and you need to stuff that into the KMS API. With UXA, that s pretty easy as all pixmaps are managed through the UXA API which knows which underlying kernel BO is tied with each pixmap. Using Glamor, only the underlying GL driver knows the mapping. Fortunately (?), we have the EGL Image extension, which lets us take a random GL texture and turn it into a file descriptor for a DMA-BUF kernel object. So, we have this cute little dance:
fd = glamor_fd_from_pixmap(screen,
                               pixmap,
                               &stride,
                               &size);
bo = drm_intel_bo_gem_create_from_prime(intel->bufmgr, fd, size);
    close(fd);
    intel_glamor_get_pixmap(pixmap)->bo = bo;
That last bit remembers the bo in some local memory so we don t have to do this more than once for each pixmap. glamorfdfrompixmap ends up calling eglCreateImageKHR followed by gbmbo_import and then a kernel ioctl to convert a prime handle into an fd. It s all quite round-about, but it does seem to work just fine. After I d gotten Glamor mostly working, I tried a few OpenGL applications and discovered flipping wasn t working. That turned out to have an unexpected consequence all full-screen applications would run flat-out, and not be limited to frame rate. Present recovers from a failed flip queue operation by immediately performing a CopyArea; not waiting for vblank. This needs to get fixed in Present by having it re-queued the CopyArea for the right time. What I did in the intel driver was to add a bunch more checks for tiling mode, pixmap stride and other things to catch pixmaps that were going to fail before the operation was queued and forcing them to fall back to CopyArea at the right time. The second adventure was with XRandR. Glamor has an API to fix up the screen pixmap for a new frame buffer, but that pulls the size of the frame buffer out of the pixmap instead of out of the screen. XRandR leaves the pixmap size set to the old screen size during this call; fixing that just meant getting the pixmap size set correctly before calling into glamor. I think glamor should get fixed to use the screen size rather than the pixmap size. Painting Root before Mode set The X server has generally done initialization in one order:
  1. Create root pixmap
  2. Set video modes
  3. Paint root window
Recently, we ve added a -background none option to the X server which causes it to set the root window background to none and have the driver fill in that pixmap with whatever contents were on the screen before the X server started. In a pre-Glamor world, that was done by hacking the video driver to copy the frame buffer console contents to the root pixmap as it was created. The trouble here is that the root pixmap is created long before the upper layers of the X server are ready for drawing, so you can t use the core rendering paths. Instead, UXA had kludges to call directly into the acceleration functions. What we really want though is to change the order of operations:
  1. Create root pixmap
  2. Paint root window
  3. Set video mode
That way, the normal root window painting operation will take care of getting the image ready before that pixmap is ever used for scanout. I can use regular core X rendering to get the original frame buffer contents into the root window, and even if we re not using -background none and are instead painting the root with some other pattern (like the root weave), I get that presented without an intervening black flash. That turned out to be really easy just delay the call to I830EnterVT (which sets the modes) until the server is actually running. That required one additional kludge I needed to tell the DIX level RandR functions about the new modes; the mode setting operation used during server init doesn t call up into RandR as RandR lists the current configuration after the screen has been initialized, which is when the modes used to be set. Calling xf86RandR12CreateScreenResources does the trick nicely. Getting the root window bits from fbcon, setting video modes and updating the RandR/Xinerama DIX info is now all done from the BlockHandler the first time it is called. Performance I ran the current glamor version of the intel driver with the master branch of the X server and there were not any huge differences since my last Glamor performance evaluation aside from GetImage. The reason is that UXA/Glamor never called Glamor s image functions, and the UXA GetImage is pretty slow. Using Mesa s image download turns out to have a huge performance benefit:
1. UXA/Glamor from April
2. Glamor from today
       1                 2                 Operation
------------   -------------------------   -------------------------
     50700.0        56300.0 (     1.110)   ShmGetImage 10x10 square 
     12600.0        26200.0 (     2.079)   ShmGetImage 100x100 square 
      1840.0         4250.0 (     2.310)   ShmGetImage 500x500 square 
      3290.0          202.0 (     0.061)   ShmGetImage XY 10x10 square 
        36.5          170.0 (     4.658)   ShmGetImage XY 100x100 square 
         1.5           56.4 (    37.600)   ShmGetImage XY 500x500 square 
     49800.0        50200.0 (     1.008)   GetImage 10x10 square 
      5690.0        19300.0 (     3.392)   GetImage 100x100 square 
       609.0         1360.0 (     2.233)   GetImage 500x500 square 
      3100.0          206.0 (     0.066)   GetImage XY 10x10 square 
        36.4          183.0 (     5.027)   GetImage XY 100x100 square 
         1.5           55.4 (    36.933)   GetImage XY 500x500 square
Running UXA from today the situation is even more dire; I suspect that enabling tiling has made CPU reads through the GTT even worse than before?
1: UXA today
2: Glamor today
       1                 2                 Operation
------------   -------------------------   -------------------------
     43200.0        56300.0 (     1.303)   ShmGetImage 10x10 square 
      2600.0        26200.0 (    10.077)   ShmGetImage 100x100 square 
       130.0         4250.0 (    32.692)   ShmGetImage 500x500 square 
      3260.0          202.0 (     0.062)   ShmGetImage XY 10x10 square 
        36.7          170.0 (     4.632)   ShmGetImage XY 100x100 square 
         1.5           56.4 (    37.600)   ShmGetImage XY 500x500 square 
     41700.0        50200.0 (     1.204)   GetImage 10x10 square 
      2520.0        19300.0 (     7.659)   GetImage 100x100 square 
       125.0         1360.0 (    10.880)   GetImage 500x500 square 
      3150.0          206.0 (     0.065)   GetImage XY 10x10 square 
        36.1          183.0 (     5.069)   GetImage XY 100x100 square 
         1.5           55.4 (    36.933)   GetImage XY 500x500 square
Of course, this is all just x11perf, which doesn t represent real applications at all well. However, there are applications which end up doing more GetImage than would seem reasonable, and it s nice to have this kind of speed up. Status I m running this on my crash box to get some performance numbers and continue testing it. I ll switch my desktop over when I feel a bit more comfortable with how it s working. But, I think it s feature complete at this point. Where s the Code As usual, the code is in my personal repository. It s on the glamor branch.
git://people.freedesktop.org/~keithp/xf86-video-intel  glamor

20 July 2014

Thomas Goirand: sysvinit not sending output to all consoles

I spent many, many hours trying to understand why I couldn t have both nova console-log showing me the output of the log, AND have the OpenStack dashboard (eg: horizon) console to work at the same time. Normally, this is done very easily, by passing multiple times the console= parameter to the Linux kernel as follow:
console=tty0 console=ttyS0,115200
But it never worked. Always, it s the last console= thing that was taken into account by sysvinit (or, shall I say, bootlogd). Spending hours trying to figure out what would be the correct kernel command to pass didn t help. Then this week-end, by the magic pure chance of being subscribed to the sysvinit bug reports, I have finally found out. We ve had this bug in Debian for more than 10 years: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=181756 And it has the patch! It just feels so lame that the issue has been pending since 2003, and with a patch since 2006, and nobody even tried to have it enter Debian. I tried the Wheezy patch in the above bug report, and then tadaaaaaa! I finally had both the nova console-log (eg: ttyS0) console output, and the interactive tty0 to work on my Debian cloud image. I have produced a fixed version of the sysvinit package for Wheezy, if anyone wants to try it: http://archive.gplhost.com/debian/pool/juno-backports/main/s/sysvinit/ This doesn t only affect only the cloud images use case. Let s say you have a server. If it s a modern server, probably you have IPMI 2.0 on it. While having access through the integrated KVM over IP may be nice, seeing the boot process through the serial console redirection is often a lot more snappy than the (often VNC based) video output, plus it wouldn t require Java. Too often, Java a requirement for these nasty IPMI web interface (that s the case for at least: Dell DRAC, Supermicro IMPI, and HP iLO). Well, it should now be possible to just use ipmitools to debug the server boot process or to go fix stuff in the single user interactive shell, AND keep the normal video output! :) But keeping this fix private doesn t help much. I would really love to get this fixed within Debian. So I have sent the patch (which needed a very small rebase) in the Git repository of sysvinit (see http://deb.li/3YxUD). I of course tested it in Sid too. Though I tested only under a Xen virtual machine, I see no reason why it would work there and not elsewhere. That being said, I would welcome more testing, given the high profile of sysvinit (everyone uses/needs it, and I wouldn t like to carry alone the unhappiness of having no boot log). Please do test it from the sysvinit git, before it s even uploaded to Sid. Also, these days, sysvinit gets often uploaded to Experimental first. It will probably also be the case for version 2.88dsf-56. If it works well and nobody complains about the patch, maybe it d be worth adding it to Wheezy as well (though that decision is of course left to the release team once the fix reaches Jessie).

18 July 2014

Mario Lang: Croudsourced accessibility: Self-made digital menus

Something straight out from the real world: Menu cards in restaurants are not nice to deal with if you are blind. It is an old problem we grow used to ignoring over time, but still something that can be quite nagging. There are a lot of psychological issues involved in this one. Of course, you can ask for the menu to be read out to you by the staff. While they usually do their best, you end up missing out on some things most of the time. First of all, depending on the current workload in the restaurant, the staff will usually try to cut some time and not read everything to you. What they usually do is to try to understand what type of meal you are interested in, and just read the choices from that category to you. While this can be considered a service in some situations (human preprocessing), there are situations were you will definitely miss a highlight on the menu that you would have liked to choose if you knew that it was there. And even if the staff decides to read the complete menu to you (which is rare), you are confronted with the 7-things-in-my-head-at-once problem. It is usually rather hard to decide amongst a list of more then 7 items, because our short-term memory is sort of limited. What the sighted restaurant goers do, is to skip back and forth between the available options, until they hit a decisive moment. True, that can take a while, but it is definitely a lot easier if you can perform "random access reads" to the list of choices yourself. However, if someone presents a substantial number of choices to you in a row, as sequential speech, you loose the random access ability. You either remember every choice from the beginning and do your choosing mentaully (if you do have extraordinary mental abilities), or you end up asking the staff to read previous items aloud again. This can work, but usually it doesn't. At some point, you do not want to bother the staff anymore, and you even start to feel stupid for asking again and again, while this is something totally normal to every sighted person, just that "they" do their "random access browsing" on their own, so "they" have no need to feel bad about how long it takes them to decide, minus the typical social pressure that arises after a a certain time for everyone, at least if you are dining in a group. In very rare cases, you happen to meet staff that is truly "awake", doing their best to not let you feel that they might be pressed on time, and really taking as much time as necessary to help you make the perfect decision. This is rare, but if it happens, it is almost a magical moment. One of these moments, where there are no "artificial" barriers between humans doing communcation. Anyway, I am drifting away. The perfect solution to this problem is to provide random access browsing of a restaurant menu with the help of digital devices. Trying to make braille menus available in all restaurants is a goal which is not realistically reachable. Menus go out of date, and need changing. And getting a physical braille copy updated and reprinted is considerably more involved as with digital media. Restaurant owners will also likely not see the benefit to rpvide a braille card for a very small circle of customers. With a digital online menu, that is a completely different story. These days, almost every blind person in at least my social circles owns an iOS (or similar) device. These devices have speech synthesis and web browsers. Of course, some restaurants especially in urban areas do already have a menu online. I have found them manually with google and friends sometimes in the past, which has already given me the ability to actually sit back, and really comfortably choose amongst the available offerings myself, without having to bother a human, and without having to feel bad about (ab)using their time. However, the case where a restaurant really has their menu online is rather rare still in the area where I am. And, it can be tedious to google for a restaurant website. Sometimes, the website itself is just marginally accessible, which makes it even more frustrating to get a relaxed dinner-experience. I have discovered a location-based solution for the restaurant-menu problem recently. Foursquare offers the ability to provide a direct link to the menu in a restaurant-entry. I figured, since all you need to do is write a single webpage where the (common) menu items are listed per restaurant, that I could begin to create restaurant menus for my favourite locations, on my own. Well, not quite, but almost. I will sometimes need help from others to get the menu digitized, but that's just a one-time piece of work I hopefully can outsource :-). Once the actual content is in my INBUX, I create a nice HTML page listing the menu in a rather speech-based browser friendly way. I have begun to do this today, with the menu of a restaurant just about 500 meters away from my apartment. Unterm goldenen Dachl now has a menu online, and the foursquare change request to publish the corresponsing URL is already pending. I don't fully understand how the Foursquare change review process works yet, but I hope the URL should be published in the upcoming days/weeks. I am using Foursquare because it is the backend of a rather popular mobile navigation App for blind people, called Blindsquare. Blindsquare lets you comfortably use Open Street Map and Foursquare data to get an overview of your surroundingds. If a food place has a menu entry listed in Foursquare, Blindsquare conveniently shows it to you and opens a browser if you tap it. So there is no need to actually search for the restaurant, you can just use the location based search of Blindsquare to discover the restaurant entry and its menu link directly from within Blindsquare. Actually, you could even find a restaurant by accident, and with a little luck, find the menu for it by location, without even knowing how the restaurant is called. Isn't that neat? Yeah, that's how it is supposed to work, that's as much independence as you can get. And, it is, as the title suggests, croudsourced accessibility. Becuase while it is nice if a restaurant owner cares to publish their menu themselves, if they haven't, you can do it yourself. Either as a user of assistive technologies, to scratch your own itch. Or as a friend of a person with a need for assistive technologies. Next time you go to lunch with your blind friend, consider making available the menu to them digitally in advance, instead of reading it. Other people will likely thank you for that, and you have actually achieved something today. And if you happne to put a menu online, make sure to submit a change request to Foursquare. Many blind people are using blindsquare these days, which makes it super-easy for them to discover the menu.

17 July 2014

Jonathan McDowell: On the state of Free VoIP

Every now and then I decide I'll try and sort out my VoIP setup. And then I give up. Today I tried again. I really didn't think I was aiming that high. I thought I'd start by making my email address work as a SIP address. Seems reasonable, right? I threw in the extra constraints of wanting some security (so TLS, not UDP) and a soft client that would work on my laptop (I have a Grandstream hardphone and would like an Android client as well, but I figure those are the easy cases while the "I have my laptop and I want to remain connected" case is a bit trickier). I had a suitable Internet connected VM, access to control my DNS fully (so I can do SRV records) and time to read whatever HOWTOs required. And oh my ghod the state of the art is appalling. Let's start with getting a SIP server up and running. I went with repro which seemed to be a reasonably well recommended SIP server to register against. And mostly getting it up and running and registering against it is fine. Until you try and make a TLS SIP call through it (to a sip5060.net test address). Problem the first; the StartCom free SSL certs are not suitable because they don't advertise TLS Client. So I switch to CACert. And then I get bitten by the whole question about whether the common name on the cert should be the server name, or the domain name on the SIP address (it's the domain name on the SIP address apparently, though that might make your SIP client complain). That gets the SIP side working. Of course RTP is harder. repro looks like it's doing the right thing. The audio never happens. I capitulate at this point, and install Lumicall on my phone. That registers correctly and I can call the sip:test.time@sip5060.net test number and hear the time. So the server is functioning, it's the client that's a problem. I try the following (Debian/testing): I'm bored at this point. Can I "dial" my debian.org SIP address from Lumicall? Of course not; I get a "Codecs incompatible" (SIP 488 Not Acceptable Here) response. I have no idea what that means. I seem to have all of the options on Lumicall enabled. Is it a NAT thing? A codec thing? Did I sacrifice the wrong colour of goat? At some point during this process I get a Skype call from some friends, which I answer. Up comes a video call with them, their newborn, perfect audio, and no hassle. I have a conversation with them that doesn't involve me cursing technology at all. And then I go back to fighting with SIP. Gunnar makes the comment about Skype creating a VoIP solution 10 years ago when none was to be found. I believe they're still the market leader. It just works. I'm running the Linux client, and they're maintaining it (a little behind the curve, but close enough), and it works for text chat, voice chat and video calls. I've spent half a day trying to get a Free equivalent working and failing. I need something that works behind NAT, because it's highly likely when I'm on the move that's going to be the case. I want something that lets my laptop be the client, because I don't want to rely on my mobile phone. I want my email address to also be my VoIP address. I want some security (hell, I'm not even insisting on SRTP, though I'd like to). And the state of the Open VoIP stack just continues to make me embarrassed. I haven't given up yet, but I'd appreciate some pointers. And Skype, if you're hiring, drop me a line. ;)

Daniel Pocock: MH17 and the elephant in the room

Just last week, air passengers were told of intrusive new checks on their electronic devices when flying. For years, passengers have also suffered bans on basic essentials like drinking water and excesses like the patting down of babies that even Jimmy Saville would find offensive. Of course, all this is being done for public safety. So if western leaders claim the safety and security of their citizens is really their number one priority, just how is it that a passenger aircraft can be flying through a war zone where two other planes were shot down this month? When it comes to aviation security, this really is the elephant in the room. The MH17 tragedy today demonstrates that terror always finds a way. It is almost like the terrorists can have their cake and eat it too: they force "free" countries to give up their freedoms and public decency and then they still knock the occasional plane out of the sky anyway. History in the making? It is 100 years since the assassination of Austrian Archduke Franz Ferdinand started World War I and just over 50 years since the Cuban missile crisis. Will this incident also achieve similar notoriety in history? The downing of MH17 may well have been a "mistake" but the casualties are real and very tragic indeed. I've flown with Malaysia Airlines many times, including the same route MH17 and feel a lot of sympathy for these people who have been affected.

Juliana Louback: Contribute a JSCommunicator Translation

To those who don t know, JSCommunicator is a SIP communication tool developed in HTML and JavaScript, currently supporting audio/video calls and SIP SIMPLE instant messaging. We ve recently added i18n internationalization support for English, Portuguese, Spanish, French and now Hebrew. We would like to have support for other languages and it s a pretty straightforward process to add a translation, so even if you do not have a tech background you can contribute! Setup First, create and account in Github. Github is a hosting service for git repositories and open source projects get to use Github for FREE! As a result, many open source project repositories are hosted on Github, including but not limited to JSCommunicator. Once you have created your git account, follow steps 1-4 in the section Setting up git listed in Github s setup tutorial. This will help you install git and configure anything you need. Fork Now fork the JSCommunicator repo. I really like the word fork . Great term. If this is your first fork, know that this will make a copy of the JSCommunicator repo and all its respective code to your Github account. Here s how we go about it:
  1. Sign in to Github
  2. Browse to https://github.com/JLouback/jscommunicator
  3. Somewhere around the upper left corner, you will see a button labeled Fork . Click that.
fork You should now have your own JSCommunicator repository on your Github account. On your main page select the tab Repositories . It should be the first on the list. The url will be https://github.com/YOUR-USERNAME/jscommunicator. My Github username is JLouback, so my jscommunicator repo can be found at https://github.com/JLouback/jscommunicator. Clone Now you should download all that code to your local machine so you can edit it. On your JSCommunicator page somewhere around the lower right corner you ll see a field entitled HTTPS clone URL: clone I don t know if this is correct, but you can just copy the URL on your browser too. I do that. Now on your terminal, navigate to the directory you d like to place your repository and enter
git clone https://github.com/YOUR-USERNAME/jscommunicator
The current directory should now have a folder named jscommunicator and in it is all the JSCommunicator code. Switch branches The JSCommunicator repo has a branch for the i18n support. A branch is basically a copy of the code that you can modify without changing the original code. It s great for adding new features, once everything is done and tested you can add the new feature to the original code. A more detailed explanation of branches can be found here. On the terminal, enter
cd jscommunicator
git branch -a
This will list all the branches in the repository, both local and remote. There should be a branch named remotes/origin/i18n-support . Switch to this branch by entering
git checkout --track origin/i18n-support
This will create a local branch named i18n-support with all the contents of the remote i18n-branch in https://github.com/opentelecoms-org/jscommunicator. Create a .properties file Now your jscommunicator directory will have some new files for the internationalization functionality. Among them you should find an INTERNATIONALIZATION_README file with instructions for adding a JSCommunicator translation. It s a less detailed version of this post, but it s worth a read in case I ve missed something here. Go to the directory internationalization . You ll see a few .properties files with different language codes. Choose the language of your preference for the translation base and make a copy of it in the internationalization directory. Your copy should be named Messages_LANGUAGECODE.properties . For example, the language code for german is de , so the german file should be named Messages_de.properties . If you re not sure about the code for your language, please check out this list of ISO 639-1 language codes so you re using the same code as (most) browsers. The .properties file is list of key-value pairs, a word or a few words joined by _ which is the key, the equals sign = then a word or phrase which is the value. Do not change anything to the left of the equals sign. Translate what is on the right of the equals sign. Modify available_languages.xml Go back to the root directory (jscommunicator) and open the file named available_languages.ruby . This .ruby has a series of language elements. At the end of the last </language> add a new language element with your language display name and code, copying the previous languge elements. You actually can put your language element anwhere, as long as it s after the ' and before the , as well as not cutting into any of the other language elements. Your display name should be the name of the language in its native language, for example for a french translation we ll put Fran ais (I think). And the code is the code we used for the .properties file. Here s how it should look:
<language>
    <display>Deutsch</display>
    <code>de</code>
</language>
Save your changes and voila! Test your work Once you ve made a .properties file, JSCommunicator will automatically load your translation if you ve set your browser preference to that language. If your browser preference is french, it will load the Messages_fr.properties. If your browser preference is a language we don t have a translation for (let s say german), JScommunicator will load the default Messages.properties file which is in english. The available_languages.ruby is used to build a language selection menu. If you add a language element without a corresponding .properties file, JSCommunicator will throw a JS error and load the default (english) translation. The same happens if you use different language codes in the .properties file name and the available_languages.ruby. The error won t disturb your use of JSCommunicator, it just won t load the language you selected. No harm, no foul. But if you want others to use your translation, do take care to do this correctly. If you read the INTERNATIONALIZATION_README, you will have seen that we need the jquery.i18n.properties.js file that s included in the .html pages. You can download that code here. Be sure to place it in the jscommunicator directory. This is only necessary if you want to see your addition at work, you don t need this file to contribute your translation. There are other 3rd party code dependencies to run JSCommunicator too as you may have noticed. Just creating the .properties file and adding a language element to available_languages.ruby, if done correctly, is enough to make a pull request. Commit Here s the part where we load all your local changes to your remote repository. In the terminal, navigate to the root directory (jscommunicator) and enter
git add internationalization/Messages_de.properties
git add available_languages.ruby
Of course, please replace Messages_de.properties with the name of your new .properties file. And mind you, we don t have a german translation yet! Then enter
git commit -m "German translation"
You can enter whatever you d like in the part, it s a good idea to explain what language you are adding. Now we can push the changes:
git push -u origin i18n-support
You should now see the changes you made in your Github account page at github.com/YOUR_USER/jscommunicator. The message you put in double quotations ( ) in the commit will be beside each modified file. This push also creates a new branch in your jscommunicator repository, now you will have a master and a i18n-support branch. Pull Now it s time to share your translation with everyone else if you feel so inclined. Your version of JSCommunicator has a new translation but not the official version. First navigate to you jscommunicator repo at github.com/YOUR_USER/jscommunicator and make sure you are on the i18n-branch as that will contain your changes. branches Click the green button directly to the left of the branch drop down menu shown above. This will create a pull request to add your new code to the official jscommunicator code. Once you click that button, you should see this: pull Make sure that the base repo (the first on the line above the green Create pull request button) is opentelecoms-org:i18n-support and not opentelecoms-org:master or another branch name. If it is, just click Edit on the right and you ll be able to select the branch from a dropdown like so: pullEdit If first repo is opentelecoms-org:i18n-support and the second is YOUR_USER:i18n-support and you ve verified (scroll down) that there are 2 files changed which are your new .properties file and the added element to the available_languages.xml, go ahead and press create pull request. It ll open a window with a text box you can add an explanation to with one final Create pull request button. Click that and you re done! You ve kindly contributed with a translation for JSCommunicator!

16 July 2014

Matthias Klumpp: AppStream 0.7 specification and library released

appstream-logoToday I am very happy to announce the release of AppStream 0.7, the second-largest release (judging by commit number) after 0.6. AppStream 0.7 brings many new features for the specification, adds lots of good stuff to libappstream, introduces a new libappstream-qt library for Qt developers and, as always, fixes some bugs. Unfortunately we broke the API/ABI of libappstream, so please adjust your code accordingly. Apart from that, any other changes are backwards-compatible. So, here is an overview of what s new in AppStream 0.7: Specification changes Distributors may now specify a new <languages/> tag in their distribution XML, providing information about the languages a component supports and the completion-percentage for the language. This allows software-centers to apply smart filtering on applications to highlight the ones which are available in the users native language. A new addon component type was added to represent software which is designed to be used together with a specific other application (think of a Firefox addon or GNOME-Shell extension). Software-center applications can group the addons together with their main application to provide an easy way for users to install additional functionality for existing applications. The <provides/> tag gained a new dbus item-type to expose D-Bus interface names the component provides to the outside world. This means in future it will be possible to search for components providing a specific dbus service:
$ appstream-index what-provides dbus org.freedesktop.PackageKit.desktop system
(if you are using the cli tool) A <developer_name/> tag was added to the generic component definition to define the name of the component developer in a human-readable form. Possible values are, for example The KDE Community , GNOME Developers or even the developer s full name. This value can be (optionally) translated and will be displayed in software-centers. An <update_contact/> tag was added to the specification, to provide a convenient way for distributors to reach upstream to talk about changes made to their metadata or issues with the latest software update. This tag was already used by some projects before, and has now been added to the official specification. Timestamps in <release/> tags must now be UNIX epochs, YYYYMMDD is no longer valid (fortunately, everyone is already using UNIX epochs). Last but not least, the <pkgname/> tag is now allowed multiple times per component. We still recommend to create metapackages according to the contents the upstream metadata describes and place the file there. However, in some cases defining one component to be in multiple packages is a short way to make metadata available correctly without excessive package-tuning (which can become difficult if a <provides/> tag needs to be satisfied). As small sidenote: The multiarch path in /usr/share/appdata is now deprecated, because we think that we can live without it (by shipping -data packages per library and using smarter AppStream metadata generators which take advantage of the ability to define multiple <pkgname/> tags) Documentation updates In general, the documentation of the specification has been reworked to be easier to understand and to include less duplication of information. We now use excessive crosslinking to show you the information you need in order to write metadata for your upstream project, or to implement a metadata generator for your distribution. Because the specification needs to define the allowed tags completely and contain as much information as possible, it is not very easy to digest for upstream authors who just want some metadata shipped quickly. In order to help them, we now have Quickstart pages in the documentation, which are rich of examples and contain the most important subset of information you need to write a good metadata file. These quickstart guides already exist for desktop-applications and addons, more will follow in future. We also have an explicit section dealing with the question How do I translate upstream metadata? now. More changes to the docs are planned for the next point releases. You can find the full project documentation at Freedesktop. AppStream GObject library and tools The libappstream library also received lots of changes. The most important one: We switched from using LGPL-3+ to LGPL-2.1+. People who know me know that I love the v3 license family of GPL licenses I like it for tivoization protection, it s explicit compatibility with some important other licenses and cosmetic details, like entities not loosing their right to use the software forever after a license violation. However, a LGPL-3+ library does not mix well with projects licensed under other open source licenses, mainly GPL-2-only projects. I want libappstream to be used by anyone without forcing the project to change its license. For some reason, using the library from proprietary code is easier than using it from a GPL-2-only open source project. The license change was also a popular request of people wanting to use the library, so I made the switch with 0.7. If you want to know more about the LGPL-3 issues, I recommend reading this blogpost by Nikos (GnuTLS). On the code-side, libappstream received a large pile of bugfixes and some internal restructuring. This makes the cache builder about 5% faster (depending on your system and the amount of metadata which needs to be processed) and prepares for future changes (e.g. I plan to obsolete PackageKit s desktop-file-database in the long term). The library also brings back support for legacy AppData files, which it can now read. However, appstream-validate will not validate these files (and kindly ask you to migrate to the new format). The appstream-index tool received some changes, making it s command-line interface a bit more modern. It is also possible now to place the Xapian cache at arbitrary locations, which is a nice feature for developers. Additionally, the testsuite got improved and should now work on systems which do not have metadata installed. Of course, libappstream also implements all features of the new 0.7 specification. With the 0.7 release, some symbols were removed which have been deprecated for a few releases, most notably as_component_get/set_idname, as_database_find_components_by_str, as_component_get/set_homepage and the pkgname property of AsComponent (which is now a string array and called pkgnames ). API level was bumped to 1. Appstream-Qt A Qt library to access AppStream data has been added. So if you want to use AppStream metadata in your Qt application, you can easily do that now without touching any GLib/GObject based code! Special thanks to Sune Vuorela for his nice rework of the Qt library! And that s it with the changes for now! Thanks to everyone who helped making 0.7 ready, being it feedback, contributions to the documentation, translation or coding. You can get the release tarballs at Freedesktop. Have fun!

Dirk Eddelbuettel: Introducing RcppParallel: Getting R and C++ to work (some more) in parallel

A common theme over the last few decades was that we could afford to simply sit back and let computer (hardware) engineers take care of increases in computing speed thanks to Moore's law. That same line of thought now frequently points out that we are getting closer and closer to the physical limits of what Moore's law can do for us. So the new best hope is (and has been) parallel processing. Even our smartphones have multiple cores, and most if not all retail PCs now possess two, four or more cores. Real computers, aka somewhat decent servers, can be had with 24, 32 or more cores as well, and all that is before we even consider GPU coprocessors or other upcoming changes. And sometimes our tasks are embarassingly simple as is the case with many data-parallel jobs: we can use higher-level operations such as those offered by the base R package parallel to spawn multiple processing tasks and gather the results. I covered all this in some detail in previous talks on High Performance Computing with R (and you can also consult the Task View on High Performance Computing with R which I edit). But sometimes we can't use data-parallel approaches. Hence we have to redo our algorithms. Which is really hard. R itself has been relying on the (fairly mature) OpenMP standard for some of its operations. Luke Tierney's (awesome) keynote in May at our (sixth) R/Finance conference mentioned some of the issues related to OpenMP. One which matters is that OpenMP works really well on Linux, and either not so well (Windows) or not at all (OS X, due the usual issue with the gcc/clang switch enforced by Applem but the good news is that the OpenMP toolchain is expected to make it to OS X is some more performant form "soon"). R is still expected to make wider use of OpenMP in future versions. Another tool which has been around for a few years, and which can be considered to be equally mature is the Intel Threaded Building Blocks library, or TBB. JJ recently started to wrap this up for use by R. The first approach resulted in a (now superseded, see below) package TBB. But hardware and OS issues bite once again, as the Intel TBB is not really building that well for the Windows toolchain used by R (and based on MinGW). (And yes, there are two more options. But Boost Threads requires linking which precludes easy use as e.g. via our BH package. And C++11 with its threads library (based on Boost Threads) is not yet as widely available as R and Rcpp which means that it is not a real deployment option yet.) Now, JJ, being as awesome as he is, went back to the drawing board and integrated a second threading toolkit: TinyThread++, a small header-only library without further dependencies. Not as feature-rich as Intel Threaded Building Blocks, but at least available everywhere. So a new package RcppParallel, so far only on GitHub, wraps around both TinyThread++ and Intel Threaded Building Blocks and offers a consistent interface available on all platforms used by R. Better still, JJ also authored several pieces demonstrating this new package for the Rcpp Gallery: All four are interesting and demonstrate different aspects of parallel computing via RcppParallel. But the last article is key. Based on a question by Jim Bullard, and then written with Jim, it shows how a particular matrix distance metric (which is missing from R) can be implemented in a serial manner in both R, and also via Rcpp. The key implementation, however, uses both Rcpp and RcppParallel and thereby achieves a truly impressive speed gain as the gains from using compiled code (via Rcpp) and from using a parallel algorithm (via RcppParallel) are multiplicative! Between JJ's and my four-core machines the gain was between 200 and 300 fold---which is rather considerable. For kicks, I also used a much bigger machine at work which came in at an even larger speed gain (but gains become clearly sublinear as the number of cores increases; there are however some tuning parameters). So these are exciting times. I am sure there will be lots more to come. For now, head over to the RcppParallel package and start playing. Further contributions to the Rcpp Gallery are not only welcome but strongly encouraged.

Vincent Sanders

It is no great secret that my colleagues at Collabora have been doing work with the Raspberry Pi Foundation.

My desk is very near Marco and I often see him working with the various Pi boards. Recently he obtained one of the new B+ units for testing and I thought it looked a little sad sat naked on his desk.

To remedy this bare board problem I designed and built a laser cut a case for him and now the B+ has been publicly released I can make the design freely available.

The design is completely original though is inspired by several other plastic "clip" type designs I have seen. Originally I created and debugged the case design for my parallella though tweaking it for the Pi was pretty easy.

The design is under a CC attribution licence and I ought to say that my employer is in no way responsible for this, its all my own fault.

Wouter Verhelst: Reprepro for RPM

Dear lazyweb, reprepro is a great tool. I hand it some configuration and a bunch of packages, and it creates the necessary directory structure, moves the packages to the right location, and generates a (signed) Debian package repository. Obviously it would be possible to all that reprepro does by hand by calling things like cp and dpkg-scanpackages and gpg and other things by hand but it's easy to forget a step when doing so, and having a tool that just does things for me is wonderful. The fact that it does so only on request (i.e., when I know something has changed, rather than "once every so often") is also quite useful. At work, I currently need to maintain a bunch of package repositories. The Debian package archives there are maintained with reprepro, but I currently maintain the RPM archives pretty much by hand: create the correct directories, copy the right files to the right places, run createrepo over the correct directories (and in the case of the OpenSUSE repository, also run gpg), and a bunch of other things specific to our local installation. As if to prove my above point, apparently I forgot to do a few things there, meaning, some of the RPM repositories didn't actually work correctly, and my testing didn't catch on. Which makes me wonder how RPM package repositories are usually maintained. When one needs to maintain just a bunch of packages for a number of servers, well, running createrepo manually isn't too much of a problem. When it gets beyond own systems, however, and when you need to support multiple builds for multiple versions of multiple distributions, having to maintain all those repositories by hand is probably not the best idea. So, dear lazyweb: how do large RPM repositories maintain state of the packages, the distributions they belong to, and similar things? Please don't say "custom scripts" ;-)

Juliana Louback: Become an Open-source Contributor Video Conference

A couple weeks ago I was contacted by Yehuda Korotkin through one of the Debian mailing lists. Yehuda is a tech professor at one of Israel s leading colleges for women. He proposed a video conference to present the open source community to his class and explain how they can contribute to open source projects. I volunteered to participate and last Monday (July 14th 2014) we had our virtual meeting, which I hope was the first of many. Shauna G. also participated from Boston, making it a Israel - USA - Brazil meetup. Pretty impressive, eh? The conference was hosted on Google Hangouts, the video is available for viewing here. Run time is around 2.5 hours so to save you time I ve kindly posted a summary of the conference! To be frank, I look ghastly half the time so I d much prefer that you read this post. First there was a Q&A period which was more of a discussion panel, followed by a hands-on session where the girls made their first contribution to an open source project. Here s the gist of it: Q&A Hands on For the practical part of our conference, Yehuda s students contributed to Jscommunicator s Issue #5 which requests i18n support for major languages. JSCommunicator is a SIP communication tool developed in HTML and JavaScript, currently supporting audio/video calls and SIP SIMPLE instant messaging. First we set up Github in their local machines and forked the Jscommunicator repo. We learned the basic git commands needed to push changes to their remote repositories and to create a pull requests. This is explained in further detail here. At the end of the conference with nearly an hour overtime due to my tendency to be overly verbose, Yehuda s class made their first contribution with a Hebrew translation for Jscommunicator! All in all, it was great to virtually meet Yehuda and his epic students who all seemed very clever engineers. I m looking forward to our next meetup once I gain some time-management skills.

13 July 2014

Laura Arjona: New GPG Key!

Achievement unlocked: I have a new GPG key:

0xF22674467E4AF4A3

pub 4096R/7E4AF4A3 2014-07-13 [caduca: 2016-07-12]
Fingerprint = 445E 3AD0 3690 3F47 E19B 37B2 F226 7446 7E4A F4A3
uid Laura Arjona Reina <laura.arjona@upm.es>
uid Laura Arjona Reina <larjona@fsfe.org>
uid Laura Arjona Reina <larjona99@gmail.com>
sub 3072R/CC706B74 2014-07-13 [expires: 2016-07-12]
sub 3072R/7E51465F 2014-07-13 [expires: 2016-07-12]
sub 4096R/74C23D6E 2014-07-13 [expires: 2016-07-12]

The master key is 4096 bit, stored in a safe place, and 2 subkeys 3072 bit, stored in an FSFE Smartcard (I cannot store 4096 keys there).

I have carefully used the FSFE SmartCard Howto and Creating the perfect GPG keypair by Alex Cabal for strenghtening hash preferences and creating revocation certificate.

It seems everything works as intended. Passphrase is strong and this time I will not forget it. As first celebration, 1/2 lt icecream is waiting for me after dinner :)

People knowing me and around Madrid, please send me an encrypted mail as test or normal communication, and ping me to meet and sign keys :) One more step towards involvement in Debian and free software, controlling my digital life and communications, and becoming familiar with these technologies so I can teach them to my son as the natural way . Yay!
Filed under: My experiences and opinion, Tools Tagged: Contributing to libre software, Debian, encryption, English, Free Software, gpg, libre software, Moving into free software, mswl-cases

Russ Allbery: Review: Neptune's Brood

Review: Neptune's Brood, by Charles Stross
Series: Freyaverse #2
Publisher: Ace
Copyright: July 2013
ISBN: 1-101-62453-1
Format: Kindle
Pages: 325
Neptune's Brood is set in the same universe as Saturn's Children, but I wouldn't call it a sequel. It takes place considerably later, after substantial expansion of the robot civilization to the stars, and features entirely different characters (or, if there was overlap, I didn't notice). It also represents a significant shift in tone: while Saturn's Children is clearly a Heinlein pastiche and parody, Neptune's Brood takes its space opera more seriously. There is some situational humor assault auditors, for example but this book is played mostly straight, and I detected little or no Heinlein. This is Stross fleshing out his own space opera concept. This being Stross, that concept is not exactly conventional. This is a space opera about economics. Specifically, it's a space opera about interstellar economics, a debt pyramid, and a very interesting remapping of the continual growth requirements of capitalism to the outward expansion of colonization. The first-person protagonist comes from a "family" (as in Saturn's Children, the concept exists but involves rather more aggressive control of the instantiated "children") of bankers, but she is a forensic accountant and historian who specializes in analysis of financial scams. As you might expect, this is a significant clue about the plot. Neptune's Brood opens with Krina in search of her sister. She is supposed to be an itinerant scholar, moving throughout colonized space to spend some time with various scattered sisters, spreading knowledge and expanding her own. But it's clear from the start of the book that something else is going on, even before an assassin with Krina's face appears on her trail. Unfortunately, it takes roughly a third of the book to learn just what is happening beneath the surface, and most of that time is spent in a pointless interlude on a flying cathedral run by religious fanatics. The religion is a callback to Saturn's Children: robots who are trying to spread original humanity (the Fragiles) to the stars. This mostly doesn't go well, and is going particularly poorly for the ship that Krina works for passage on. But this is a brief gag that I thought went on much too long. The plot happens to Krina for this first section of the book rather than the other way around, little of lasting significance other than some character introductions occurs, and the Church itself, while playing a minor role in the later plot, is not worth the amount of attention that it gets. The best parts of the early book are the interludes in which Krina explains major world concepts to the reader. These are absolutely blatant infodumping, and I'm not sure how Stross gets away with them, but somehow he does, at least for me. They remind me of some of his blog posts, except tighter and fit into an interesting larger structure. Thankfully, once Krina finally arrives on Shin-Tethys, the plot improves considerably. There was a specific moment for me when the book became interesting: when Krina finds her sib's quarters in Shin-Tethys and analyzes what she finds there. It's the first significant thing in the book that she does rather than have done to her or thrust upon her, and she's a much better character when she's making decisions. This is also about the point where Stross starts fully explaining slow money, which is key to both the economics and the plot, and the plot starts to unwind its various mysteries and identify the motives of the players. Even then, Krina suffers from a lack of agency. Only at rare intervals does she get a chance to affect the story. Most of what she did of relevance to this book she did in the past, and while those descriptions of the backstory are interesting, they don't entirely make up for a passive protagonist. Thankfully, the other characters are varied and interesting enough, and the political machinations and cascading revelations captivating enough, that the last part of the book was very satisfying even with Krina on for the ride. This is a Stross novel, so it's full of two-dollar technical words mixed with technobabble. However, it shares with Saturn's Children the recasting of robots as the norm and fleshy humans as the exception, which means much of the technobabble is a straight substitution for our normal babble about meaty bodies and often works as an alienation technique. That makes it a bit more tolerable for me, although I still wished Stross would turn down the manic vocabulary in places. This bothers some people more than others; if you had no trouble with Accelerando, Neptune's Brood will pose no problems. I don't think the first section of this book was successful, but I liked the rest well enough to recommend it. If you like your space opera with a heavy dose of economics, a realistic attitude towards deep space exploration without faster-than-light technology, and a realistic perspective on the hostility of alien planets to Earth life, Neptune's Brood is a good choice. And any book that quotes David Graeber's Debt, and whose author has clearly paid attention to its contents, wins bonus points from me. Rating: 8 out of 10

11 July 2014

Russell Coker: Improving Computer Reliability

In a comment on my post about Taxing Inferior Products [1] Ben pointed out that most crashes are due to software bugs. Both Ben and I work on the Debian project and have had significant experience of software causing system crashes for Debian users. But I still think that the widespread adoption of ECC RAM is a good first step towards improving the reliability of the computing infrastructure. Currently when software developers receive bug reports they always wonder whether the bug was caused by defective hardware. So when bugs can t be reproduced (or can t be reproduced in a way that matches the bug report) they often get put in a list of random crash reports and no further attention is paid to them. When a system has ECC RAM and a filesystem that uses checksums for all data and metadata we can have greater confidence that random bugs aren t due to hardware problems. For example if a user reports a file corruption bug they can t repeat that occurred when using the Ext3 filesystem on a typical desktop PC I ll wonder about the reliability of storage and RAM in their system. If however the same bug report came from someone who had ECC RAM and used the ZFS filesystem then I would be more likely to consider it a software bug. The current situation is that every part of a typical PC is unreliable. When a bug can be attributed to one of several pieces of hardware, the OS kernel and even malware (in the case of MS Windows) it s hard to know where to start in tracking down a bug. Most users have given up and accepted that crashing periodically is just what computers do. Even experienced Linux users sometimes give up on trying to track down bugs properly because it s sometimes very difficult to file a good bug report. For the typical computer user (who doesn t have the power that a skilled Linux user has) it s much worse, filing a bug report seems about as useful as praying. One of the features of ECC RAM is that the motherboard can inform the user (either at boot time, after a NMI reboot, or through system diagnostics) of the problem so it can be fixed. A feature of filesystems such as ZFS and BTRFS is that they can inform the user of drive corruption problems, sometimes before any data is lost. My recommendation of BTRFS in regard to system integrity does have a significant caveat, currently the system reliability decrease due to crashes outweighs the reliability increase due to checksums at this time. This isn t all bad because at least when BTRFS crashes you know what the problem is, and BTRFS is rapidly improving in this regard. When I discuss BTRFS in posts like this one I m considering the theoretical issues related to the design not the practical issues of software bugs. That said I ve twice had a BTRFS filesystem seriously corrupted by a faulty DIMM on a system without ECC RAM.

8 July 2014

Wouter Verhelst: HP printers require systemd, apparently

printer-driver-postscript-hp Depends: hplip
hplip Depends: policykit-1
policykit-1 Depends: libpam-systemd
libpam-systemd Depends: systemd (= 204-14)
Since the last in the above is a versioned dependency, that means you can't use systemd-shim to satisfy this dependency. I do think we should migrate to systemd. However, it's unfortunate that this change is being rushed like this. I want to migrate my personal laptop to systemd but not before I have the time to deal with any fallout that might result, and to make sure I can properly migrate my configuration. Workaround (for now): hold policykit-1 at 0.105-3 rather than have it upgrade to 0.105-6. That version doesn't have a dependency on libpam-systemd. Off-hand questions: All this feels like another case of overengineering, like most of the *Kit thingies. Update: so the systemd package doesn't actually cause systemd to be run, there are other packages that do that, and systemd-shim can be installed. I misread things. Of course, the package name is somewhat confusing... but that's no excuse.

Matthew Palmer: Doing Password Complexity Wrong

I just made an account on yet another web service. On the suggestion of my password manager, I attempted to use the password W:9[$X*F . It was rejected because Password must contain at least one non-alphabet character, one lowercase letter, one uppercase letter . OK, how about Passw0rd ? Yep, that s fine. Anyone want to guess which of those two passwords is going to fall victim to a brute-force attack first? Go on, don t be shy, take a wild shot in the dark!

Joey Hess: laptop death

So I was at Ocracoke island, camping with family, and I brought my laptop along as I've done probably half a dozen times before. An enormous thuderstorm came up. It rained for 8 hours and thundered for 3 of those. Some lightning cracks quite close by as we crouched in the food tent, our feet up off the increasingly wet ground "just in case". The campground flooded. Luckily we were camped in the dunes and tents mostly avoided being flooded with 2-3 inches of water. (That was just the warmup; a hurricane hit a week after we left.) My laptop was in my tent when this started, and I got soaked to the skin just running over there and throwing it up on the thermarest to keep it out of any flooding and away from any drips. It seemed ok, so best not to try to move it to the car in that downpour. Next time I checked, it turned out the top vent of the tent was slightly open and dripping. The laptop bag was damp. But inside it seemed ok. Rain had slackened to just heavy, so I ran it down to the car. Laptop appeared barely damp, but it was hard to tell as I had quite forgotten what "dry" was. Turned it on for 10 seconds to check the time. It was 7:30 and we still had to cook dinner in this mess. Transferred it to a dry bag. (By the way, in some situations, discovering you have a single dry towel you didn't know you had is the best gift in the world!) Next morning, the laptop was dead. When powered on, the fan came on full, the screen stayed black, and after a few seconds it turned itself back off. I need this for work, so it was a crash priority to get it fixed or a replacement. Before I even got home, I had logged onto Lenovo's website to check warantee status and found 2 things:
  1. They needed some number from a sticker on the bottom of my laptop. Which was no longer there.
  2. The process required some stange login on an entirely different IBM website.
At this point, I had a premonition of how the beuracracy would go. Reading Sesse's Blehnovo, I see I was right. I didn't even try. I ordered a replacement with priority shipping. When I got home, I pulled the laptop apart to try to debug it. I still don't know what's wrong with it. The SSD may be damaged; it seems to cause anything I put it into to fail to work. New laptop arrived in 2 days. Since this model is now a year old, it was a few hundred dollars cheaper this time around. And now I have an extra power supply, and a replacment keyboard, and a replacement fan etc. And I've escaped the dead USB port and broken rocker switch of the old laptop too. The only weird thing is that, while my old laptop had no problem with my Toshiba passport USB drive, this new one refuses to recognize it unless I plug it into a USB 1.0 hub. Oh well..
Update: Ocracode for this trip: OBX1.1 P6 L7 SA3d+++b+c++ U2(rearended,laptop death) T4f2b1 R2T Bb+m++++n++
F+++u++ SC+s-g6 H+f0i3 V+++s++m0 E++r+

6 July 2014

Dominique Dumont: Status and next step on lcdproc automatic configuration upgrade with Perl and Config::Model

Back in March, I uploaded a Debian s version of lcdproc with a unique feature: user and maintainer configurations are merged during package upgrade: user customizations and developers enhancements are both preserved in the new configuration file. (See this blog for more details). This avoids tedious edition of the configuration LCDd.conf file after every upgrade of lcdproc package. At the beginning of June, a new version of lcdproc (0.5.7-1) was uploaded. This triggered another round of automatic upgrades on user s systems. According to the popcon rise of libconfig-model-lcdproc-perl, about 100 people have upgraded lcdproc on their system. Since automatic upgrade has an opt-out feature, one cannot say for sure that 100 people are actually using automatic upgrade, but I bet a fair portion are them are. So far, only one people has complained: a bug report was filed about the many dependencies brought by libconfig-model-lcdproc-perl. The next challenge for lcdproc configuration upgrade is brought by a bug reported on Ubuntu: the device file provided by imon kernel module is a moving target: The device file created by the kernel can be /dev/lcd0 or /dev/lcd1 or even /dev/lcd2. Static configuration files and moving target don t mix well. The obvious solution is to provide a udev rule so that a symbolic link is created from a fixed location (/dev/lcd-imon) to the moving target. Once the udev rule is installed, the user only has to update LCDd.conf file to use the symlink as imon device file and we re done. But, wait The whole point of automatic configuration upgrade is to spare the user this kind of trouble: the upgrade must be completely automatic. Moreover, the upgrade must work in all cases: whether udev is available (Linux) or not. If udev is not available, the value present in the configuration file must be preserved. To know whether udev is available, the upgrade tool (aka cme) will check whether the file provided by udev (/dev/lcd-imon) is present or not. This will be done by lcdproc postinst script (which is run automatically at the end of lcdproc upgrade). Which means that the new udev rule must also be
activated in the postinst script before the upgrade is done. In other words, the next version of lcdproc (0.5.7-2) will: In the lcdproc configuration model installed by libconfig-model-lcdproc-perl, the imon device parameter is enhanced so that running cme check lcdproc or cme migrate lcdproc issues a warning if /dev/lcd-imon exists and if imon driver is not configured to use it. This way, the next installation of lcdproc will deliver a fix for imon and cme will fix user s configuration file without requiring user input. The last point is admittedly bad marketing as users will not be aware of the magic performed by Config::Model Oh well In the previous section, I ve briefly mentioned that imon_device parameter is enhanced in lcdproc configuration model. If you re not already bored, let s lift the hood and see what kind of enhancements was added. Let s peek in lcdproc configuration file, LCDd.conf file which is used to generate lcdproc configuration model. You may remember that the formal description of all LCDd.conf parameters and their properties is generated from LCDd.conf to provide lcdproc configuration model. The comments in LCDd.conf follow a convention so that most properties of the parameters can be extracted from the comments. In the example below, the comments show that NewFirmware is a boolean value expressed as yes or no, the latter being the default :
# Set the firmware version (New means >= 2.0) [default: no; legal: yes, no]
NewFirmware=no
Back to the moving target. In LCDd.conf, imon device file parameter is declared this way:
# Select the output device to use
Device=/dev/lcd0
This means that device is a string where the default value is /dev/lcd0. Which is wrong once the special udev rule provided with Debian packages is activated. With this rule, the default value must be /dev/lcd-imon. To fix this problem, a special comment is added in the Debian version of LCDd.conf to tune further the properties of the device parameter:
# select the device to use
#  %
#   default~
#   compute
#     use_eval=1
#     formula="my $l = '/dev/lcd-imon'; -e $l ? $l : '/dev/lcd0';"
#     allow_override=1 -
#   warn_if:not_lcd_imon
#     code="my $l = '/dev/lcd-imon';defined $_ and -e $l and $_ ne $l ;"
#     msg="imon device does not use /dev/lcd-imon link."
#     fix="$_ = undef;"
#   warn_unless:found_device_file
#     code="defined $_ ? -e : 1"
#     msg="missing imon device file"
#     fix="$_ = undef;"
#   - % 
Device=/dev/lcd0
This special comment between % and % follows the syntax of Config::Model::Loader. A small configuration model is declared there to enhance the model generated from LCDd.conf file. Here are the main parts: In both warn_unless and warn_if parts, the fix code snippet is run when by the command cme fix lcdproc and is used to repair the warning condition. In this case, the fix consists in resetting the device configuration value so the computed value above can be used. cme fix lcdproc is triggered during package post install script installed by dh_cme_upgrade. Come to think of it, generating a configuration model from a configuration file can probably be applied to other projects: for instance, php.ini and kdmrc are also shipped with detailed comments. May be I should make a more generic model generator from the example used to generate lcdproc model Well, I will do it if people show interest. Not in the form yeah, that would be cool , but in the form, yes, I will use your work to generate a configuration model for project [...] . I ll let you fill the blank ;-)
Tagged: Config::Model, configuration, debian, lcdproc, Perl, upgrade

Next.

Previous.