Search Results: "andy"

21 August 2012

Russell Coker: Mirror Displays

Image of a Macbook Pro with a Retina display showing how badly it reflects When I previously wrote about the Retina display in the new Macbook Pro I was so excited that I forgot to even check whether the display reflects light [1]. A TFT display with a mirrored surface apparently permits more intense color which is generally a good thing. It also makes it easier to clean the surface which is really important for phones and tablets. The down-side of a mirrored surface on a display is that it can reflect whatever else is in the area. This generally isn t a problem in an office as you can usually adjust the angle of the monitor and the background lighting to avoid the worst problems. It s also not a serious problem for a hand-held device as it s usually easy to hold it at an angle such that you don t have light from anything particularly bright reflecting. But my experience of laptop use includes using them anywhere at any time. I ve done a lot of coding on all forms of public transport in all weather conditions. Doing that with a Thinkpad which has a matte surface on it s screen is often difficult but almost always possible. Doing that on a system with a mirrored display really isn t possible. The above photo of a 15 Macbook Pro model MD103X/A was taken at a Myer store which was specifically designed to make the computers look their best. The overall lighting wasn t particularly bright so that the background didn t reflect too much and the individual lights were diffuse to avoid dazzling point reflections. But even so the lights can be clearly seen. Note that the photo was taken with a Samsung Galaxy S, far from the best possible camera. If I was buying a laptop that would only ever be used in the more northern parts of Europe or if I was buying a laptop to use only at home and at the office then I might consider a mirror display. But as I mostly use my laptop in mainland Australia including trips to tropical parts of Australia and I use it in all manner of locations a mirror display isn t going to work for me. This isn t necessarily a bad decision by Apple designers. My observation of Macbook use includes lots of people using them only in offices and homes. Of the serious geeks who describe their laptop as My Precious hardly anyone has a Macbook while Thinkpads seem quite popular in that market segment. I don t think that it s just the matte screen that attracts serious geeks to the Thinkpad, but it does seem like part of a series of design decisions (which include the past tradition of supporting hard drive removal without tools and the option of a second hard drive for RAID-1) that make Thinkpads more suitable for geeks than Macbooks. While the new tradition in Apple design of gluing things together so they can never be repaired, recycled, or even have their battery replaced seems part of a pattern that goes against geek use. Even when Apple products are technically superior in some ways their catering to the less technical buyers makes them unsuitable to people like me. Maybe the ability to use a Macbook as a shaving mirror could be handy, but I d rather grow a beard and use a Thinkpad. Related posts:
  1. The Retina Display Last night I played with an Apple Macbook Pro with...
  2. are Thinkpads meant to run 24*7? My Thinkpad has started to run hot recently. If I...
  3. presentations and background color In response to my last post about using laptops for...

8 August 2012

Aigars Mahinovs: Fun with the new camera

Triple hit! So, I finally received my Canon 650D and Canon EF 40mm STM lens. I am very happy with the purchase. The STM lens is very, very compact, it is barely bigger than the lens cap that comes with the camera. The focusing is fast enough and it is very smooth and silent enough not to be audible in video mode. The largest bonus of the 650D compared to my old 550D is the articulated display. It has come in handy many times already. It is especially useful with the STM lens, live view and touch to focus function. Very useful to focus on something even when the camera is at an weird angle. And a couple days ago I got to really use the new camera capturing the shot above (an others in that set). It turns out capturing cool lightning shots is quite easy: It is basically the same as shooting photos of a fireworks display, with the exception that lightning is not so bright sometimes and that there is a high chance of rain in a thunderstorm ;) P.S. Got over 5k photos in Flickr now with this set, and closing in for 500k views. That feels cool :)

28 July 2012

Luke Faraone: Simple two-factor authentication



Do you want to allow "Rebooting of Cluster 19" as Jane Doe, Foo Corp? Yes/NoI've been thinking a lot recently about information security and passwords. It's widely agreed that two-factor authentication, which combines something-you-know (a password) with something you have (eg. a smart card or some one-time-password) is superior to using a password by itself. Most solutions I've seen, short of client-side certificates, both require a trusted third party and provide little auditability.

I'm working on a project which attempts to address those problems. Basically, the idea is that you attempt to perform an action on a website, like logging in, moving some money between accounts, or restarting a cluster. Then, your mobile device ((iPhone used in the graphic to the right because a SVG was handy. Image adapted from work by Virgile Pypaert [CC-BY-SA-3.0], via Wikimedia Commons)) (Android, iPhone, or some custom hardware) starts buzzing, displaying the text of the action. When you approve or deny the action, your device uses a locally stored private key to sign the text of the approval and send it back to the server. The server checks that you approved the same transaction number and text, then allows the action. The server can store the approval notice to allow auditors later to determine that somebody in possession of the private key signed the action request.

So far, I've written a small Django server application, and a Python CLI device and client. They're not yet ready for a public release; I need to write up installation instructions and test it a bit more, but I'll publish them soon on Launchpad. On the usability side of things, I plan to conduct some trials in January comparing it to things like Google Authenticator.

This should be less susceptible to MITM attacks than one-time-passwords, since you can authenticate specific transactions, rather than entire sessions.

On the other hand, I'm no security researcher. Dear lazyweb, are there any problems with the above approach? Also, if I'm going to release this (as I plan to once I get something working), I need a name. Ideas?

27 July 2012

Keith Packard: sandybridge-caching

Playing around with LLC controls on Sandybridge today. What I wanted to know was how caching backbuffers affects application performance. The reason for this question is that when the driver uses page flipping to display a new frame, we force the new scanout buffer to be uncached because the display hardware only scans out from main memory. However, when we go back to using this buffer as the next back-buffer, we don t turn the caching bits back on as that requires writing GTT entries to change caching modes. This means that swapping via page-flipping and swapping via copying has a large change in main memory access patterns during rendering page flipping applications use an uncached back buffer while copying applications use a cached back buffer. What effect does the back-buffer caching-mode have on application performance? The changes In the kernel, I have three modes 1) Existing code. Set buffer objects to cache mode none when preparing them as scanout buffers. The first time the application flips to a back buffer, caching is disabled on that buffer forever 2) Flip back and forth. Set buffers to cache mode none when used as scanout. When that stops, flip them back to cache mode llc. This involves a whole lot of clflushing and GTT rewriting, obviously. 3) Flush to memory, but don t disable caching. In this mode, the buffer gets flushed (from both CPU and GPU) before being used as a scanout, but the cache mode isn t changed. Benchmarking with glxgears The first test was pretty simple I hacked up glxgears (yes, glxgears is not a benchmark) to redraw the same frame multiple times (including the clear) per swap to provide a scalable amount of work per swap. No pixel operations, so this is just writing to the back buffer. Thes numbers are the median of 5 runs, except for those obviously limited by the refresh rate.
RepsExistingFlipFlush
10606060
20606060
4042.07034.11337.130
8021.25917.95718.658
Note the linear scaling over reps (once you get below refresh). I think this means that the swap overhead is in the noise, including flushing and rewriting GTT entries. Given that my screen is (yuck) 1366x768 pixels, or just over 4MB, and my total LLC is 4MB, it s seems like the performance changes here are likely caused by the rendering ejecting every other object from the cache. What s wrong with benchmarking with glxgears? Of course, glxgears is not a benchmark, and I realized while collecting this data that, in particular, it never reads from the backbuffer, and most pixels are only written once by the clear operation at the start of the rendering code. So, it forces everything else out of the cache at the start of each drawing cycle, but doesn t stick anything useful into the cache. Benchmarking nexuiz, a (more) real application For the next test, I ran nexuiz demo1.dem in all three modes. Numbers out of that weren t nearly as stable as glxgears, and I m afraid I didn t write all of them down, but I got numbers between 45 and 50 frames per second in all three modes. None significantly faster, none significantly slower. So, for these two tests, caching has no positive effect on overall rendering performance. Obviously, I need to collect data from more applications to see if the effect is general. I sure hope so, because the alternative will be to find some heuristic to direct when to enable caching. The patch This is on top of kernel version 3.5.
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index b0b676a..f69fba8 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1335,6 +1335,10 @@ int __must_check
 i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
                     u32 alignment,
                     struct intel_ring_buffer *pipelined);
+
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj);
+
 int i915_gem_attach_phys_object(struct drm_device *dev,
                struct drm_i915_gem_object *obj,
                int id,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 288d7b8..b606bd2 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2851,6 +2851,9 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
    return 0;
  
+#define JUST_FLUSH 0
+#define FLIP_CACHING   0
+
 /*
  * Prepare buffer for display plane (scanout, cursors, etc).
  * Can be called from an uninterruptible phase (modesetting) and allows
@@ -2883,7 +2886,11 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
     * of uncaching, which would allow us to flush all the LLC-cached data
     * with that bit in the PTE to main memory with just one PIPE_CONTROL.
     */
+#if JUST_FLUSH
+   ret = i915_gem_object_finish_gpu(obj);
+#else
    ret = i915_gem_object_set_cache_level(obj, I915_CACHE_NONE);
+#endif
    if (ret)
        return ret;
@@ -2913,6 +2920,16 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
    return 0;
  
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj)
+ 
+   i915_gem_object_unpin(obj);
+#if FLIP_CACHING
+   if (HAS_LLC(obj->base.dev))
+       i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+#endif
+ 
+
 int
 i915_gem_object_finish_gpu(struct drm_i915_gem_object *obj)
  
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a8538ac..76a2012 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -1821,7 +1821,7 @@ err_interruptible:
 void intel_unpin_fb_obj(struct drm_i915_gem_object *obj)
  
    i915_gem_object_unpin_fence(obj);
-   i915_gem_object_unpin(obj);
+   i915_gem_object_unpin_from_display_plane(obj);
  
 static int i9xx_update_plane(struct drm_crtc *crtc, struct drm_framebuffer *fb,

Paul Tagliamonte: ERMAHGIT!

After seeing this post on reddit, I ported the OP s code to ERMAHGIT after following the following (simple) instructions:
sudo ./setup.py install
then, head over to a friendly git repo, and do the following:
git config core.editor /usr/local/bin/ermahgit
Anyway, here s a few lines from some random git commits I have handy:
Added chamber to vote page heading
ERDDERD CHERMBER TO VERTE PERGE HERERDERNG
Fixing the import I just broke
FERXERNG THE ERMPERT I JERST BRERKER
Bugfix for non-ascii chars using encode
BERGFERX FER NERN-ERSCERI CHERS ERSERNG ERNCERDER
Add this sorry excuse for a manpage.
ERDD THERS SERRER ERXCERSE FER A MAHNPERGER.
Please, never, ever use this :)

9 July 2012

Julien Danjou: Logitech K750 keyboard and Unifying Receiver Linux support

A year ago, I bought a Logitech Wireless Solar Keyboard K750. I'm particularly picky on keyboards, but this one is good. It has an incredible useful feature: while being wireless, it has no need for disposable or rechargeable batteries, it uses solar power! Logitech Wireless Solar Keyboard K750 My problem is that there's obviously no way to know the battery status from Linux, the provided application only working on Windows. And one dark night, while fragging on QuakeLive, my keyboard stopped working: it had no battery left. This activity being quite energy consuming, it emptied the whole battery. Someone should write code to get the battery status and light meter from Linux: challenge accepted! How the keyboard works Logitech Unifying Receiver This keyboard, like many of the new wireless devices from Logitech, uses the Unifying interface. It's an USB receiver that can be attached up to 6 differents devices (mouse, keyboards ). On old Linux kernel, the Unifying receiver is recognized as only one keyboard and/or one mouse device. Recently, a driver called hid-logitech-dj has been added to the Linux kernel. With this driver, each device attached to the receiver is recognized as one different device. What the Logitech application does Logitech Solar App The Logitech application under Windows works that way: you launch it, and it displays the battery charge level. On the keyboard, there's a special "light" button (up right). When pressed, a LED will light up on the keyboard: green if the keyboard is receiving enough light and is charging, red if the keyboard does not receive enough light and is therefore discharging. Pushing this same button while the application is running will makes the light meter activated: the application will tell you how much lux your keyboard is receiving. Let's reverse engineer this As far as I know, there's nothing in the USB HID protocol that handles this kind of functionality (battery status, light meter ) in a standard way. So the first task to accomplish is, unfortunately, to reverse engineer the program. I discovered a bit too late that Drew Fisher did a good presentation on USB reverse engineering at 28c3. You might want to take a look at it if you want to reverse engineer on USB. I did not need it, but I learned a few things. Anyway, my plan was the following: run the Logitech application inside a virtual machine running Windows, give it direct access to the USB keyboard, and sniff what happens on the USB wire. To achieve that, you need a virtual machine emulator that can do USB pass-through. Both KVM and VirtualBox can do that, but VirtualBox works much better with USB and allow hot(un)plugging of devices, so I used it. To sniff what happens on the USB, you need to load the usbmon Linux kernel module. Simply doing modprobe usbmon will work. You can then use Wireshark which know how to use usbmon devices and understand the USB protocol. USB stuff you need to know You don't need to know much about USB to understand what I'll write about below, but for the sake of comprehensibility I'll write a couple of things here before jumping in. To communicate with an USB device, we communicate with one of its endpoints. Endpoints are regrouped into an interface. Interfaces are regrouped into a configuration. A device might contains one or several configurations. There's also several types of packets in the USB wire protocol, and at least two of them interest us there, they are: All of this and more is well (and better) explained in the chapter 13 of Linux Device Drivers, Third Edition. Sniffed data Once everything was set-up, I ran my beloved Wireshark. There's a an URB of type interrupt sent each time you press any key with some data in it. Therefore I advise you to plug another keyboard (or use the laptop keyboard if you're doing this on a laptop), otherwise you'll get crazy trying to sniff the keyboard you're typing on. At this point, just launching the application does a bunch of USB traffic. Pressing the "light" button on the keyboard makes even more USB packets coming in and out. Here's the interesting packets that I noticed once I excluded the noise: With all this, the next step was clear: understand the packets and reproduce that exchange under Linux. What the packets mean The "go for the light meter" packet The packet sent from the computer to the keyboard is the following. <figure>
Frame 17: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
    Frame Length: 71 bytes (568 bits)
    Capture Length: 71 bytes (568 bits)
USB URB
    URB id: 0xffff880161997240
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x00, Direction: OUT
        0... .... = Direction: OUT (0)
        .000 0000 = Endpoint value: 0
    Device: 6
    URB bus id: 1
    Device setup request: relevant (0)
    Data: present (0)
    URB sec: 1340124450
    URB usec: 495643
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 7
    Data length [bytes]: 7
    [Response in: 18]
    [bInterfaceClass: HID (0x03)]
    URB setup
        bmRequestType: 0x21
            0... .... = Direction: Host-to-device
            .01. .... = Type: Class (0x01)
            ...0 0001 = Recipient: Interface (0x01)
    bRequest: SET_REPORT (0x09)
    wValue: 0x0210
        ReportID: 16
        ReportType: Output (2)
    wIndex: 2
    wLength: 7
0000  40 72 99 61 01 88 ff ff 53 02 00 06 01 00 00 00   @r.a....S.......
0010  22 ad e0 4f 00 00 00 00 1b 90 07 00 8d ff ff ff   "..O............
0020  07 00 00 00 07 00 00 00 21 09 10 02 02 00 07 00   ........!.......
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040  10 01 09 03 78 01 00                              ....x..
</figure> What's here interesting is the last part representing the data. wLength says that the length of the data is 7 bytes, so let's take a look at those 7 bytes: 10 01 09 03 78 01 00. Well, actually, you can't decode them like that, unless you're a freak or a Logitech engineer. And I have actually no idea what they mean. But sending this to the keyboard will trigger an interesting thing: the keyboard will start sending URB interrupt with some data without you pressing any more key. The "light meter and battery values" packet This is most interesting packet. This is the one sent by the keyboard to the host and that contains the data we want to retrieve. <figure>
Frame 1467: 84 bytes on wire (672 bits), 84 bytes captured (672 bits)
    Frame Length: 84 bytes (672 bits)
    Capture Length: 84 bytes (672 bits)
USB URB
    URB id: 0xffff88010c43c380
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_INTERRUPT (0x01)
    Endpoint: 0x83, Direction: IN
        1... .... = Direction: IN (1)
        .000 0011 = Endpoint value: 3
    Device: 2
    URB bus id: 6
    Device setup request: not relevant ('-')
    Data: present (0)
    URB sec: 1334953309
    URB usec: 728740
    URB status: Success (0)
    URB length [bytes]: 20
    Data length [bytes]: 20
    [Request in: 1466]
    [Time from request: 0.992374000 seconds]
    [bInterfaceClass: Unknown (0xffff)]
Leftover Capture Data: 1102091039000c061d474f4f4400000000000000
0000  80 c3 43 0c 01 88 ff ff 43 01 83 02 06 00 2d 00   ..C.....C.....-.
0010  5d c5 91 4f 00 00 00 00 a4 1e 0b 00 00 00 00 00   ]..O............
0020  14 00 00 00 14 00 00 00 00 00 00 00 00 00 00 00   ................
0030  02 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00   ................
0040  11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00   ....9....GOOD...
0050  00 00 00 00                                       ....
</figure> This packets come in regularly (1 per second) on the wire for some time once you sent the "go for the light meter" packet. At one point they are emitted less often and do not contain the value for the light meter anymore, suggesting that the control packet sent earlier triggers the activation of the light meter for a defined period. Now you probably wonder where the data are in this. They're in the 20 bytes leftover in the capture data part, indicated by Wireshark, at the end of the packet: 11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00. Fortunately, it was easy to decode. Knowing we're looking for 2 values (battery charge and light meter), we just need to observe and compare the packet emitted on the wire with the values displayed by the Logitech Solar App. To achieve this, I looked both at the Logitech Solar App and Wireshark while bringing more and more light near the keyboard, increasing the lux value received by the meter on the Solar App, and saw that the fields represented in blue (see below) where changing in Wireshark. Since 2 bytes were changing, I guessed that it was coded on 16 bits, and therefore it was easy to correlate the value with the Solar App. <figure>
[ ....9....GOOD....... ]
11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00
4 bytes - 1 byte for battery charge - 2 bytes for light meter - 2 bytes - 4 bytes for GOOD - 7 bytes
</figure> In this example, the battery has a charge of 0x39 = 57 % and the light meter receives 0x0c = 12 lux of light. It's basically dark, and that makes sense: it was night and the light was off in my office, the only light being the one coming from my screen. I've no idea what the GOOD part of the packet is about, but it's present in every packet and it's actually very handy to recognize such a packet. Therefore I'm considering this as some sort of useful mark for now. For the other bytes, they were always the same (0x11 0x2 0x9 0x10 at the beginning, 7 times 0x00 at the end). The 2 bytes between the light meter and GOOD probably mean something, but I've no idea what for now. Building our solar app Now we've enough information to build our own very basic solar application. We know how to triggers the light meter, and we know how to decode the packets. We're going to write a small application using libusb. Here's a quick example. It's not perfect and does not check for error codes, be careful. <figure>
/* Written by Julien Danjou <julien> in 2012 */
#include <linux/hid.h>
#include <libusb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
 
    libusb_context *ctx;
    libusb_init(&ctx);
    libusb_set_debug(ctx, 3);
    /* Look at the keyboard based on vendor and device id */
    libusb_device_handle *device_handle = libusb_open_device_with_vid_pid(ctx, 0x046d, 0xc52b);
    fprintf(stderr, "Found keyboard 0x%p\n", device_handle);
    libusb_device *device = libusb_get_device(device_handle);
    struct libusb_device_descriptor desc;
    libusb_get_device_descriptor(device, &desc);
    for(uint8_t config_index = 0; config_index < desc.bNumConfigurations; config_index++)
     
        struct libusb_config_descriptor *config;
        libusb_get_config_descriptor(device, config_index, &config);
        /* We know we want interface 2 */
        int iface_index = 2;
        const struct libusb_interface *iface = &config->interface[iface_index];
        for (int altsetting_index = 0; altsetting_index < iface->num_altsetting; altsetting_index++)
         
            const struct libusb_interface_descriptor *iface_desc = &iface->altsetting[al>tsetting_index];
            if (iface_desc->bInterfaceClass == LIBUSB_CLASS_HID)
             
                libusb_detach_kernel_driver(device_handle, iface_index);
                libusb_claim_interface(device_handle, iface_index);
                unsigned char ret[65535];
                unsigned char payload[] = "\x10\x02\x09\x03\x78\x01\x00";
                if(libusb_control_transfer(device_handle,
                                           LIBUSB_REQUEST_TYPE_CLASS   LIBUSB_RECIPIENT_INTERFACE,
                                           HID_REQ_SET_REPORT,
                                           0x0210, iface_index, payload, sizeof(payload) - 1, 10000))
                 
                    int actual_length = 0;
                    while(actual_length != 20   strncmp((const char *) &ret[9], "GOOD", 4))
                        libusb_interrupt_transfer(device_handle,
                                                  iface_desc->endpoint[0].bEndpointAddress,
                                                  ret, sizeof(ret), &actual_length, 100000);
                    uint16_t lux = ret[5] << 8   ret[6];
                    fprintf(stderr, "Charge: %d %%\nLight: %d lux\n", ret[4], lux);
                 
                libusb_release_interface(device_handle, iface_index);
                libusb_attach_kernel_driver(device_handle, iface_index);
             
         
     
    libusb_close(device_handle);
    libusb_exit(ctx);
 
</figure> What the program is doing is the following: This gives the following:
Found keyboard 0x0x24ec8e0
Charge: 64 %
Light: 21 lux
Challenge accomplished! To be continued Unfortunately, this approach has at least one major drawback. We have to disconnect the Logitech Unifying Receiver from the kernel. That means that while we're waiting for the packet, we're dropping packets corresponding to other events from every connected device (key presses, pointer motions ). In order to solve that, I sent a request for help on the linux-input mailing list. That way, I learned that Logitech is actually using the HID++ protocol to communicate with the devices using the Unifying Receiver. Lars-Dominik Braun managed to get the HID++ specifications from Logitech and published them with their authorization. This opens a whole new world. With that document, I may be able to understand the part I reverse engineered and convert this to a more useful and generic library using the hidraw interface (so we don't have to disconnect the devices from the kernel driver).

2 July 2012

Mehdi Dogguy: Adding useful links on bugreports pages

I'm a heavy user of Debian's Bug Tracking System (especially during this period of the year ;)) and I quite like it. And as a Debian developer, I have to check multiple sources when looking at some source package. Namely, I mainly check the following sources:
All this is great since you can move from the first one to the second one (and vice-versa) just by clicking with your mouse. It becomes less handy when you are looking at any bug report. In fact, the BTS doesn't link back to the previously mentioned services [1]. So, you have to copy/paste the name of the package or start a new tab and write down the address of the desired service. All this is not very nice for a lazy person like me :)

A few days ago, I remembered that I was already using GreaseMonkey and thought : "Hey, it would be nice to hack quickly something using GM to add the missing links". After a few tests, I've come up with a (very simple, yet extremely useful to me) user script that I called Debian BTS extra links [2]. It adds what I've been missing for years. If you are lazy and don't want to install right now, you can check the following pages to see what it adds:
The userscript will add links to PTS, Buildd status pages and Popcon near every pkgreport.cgi?src= link. Simple and useful. I hope you'll enjoy it as much as I do.

Of course, I've submitted a bug (#679282) against the pseudo package bugs.debian.org kindly asking its maintainer to add a link to (for now) the PTS. But I know that they are busy so I'm not expecting a result soon. Fow now, my simple and dumb userscript WFM.

[1] This is partly true since it links back to the PTS on main pages (pages that list all bug reports of a given set of packages).

[2] Only tested with a recent Iceweasel (from experimental) and Chromium (from unstable).

P.S.: If you do wonder, #4300 is the least bug number known to the BTS. Well, or at least, the result of running ls -1 /srv/bugs.debian.org/spool/archive/00/ sort -h head -n1 on bugs.debian.org.

30 May 2012

Paul Tagliamonte: My first foray into non-compliant Debian packages

I ve decided to start putting together packages to rebuild the bare bones install I use for most of my systems, since rebuilding everything by hand is starting to get annoying. I call it hairy candy. Because my setup relies on some chem interesting chem hacks, none of this is fit in any way for the archive. If the vim team was to catch wind of what I was doing, they d be none too happy :) While these packages work great for me, I can t stress enough how badly they can break something. Using Zack s guide, I was able to get an archive up and running, no problem. Taking his advice, I even got signing working. Simple. mini-dinstall rocks a nice alternative to a PPA, for sure. At least, for one arch :) Now, the one spin I took on Zack s work was to add a landing page a listing of what s in the archive. I did this using a bit of Python & Jinja2. The script follows.
#!/usr/bin/env python
# Copyright (c) Paul Tagliamonte, under the terms of the Expat license.
from jinja2 import Template
import sys
def parse_packages(fd):
    ret = []
    cur =  
    key = None
    for line in fd.readlines():
    if line.strip() == "":
        ret.append(cur)
        cur =  
        continue
    if line[0] == " ":
        cur[key] += "\n" + line.strip()
    else:
        key, val = line.split(":", 1)
        cur[key] = val.strip()
    return ret
pkgs = parse_packages(open(sys.argv[1], 'r'))
context =  
    "pkgs": pkgs
 
t = Template(open(sys.argv[2], 'r').read())
print t.render(**context)
Invocation looks something like:
render.py $ARCHIVE/debian/wicked/Packages $DATA/index.html
I kick this script off in the singing script, since mini-dinstall doesn t have a exit hook. Works great, thanks everyone!

19 May 2012

Richard Hartmann: Mongolia

I will start linking to places we stayed at etc, from now on. On the one hand, we already left those places so my paranoid self can rest assured that no one will come hunt us down, on the other hand, I decided to only to name and link to those places which really deserve a mention. If they are linked here, I can whole-heartedly recommend them. Mongolia After leaving through the Russian border fortifications, Mongolia was an instant and welcome change. Trees growing right next to the tracks, branches forming a bit of a natural tunnel; the trains passing by keeping branches at bay, but the trees free to grow wherever else they pleased. A few hundred meters later below a rocky ledge, a cow had apparently fallen down and died. Partially decomposed flesh still clinging halfway to its rib-cage, it proved to be a sign of the (mostly) more laid-back attitude Mongolians tend to take. After finally getting our passports back and being allowed to leave the train, we first headed to the station to get some money. At ~1800:1, the exchange rate for Tugruk against Euro speaks volumes about inflation; imagine my surprise that the MNT 20,000 note is the largest one. It's also the one and only amount any and all ATMs will default to. We were quite surprised by how cheap everything was, even in the one single store within the station directly behind the border. In Russia, travellers expect to pay hefty markups when shopkeepers know they can charge them, not so in Mongolia. The woman in the trash Upon returning from the station, our waggon was just merrily being driving away on its own by a lone engine, something we had not anticipated or appreciated a whole lot. A woman (we hadn't noticed her before) who was busy scavenging a trash container for food for herself and her scrawny-looking son happily stopped what she was doing and, with a big smile, moved her hands around and smacked them back together, signalling us that our waggon would be attached to a different train for the subsequent journey. To put it mildly, I was stunned; we had met people eating from the trash in Russia as well (yes, I always make a point of giving them enough for a few decent meals; anyone eating out of the trash actually needs and definitely deserves something good happening to them), but they were glum and disheartened. Here, there was someone who not only had to look out for herself, but for a three or four year old boy as well, and still was just so... positive... I don't know if I would have it in me to act the same way... As an aside, even though we were just a few kilometers across the border from Russia, she did not actually take the money from me, she cupped her hands to receive it by having me place it into her hands; this is a very Asian thing in my experience and something we would see both in Mongolia and China consistently. Later, when we were back in our waggon, she passed by and happened to see the Spot Messgenger 2 dangling in front of our window, its lights blinking every five seconds. For a solid twenty minutes, until the train started moving again, she squatted down with her son, oggling the Spot. I have no idea what she was thinking it was and I had no chance to explain it to her, but she was really fascinated and just kept staring and staring. When we finally left the station, we waved to and smiled at each other, she ran after us, and then she was gone. All in all, this was one the more memorable things which happened to us. Mongolian Infrastructure When I say that Mongolians as a whole tend to be a lot more laid back than Russians, this extends to infrastructure as well, but not in the good way. Ulan Bator Little Dhingis Khan We arrived in Ulan Bator at six in the morning and started haggling with taxi drivers. Our hostel had (stale, it turned out) information about acceptable fare from station to their place on their website, 5,000 MNT was supposedly OK. Our driver tried to get 20,000, then 15,000, then 12,000, then 10,000, went down to 7,000 and finally settled for 5,000, all this interspersed with frequent "no" by us and walking away. Once on the road, he wanted to convince me that we had agreed upon 5,000 per person, something that did not quite work out for him. When we arrived, he started to tell me that he was good friends with the hostel manager and that said manager would surely want us to pay more; again, that did not work too well. He made the mistake of getting our luggage out of his trunk immediately (we always kept all our stuff in the passenger area afterwards...) before I paid and even though I only had a 10,000 or 20,000 note, I insisted on correct change. The same old dance began anew, with him edging closer to the correct change in steps of 1,000 MNT. He tried to get away several times, but I stood in his door and held it open; if he had just driven off, I couldn't have stopped him, but that thankfully didn't occur to him. After he shoved all small change (MNT is bill only, no coins) he had with him at me and wanted to make off yet again, I was close to throwing all the small crap at him, but after he, a man of maybe 1.65 meter and sitting down, tried to signal his willingness for a fist fight and me, a man of 1.94 meter and standing started laughing, he gave me 2,000 more and drove off cursing. In case you forgot, at that exchange rate, we argued about two to three Euro, but as he tried to cheat, it became a matter of principle. The Hostel, part one All parking lots and other property in Ulan Bator have a small watchtower with private 24/7 guards; said guard ambled down, let us in and woke the poor woman who was forced to let us check in. As she was Mongolian as well, we were a bit surprised when she switched from broken English to perfect German, but the owners of the OASIS are a German-Austrian couple so I guess that makes sense. Fighting through the war Traffic in Ulan Bator is war. Lonely Planet talks about the city being too newly motorized and that it did not have to develop a culture of driving, yet. This is a euphemism for "no one has any qualms maiming or killing you". Everyone is driving like mad, nudging and racing their way in front of each other. If a driver has an oppurtunity to get ahead by a few centimeters with his small car, thereby blocking the way for several buses, making a whole parking lot grind to a halt and thereby deadlocking themselves, they will take said opportunity, and gladly. Red lights are gentle suggestions, there to be mainly ignored; police will not drive over red lights, but they will not stop anybody else from doing so, either. A hearse will have a dozen or more cars following it, all driving in a tight pack, aggressively attacking anyone who dares to wedge in between them and expecting all other traffic to make way even if they are driving over a red light. Crossing a main street in Ulan Bator as a pedestrian is hell. There are a few traffic lights and they help to some extent, but you still need to be extremely careful when crossing a street. Thailand, India, Russia, China, no matter where it's supposedly hard to cross a street by foot, I never had any issues weaseling through. In UB, it takes planning, determination, and a lot of attention, especially as there are stretches where there's not a traffic light in sight. While maintaining eye contact with a driver usually ensures that they will try not to hit you, this does not work in the least in UB. In the evening, I took to shining a high-powered flashlight at the ground to the oncoming traffic's side to mark ourselves and decrease the likelihood of being hit. This worked well in combination with sprinting through gaps in the traffic, except for one guy who actively aimed for us and did not slow down. He even changed lanes, just to keep us in front of his car... Shining the flashlight directly at him in highest mode made him break quite violently though; a good thing, as he may have hit us had he not slowed down. UB itself Ulan Bator is weird. It has clear signs of westernization, but it's still very much an old city. There is one large store which would fit into any random shopping center in Germany (which are tiny when compared to most other countries) and which carries most goods, if with a limited selection. Its supermarket is extremely well stocked with food and drink from all other the world and laughably cheap (for us) prices. While its infrastructure is definitely crumbling, the city is clean, especially when compared to Russia. Two girls from Stuttgart who we met during our Trans-Siberian travels commented on how clean Russian cities were when compared to home, but that may be more of a statement about Stuttgart than about Russia... Anyway, in Mongolia in general and in UB in particular, there's a concise effort to make what they have look nice and clean. There is no trash in the streets, loose gravel and earth is removed from the curb by dedicated workers, all areas of packed earth are sieved and all large stones removed. All in all, it's a lot cleaner (if more dusty and sandy) than Russian cities if somewhat older and more shabby-looking. That being said, Mongolia's countryside is littered There were several street stalls consisting of nothing more than a cardboard box on a stool selling single chewing gums and cigarettes from the original packaging; it's normal for people to buy one cigarette or one piece of chewing gum and walk on. The sights of UB were nice, but nothing too special. Outlook In the next part, I'll cover "The Hostel, part disaster" and generally talk more about excrement than I thought one could write before moving on to China (were I am located at the time of this writing).

Richard Hartmann: Motherland's bosom

I read a translated poem about Russia being "the Motherland" and its vast bosom years ago. Having driven through a significant part of it, I can agree on the "vast" part... Also, as I am on a train and without access to the Internet, I will refrain from linking to a lot of pages; sorry. (Turns out I am posting this a week later, but I will still not link to stuff now; no time). Russia in general Moscow Sights Kreml Our remaining time in Moscow was spent with touring the usual suspects; the Kreml is a lot less impressive in real life, the Red Square is tiny when compared to the stories I heard about it and the Chapel ofi St. Basil is even more colorful and impressive in real life. Lenin's body was inaccessible because workers built seats for the May 9th parade to the left and the right of it and they apparently thought it would be a good idea to block access to one of the main tourist attractions while doing so. A river tour of Moscow was a nice cool-off and we got to see quite a few things. We managed to see the weekly military parade within the Kreml grounds, but it was mostly pomp and little substance. The National Treasure which you can access with an extra ticket within the Kreml grounds is nice, but less impressive than the tourist guides would make you believe. That being said... There's another museum within the museum and.... Whoah... Tourists pay extra, visitors go through the only non-security-theater check I encountered in Russia, guards are armed, people can only enter and leave in batches, and the stuff which is presented is mind-boggling. Disregarding the fist-to-calf-sized chunks of gold and platinum which are still in their original form directly from the mine, there is real, actual treasure galore. Little heaps of uncut and cut diamonds, an outline of Russia filled with cut diamonds and other random "we have this stuff" displays can be found as well. Then, you have various tiaras and other jewellery made from various gems. Not incorporating, but largely made of. All that pales in comparison to the crown, royal apple, scepter, etc. It's hard to put the amount of tiny multi-colored light points that shine at you into words. I was just standing there, swaying back and forth to catch the moving pattern of pinpoints. It's said that this collection is equalled only by the ones in the Tower of London and the one Shaw of Iran had and boy do I believe it. TV Tower Getting up there was funny. The old-style Soviet queuing system was used: "Security" for approaching the tower was multi-level, the guards see you approach along a long walkway way in advance and the main guard shed had several small cabins separated by thick glass. So good so menacing. But in a twist that would make Bizarro and Garry Larson proud, I was required, by means of metal detector gate, metal detector wand and even an x-ray machine to remove every shred of metal and other hard objects from myself and the camera bag and put them onto a table. Once I was without anything except my clothes and the bag was completely empty, I could pass. Everything I had had to remove was just laying there, not inspected in the least, for me to stuff back into pockets and bag and to take with me. This "everything" included a Spot Messenger 2 with lots of green and red blinky lights. The guard did not even glance and it. Security theater? Security theater. The view from 364 meters down on Moscow was nice, but there was a lot of Smog so I couldn't see very far. Jumping on the glass floor while looking down was a lot of fun, though. Subway to Thiefing I bet Christopher Nolan rode the subway in Moscow at least once. That unnerving sound you hear during several key scenes in "The Dark Knight"? Two thirds of all subways make the same sound while moving. Also, I had an encounter with a pickpocket down there; very classical, too. Guy approaches quickly, talks loudly and sounds as if it's really important (in Russian... duh... that's sure to keep me interested). His approach made me turn and protect my left leg pocket automatically, most likely marking the target for the tiny woman standing behind me. Now, I have to tell you something about my usual travel layout. As my normal pockets are very deep, it looks as if their content was in the leg pocket. Plus, there's an extra, hidden leg pocket where I keep the passports and train tickets. The outermost leg pocket is protected by a velcro flap, but it contains nothing of value; usually the appropriate phrasebook, local map, maybe a tissue or chewing gum. Due to this layering, the outermost pocket looks as if it's full to the brim with stuff. Also, I took pains to make it a habit to protect said leg pocket with my hand, nothing else. This looks as if that's the target, but what I am actually doing is protect my normal pocket with my forearm. The right side is different, but the most easily accessibly pocket always holds some small change. I pay from that stash but my actual wallet is well out of reach. Anyway, once the guy ran off, talking to several others, most likely marking all them for the actual pickpockets, I wanted to enter the subway. While the Russian-style queuing took place, I felt an unusual tug at the velcro flap. I looked down and saw a tiny woman to the left of me with a jacket held over her right side with the left arm; I look up to check no one is trying to steal from my permanently assigned female, feel another tug, look the woman into the eyes, look up again and around me, look down again and she is gone. All that took maybe three seconds and I had boarded the subway after an additional two. In hindsight, it makes sense to choose the time of entry for attack. It's crowded, you are being pushed around, and once you are in the subway, it will start moving more or less immediately while the thief remains in the station. In this case, she would only have gotten a grubby map of Moscow's subway and an English-Russian phrasebook, but she got nothing at all. Moscow-Novosibirsk Where to begin... If you think a few hours on a train are a long time, try over fifty hours. Things get so bad, you start getting land-sick while not in a moving train. You even start missing the familiar tunk-cachunk, tunk-cachunk, tunk-cachunk... of driving over rails with gaps in them when you are not moving. The defining element of the Trans-Siberian Railway are birch trees. And birch trees. And then more birch trees. You would not believe how many birch trees there are. This is made "worse" by the way the Russian Railway protects their rails. Left and right of the track, there's a cleared area of maybe ten to twenty meters, sometimes as little as three. Outside of that, they plant ten to twenty meters of birch trees, presumably to catch snow during winter. Beyond that protective perimeter, there's the normal landscape.As a result, on top of the near endless stretches of birch woods, you see most if not all scenery through a layer of birch trees. You get sick sick of birch trees after a few hours and you see them for days on end. Bullet points to save myself some typing and you some reading... Novosibirsk The non-existent hostel We arrived at ~0200 local and made our way to the hostel we had booked a room with. Walking to the correct address, we saw several signs but they all turned out to be for a police station and some other state agency. We walked back, forth, double-checked, triple-checked: no hostel. We then walked around the building through some not-quite-nice back alleys, but other than a few entries to private flats, there was nothing. Thankfully, the booking slip included a number which we called and after at least twenty rings (no kidding), when I had given up and wanted to hang up, it stopped ringing. Dead silence. After maybe ten seconds, someone started talking in Russian. I asked him if he spoke English and told him that we could not find the hostel. He mumbled something about being sorry and that we should wait, he would come down. Fast forward a minute or two and someone walked towards us. Again, he mumbled about being sorry, that the hostel "did not work" at the moment and that we would need to sleep in his private apartment. He ushered us into some back alley entrance, into his flat, and proceeded to remove the sheets from the couch on which he had slept; after putting on new sheets, we had our "hostel" bed, ready to sleep on. We briefly considered if he would murder us in our sleep, but him and me even got to talking a bit. Over cheese, sausage and rum (at 0300), he admitted that the hostel did not exist and he merely planned to turn his flat into a hostel for the summer while he and his family moved into their summer house (the Russian term of which escapes me, at the moment) in the countryside. He had accepted our reservation as he thought he would be finished by that time. He did not even get started, though. While he sent us an overbooking notice through booking.com two days before, we were on the train at that time, so... booking.com even called him to check what happenend to us as we did not book another place through them. Good customer service/protection, that. Next morning, he didn't even want to take our money (we paid anyway) and, as a means of compensation, drove us into the city in the morning and to a train museum well outside the city limits, one of the fabled scientist cities, and a large lake which everyone in Novosibirsk claims is an ocean, in the afternoon. Foreigners, foreigners! All in all, Novosibirsk was relatively uneventful, safe for one bizarre episode. We took our lunch in a local fast food joint (why do all the good stories happen there, and not at the various truly local places?) and threw the cashier our well-rehearsed "Niet Russkie; anglisky?" with phrasebook in hand and he actually understood a few words of English (beef, chicken, fries). We told him, in our worst Russian, that we are from Germany wished him a nice day and went to sit down. A few minutes later, a girl approached us, literally hopping from one foot to the other and wringing her hands. She told us that the cashier had told her that we spoke English and if it would be OK if she talked to us. We suspected some sort of elaborate ruse, but went with it. Turns out, she had English at school and really wanted someone to practice English on. Two young men passed our table and exchanged a few words with her, sitting down out of sight. When she told us that she had to leave now but if it would be OK if the two boys joined us we suspected a ruse yet again. But those two were law students, one with a minor in English and one with a minor in German; both of them also extremely nervous, asking us if we would talk to them. When they had to leave, they told us that the three of them worked at the burger joint and that their shift was just about to start when the news that foreigners were here spread amongst staff like wildfire. The girl stopped by several times in between cleaning tables, getting in a sentence or two before being cussed at by her supervisor. All in all, this took about twenty minutes and seeing three people so nervous and grateful to talk with us felt beyond absurd. On the other hand, not a single traveller we met even considered stopping in Novosibirsk during their transit so there really does seem to be a shortage of non-Russians there. Weird, and memorable. Novosibirsk-Irkutsk Irkutsk / Listvianka / Lake Baikal Listvianka Aah, lake Baikal... the oldest and deepest lake on Earth which holds a fifth of the global non-salt water reserves; a must-see in my book. Quad tours at break-neck speeds, dry-suit diving with Russian regulators, walking barefoot in between and across drift ice that made its way onto the shorei, and extended hiking around the lake's coast... All of which I could not do because I was ill and had to spend two solid days in bed. The draft from the open window in between Novosibirsk and Irkutsk was enough to give me a rather bad cold which peaked at Lake Baikal. Still, the area was lovely and we were glad to be out of a train and able to unpack our stuff without having to repack immediately for once. I am not sure where my current losing streak with regards to diving is coming from (Grimsey, diving north of the Arctic circle with birds that plummet into the water and hunt fish: Only guy who does this is on the Icelandic mainland that day; Svalbard, diving north of the Arctic circle in permanent darkness: The few people who do this privately did not reply while I was there; Baikal, oldest, deepest, largest lake on Earth: ill), but I will most likely return to Russia for a week of ice diving in Lake Baikal next winter or the one after that. As an aside, I saw several people walking to Lake Baikal with buckets to get their water. Other people got it from a well which was still half frozen. If you have running water consider yourself lucky... Irkutsk Nice city, largely uneventful. The farther east you get within Russia, the more normal women look. In Moscow, just as in Paris, they are way over-dressed and even service personnel will walk with high heels. Thankfully, I don't have to wear heels, but for the other males out there: Walking and standing in these things hurts and thus most if not all people who stand and walk for a living have flat shoes. We happened upon preparations for a military parade, complete with cordon, viewing podests, at least half a dozen TV cameras etc, but were not sure if it would start soon enough for us to catch our train.We asked someone who told us it would start at 2100 local, at 1945 local it seemed about to start, and sure enough at 1955 sharp, the whole thing went under way. About a dozen groups of 50-100 people each, all in their own, respective uniforms stood against one side of a cordoned-off street and several higher-ups on the other side. Two highest-ups shouted into microphones and the throng of people on the other side shouted back answers. Then, the two highest-ups stood in the back of a jeep each and drove past said throng, stopping in front of each group, shouting into microphones mounted in the back of the jeeps and the groups shouted back once again. After that, all groups marched around the make-shift plaza once, saluting the higher ups. Once they were done, and they took ages, two trucks drove by with soldiers jumping out of the moving trucks and moving into crouching positions. They ran around in a circle a few times and engaged in pretend hand-to-hand combat. I am sure they are skilled at whatever style they wanted to show, but they were overdoing things so badly, they were funny, not imposing. When they jumped over some barriers, the barriers fell to pieces and everyone scrambled to make it look as if that was part of the show. While carrying off the gear, it fell into further pieces which was even more funny. An armoured personnel carrier ended the show; several tougher looking guys jumped off of that one and their mock combat involved fully automatic fire (of blanks), several flashbangs, smoke grenades and, to top things off, the machine gun mounted on the APC moving down the opposing team with blanks. I never witnessed a "real" military parade in person but this one was somewhat disappointing. On the one hand, there was a distinct lack of ballistic missile carriers and tanks like you see in movies, documentaries and games, on the other hand, the whole thing had a make-do feeling to it. The cordoning police had designated spots to stand on, yet walked around. They were standing to attention, yet checking their cell phones. Several people in one uniformed group were wearing track suits and jeans. Another uniformed guy had a grocery bag with him; yet another one was carrying a huge water bottle. Bikers zig-zagged through the cordon and when the whole show was just about to wrap up the police finally started putting up barriers around the unmoving pedestrians, not blocking the bikers. One little girl was standing well within the cordoned area, watching with big eyes and after she did not react to the police talking to her, they just built the barriers in a curve around her. And to top it all off, some guy with a cane walked all through the parade with his personal camcorder, trying to direct the whole show while being ignored by everyone. Still, I am sure he managed to mess up some otherwise perfectly good TV scenes. Irkutsk-Russian border TL;DR 3000 kilometers of birch trees

16 May 2012

Kees Cook: USB AVR fun

At the recent Ubuntu Developer Summit, I managed to convince a few people (after assurances that there would be no permanent damage) to plug a USB stick into their machines so we could watch Xorg crash and wedge their console. What was this evil thing, you ask? It was an AVR microprocessor connected to USB, acting as a USB HID Keyboard, with the product name set to %n . Recently a Chrome OS developer discovered that renaming his Bluetooth Keyboard to %n would crash Xorg. The flaw was in the logging stack, triggering glibc to abort the process due to format string protections. At first glance, it looks like this isn t a big deal since one would have to have already done a Bluetooth pairing with the keyboard, but it would be a problem for any input device, not just Bluetooth. I wanted to see this in action for a normal (USB) keyboard. I borrowed a Maximus USB AVR from a friend, and then ultimately bought a Minimus. It will let you put anything you want on the USB bus. I added a rule for it to udev:
SUBSYSTEM=="usb", ACTION=="add", ATTR idVendor =="03eb", ATTR idProduct =="*", GROUP="plugdev"
installed the AVR tools:
sudo apt-get install dfu-programmer gcc-avr avr-libc
and pulled down the excellent LUFA USB tree:
git clone git://github.com/abcminiuser/lufa-lib.git
After applying a patch to the LUFA USB keyboard demo, I had my handy USB-AVR-as-Keyboard stick ready to crash Xorg:
-       .VendorID               = 0x03EB,
-       .ProductID              = 0x2042,
+       .VendorID               = 0x045e,
+       .ProductID              = 0x000b,
...
-       .UnicodeString          = L"LUFA Keyboard Demo"
+       .UnicodeString          = L"Keyboard (%n%n%n%n)"
In fact, it was so successfully that after I got the code right and programmed it, Xorg immediately crashed on my development machine. :)
make dfu
After a reboot, I switched it back to programming mode by pressing and holding the H button, press/releasing the R button, and releasing H . The fix to Xorg is winding its way through upstream, and should land in your distros soon. In the meantime, you can disable your external USB ports, as Marc Deslauriers demonstrated for me:
echo "0" > /sys/bus/usb/devices/usb1/authorized
echo "0" > /sys/bus/usb/devices/usb1/authorized_default
Be careful of shared internal/external ports, and having two buses on one port, etc.

2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

5 May 2012

Russell Coker: Liberty and Mobile Phones

I own two mobile phones at the moment, I use a Samsung Galaxy S running Cyanogenmod [1] (Android 2.3.7) for most things, and I have a Sony Ericsson Xperia X10 running Android 2.1 that I use for taking photos, some occasional Wifi web browsing, and using some applications. Comparing Android Hardware The hardware for the Xperia X10 is better than that of the Galaxy S in many ways. It has a slightly higher resolution (480*854 vs 480*800), a significantly better camera (8.1MP with a flash vs 5MP without), and a status LED which I find really handy (although some people don t care about it). The only benefit of the Galaxy S hardware is that it has 16G of internal storage (of which about 2G can be used for applications) and 512M of RAM while the Xperia X10 has 1G of internal storage and 384M of RAM. These are significant issues, I have had applications run out of RAM on the Xperia X10 and I have been forced to uninstall applications to make space. Overall I consider the Xperia X10 to be a significantly better piece of hardware as I am willing to trade off some RAM and internal storage to get a better resolution screen and a better camera. The problem is that Sony Ericsson have locked down their phones as much as possible and they don t even give users the option of making a useful backup they inspired my post about 5 principles of backups [2]. The fact that the Galaxy S allows installing CyanogenMod which then gives me the liberty to do whatever I want with my phone is a massive feature. It outweighs the hardware benefits of the Sony Ericsson phones over Samsung phones prior to the Galaxy Nexus and Galaxy Note. For an individual user the ability to control their own hardware is a feature. Such an ability wouldn t be much use if there wasn t a community of software developers, so if you buy an Android phone that isn t supported by CyanogenMod or another free Android distribution then whether it is locked probably won t matter to you. But for any popular Android phone that s sold on the mass market it seems that if it s not locked then it will get a binary distribution of Android in a reasonable amount of time. Comparing with Apple It seems that Apple is the benchmark for non-free computing at the moment. The iPhone is locked down and Apple takes steps to re-lock phones that can be rooted as opposed to the Android vendors who ship phones and then don t bother to update the firmware for any reason. The Apple app market is more expensive and difficult to enter and if an app isn t in the market then you have to pay if you want to install it on a small number of development/test phones. This compares to Android where the Google market is cheaper and easier to enter and anyone can distribute an app outside the market and have people use it. But for an individual this doesn t necessarily cause any problems. I have friends and clients who use iPhones and are very happy with them. In terms of software development it s a real benefit to have a large number of systems running the same software. As Apple seems to have higher margins and larger volume than any other phone vendor as well as shipping only one phone at any time (compared to every other phone vendor which seems to ship at least 3 different products for different use cases) they are in a much better economic position to get the software development right. As far as I can tell the hardware and software of the iPhone is of very high quality. The iPad (which has a similar market position) is also a quality product. The fact that the Apple app market is more difficult to enter (both in terms of Apple liking the application and the cost of entry) also has it s advantages, I get the impression that the general quality of iPhone apps is quite high as opposed to Android where there are a lot of low quality apps and many more fraudulent apps than there should be. The lack of choice in Apple hardware (one phone and one tablet) is a disadvantage for the user. There is no option for a phone with a slide-out keyboard, a large screen (for the elderly and people with fat fingers), or any of the other features that some Android phones have. The lack of a range of sizes for the iPad is also a disadvantage. But it seems that Apple has produced hardware that is good enough for most users so there aren t many complaints about a lack of choice. It seems to me that the biggest disadvantage of the closed Apple ecosystem is for society in general. Anyone who wants to write a mobile app to do something which might be considered controversial would probably think twice about whether to develop for the iPhone/iPad as Apple could remove the app at a whim which would waste all the software development work that was invested in writing the app. Google seem to have much less interest in removing apps from their store and if they do remove an app then with some inconvenience it can be distributed on the web without involving them so the work won t be wasted. How Much Freedom Should a Vendor Provide? The Apple approach of locking everything down is clearly working for them at the moment. The Samsung approach of taking the Google prescribed code and allowing users to replace it is good for the users and works well. The Sony Ericsson approach of taking the Google code, adding some proprietary code, and then locking the phone down is bad for the users and I think it will be bad for Sony Ericsson. People are more likely to tell others about negative experiences and negative reviews are more likely to be noticed than positive reviews. So while many people are reasonably happy with Sony Ericsson products (until they find themselves unable to restore from a backup) it s still not a good situation for Sony Ericsson marketing. It seems that there are benefits to hardware vendors for being really open and for locking their users in properly. But being somewhat open isn t a good choice, particularly for a vendor that ships poor quality proprietary apps such as the Sony Ericsson ones. In terms of application distribution Google isn t as nice as they appear. The Skyhook case revealed that Google will do whatever it takes to prevent apps that compete with Google apps from being installed by default [3]. Google is also trying to make money from DRM sales via Youtube which it denies to rooted phones [4]. Again it seems to me that the best options here are being more open than Google is and being as closed as Apple. Google might gain some useful benefits from applying DRM (even though everyone with technical knowledge knows that it doesn t work) but the Skyhook shenanigans have got to be costing Google more than it s worth. How to make Android devices more Free The F-droid market is an alternative to the Google App market which only has free software [5]. On it s web site there are links to download the source for the applications, including the source and binaries for old versions. In the Google App market if an upgrade breaks your system then you just lose, with F-droid you can revert to the old version. A self-hosted OwnCloud installation for a private or semi-private cloud [6] can be used as an alternative to the Google Music store as well as for hosting any other data that you want to store online. The Open Street Map for Android (Osmand) project provides an alternative to the Google Map service [7]. Osmand allows you to download all the vector data for the regions you will ever visit so it can run without Internet access. But it doesn t have the ability to search for businesses and the search for an address functionality is clunky and doesn t accept plain text, which among other things precludes pasting data that s copied from email or SMS. While Osmand provides some important features that Google Maps will probably never provide, it doesn t provide some of the most used features of Google Maps so uninstalling Google Maps isn t a good option at the moment. The K9mail project provides a nice IMAP client for Android [8]. Use K9 with a mail server that you run and you won t need to use Gmail. There are alternatives to all the Google applications. It seems that apart from the lack of commercial data and search ability in Osmand an Android device that is used for many serious purposes wouldn t lack much if it had no Google apps. Google seems to be going too far in controlling Android. Escaping from their control and helping others to do the same seems to be good for society and good for the users who don t need apps which are only available in proprietary form. Related posts:
  1. Dual SIM Phones vs Amaysim vs Contract for Mobile Phones Currently Dick Smith is offering two dual-SIM mobile phones for...
  2. My Ideal Mobile Phone Based on my experience testing the IBM Seer software on...
  3. Old Mobile Phones as Toys In the past I have had parents ask for advice...

30 April 2012

Russell Coker: The Royal Caribbean Official Android app

I ve just played with the official Android app [1] for the Royal Carribean cruise line [2]. The cruise line is apparently great (I ve never been on one of their ships but the reviews are good) but the Android app isn t. Net Access The most obvious and significant problem with this app is that it s entirely useless without net access. All data of note comes from the Internet which means that the program is useless in any location where Internet access is unavailable (or unreasonably expensive). They wrote an app about cruising that can t be used on a cruise ship! Did they even think about what they were doing? The correct thing to do when writing such an application is to have all basic data about all ships included in the app. This means that when they change the deck plan of a ship they need to release a new version of the app and have people download it. Having done a lot of software development I understand that forcing software updates (even updates to included data files) involves some effort and expense. But when they spend $20,000,000 to update a ship (which is about the minimum that is spent for a major ship in dry-dock according to TV documentaries I ve watched) it seems quite reasonable to budget $10,000 to release new software. Also one benefit of updating the software is that it can promote the changes, after spending tens of millions of dollars improving a ship they probably want to promote that to customers and pushing a new app update with adverts for the improved ship seems like a good way of doing that. There is some data that can t reasonably be included in the app due to size constraints with photos of ships being the most notable example. The solution to this is to provide an option for the user to cache the data that interests them. For example if I was meeting some people to discuss the possibility of a group cruise on a RCL ship then I could download all the pictures of that ship on my home Wifi network and then have them all available with no delay or 3G costs. Also the app seems to hang if net access is temporarily interrupted. As phones are expected to have unreliable net access this is also a significant flaw. Maps The maps of the ships are comprised of a series of pictures which each show one deck. In addition to being downloaded (not cached or included in the app) they aren t scalable (they should be SVG or at least allow zooming the high resolution pictures) and they don t allow a 3D view. The paper maps used to promote cruises (including RCI cruises) and which are given to all passengers on Princess cruises (I m not sure about other lines) show a side cutaway view of the ship which is handy for working out which things are near where you are. It seems that an ideal cruise ship mapping program would have some sort of 3D component, maybe X3D. My experience is that a two night cruise isn t long enough to become familiar with one of the smaller cruise ships. Using a map is essential and a smart phone is a good way of managing such a map as typical 2D paper maps just aren t good enough for such a large and complex structure. Photos One of the significant things that is wrong with the app is a lack of care in displaying the photos of ships. They display three pictures of the Allure of the Seas (one of the two newest, biggest, and most luxurious ships in their fleet), but one of those three photos is actually of the Oasis of the Seas. The fact that the two ships are almost identical is no excuse, there is a principle at stake! Also only having three pictures is pretty poor, there is no way that less than 50 pictures could do justice to such a big ship! A Google search for the words cruise and photos turns up many sites with pictures of cruise ships and it s not particularly difficult to find pictures of any particular ship. Photographs by customers are often of high quality as some of the better DSLR cameras are in the same price range as some of the cheaper tickets for cruise ships. Probably the best thing that RCI could do is to run a contest and allow their customers to enter photos and vote towards the winning entries. That would get them photos that aren t as sterile as the official photos and which include the things that are of most interest to customers. Finally in terms of caching, pictures are the most easily cached source of data and as phones get higher resolution they keep getting bigger. The storage space for a modern phone is equivalent to the entire 3G download quota for about a year on an affordable Australian 3G plan. When dealing with photos downloaded from the net the default should be to cache everything. Navigation Navigating a smart-phone app is a lot more difficult than navigating the same data on a desktop system (which would be in a web browser). Users can compensate for some deficiencies with web site organisation by using a large monitor and having several web browser windows with multiple tabs. But with a phone it should be possible to switch between things quickly. The main menu has a View Our Ships option (which allows viewing deck plans and pictures) and a View Our Staterooms option which offers a list of ships and then describes the state rooms available for each one. This means that you can t see all the information about a ship in one place and even worse you can t easily compare ships. As it seems likely that people will want to use this app for selecting a cruise it should be possible to select a few ships that are of interest and then quickly flip between them. For example the Rhapsody of the Seas and the Voyager of the Seas are cruising in my part of the world so it would be nice if I could tell the app to compare those ships and then allow me to view a page about one ship and then flick to the equivalent page about the other ship. Another notable problem is that the ships are listed in alphabetical order. The sensible thing to do is to list them by class going from biggest to smallest. Lessons to be Learned These problems aren t specific to the RCI app, many other Android apps have the same flaws. For example the Google Play market app doesn t cache the icons of the installed apps so every time I want to see a list of installed apps it goes slow and wastes some of my bandwidth. Doing something wrong in the same way as Google isn t necessarily a great mistake, although using the Google Play market on a cruise ship is probably very uncommon. Probably the biggest problem is a lack of testing. They should have sent the developers on a cruise as a live test. Every cruise ship has a sales desk for booking future cruises so it wouldn t be difficult to have a dozen Android phones at the sales desk to see how real customers who really want to book a cruise find it. I presume that even if net access was available then such a test would fail dismally. If a 3D display of a ship combined with all the data management capabilities of a modern smart phone (which is a lot more powerful than the desktop systems I used prior to 2000) can t at least be a useful supplement to a stack of paper brochures then it s probably a failure. I think that the RCI app is an example of how to make an Android app which doesn t fall into the more common failings (such as being a quick and dirty port from iOS) but yet still isn t useful to customers. I recommend that people who develop apps which have an objective of imparting information to users try it out as an example of what not to do. Try a few basic tasks like comparing the three biggest classes of RCI ships in terms of features, after failing to do that with the app you can then use Wikipedia to get the result. But don t use the Wikipedia client apps, use a tabbed browser such as Opera Mini. Related posts:
  1. An Introduction to Android I gave a brief introductory talk about Android at this...
  2. Choosing an Android Phone My phone contract ends in a few months, so I m...
  3. Galaxy S vs Xperia X10 and Android Network Access Galaxy S Review I ve just been given an indefinite loan...

23 April 2012

Christian Perrier: Boston marathon 2012

I just read reports about Boston marathon 2012. Incredibly though race this year as the heat raised up to 32-35 C (around 90 F?) for the last runners and 26 C at 10:00 at marathon start. The outcome is indeed impressive: the 2012 winner won with a "not so good" time of 2h12 while last year saw an unofficial world best time of 2h03'02" (there can't be any world record in Boston because of the race being one-way, thus prone to too favourable wind, plus a negative slope between start and finish). Even though this year is less likely to see great records (because this is an olympic year and the best runners won't run another marathon before the Olympics), we still had times around 2h05 in Paris, Rotterdam....and this year's best time in London with 2h04'44" (Dubai time is 2h04'23" but I do not count this one as a mass marathon). So, 2h12 in Boston just says how hard this race was. But, still, that's my dream marathon, even more than NYC marathon. And I'll do it, I promise....though I now have to run below 3h30 to qualify....unless I'm still running marathons when I reach 55, where the qualificaiton mark will be only 3h40 (which is exactly my PB as of now). Will I still be able to run in 3h40 in 4 years? Or will I succeed in running in less than 3h30 in the meantime? We'll see..:-) Until then....my first marathon for this year will be on June 9th, in Caen, as planned. Target: less than 3h40, despite the quite hard profile (and possible windy conditions as the first half of the race is along the famous Normandy beaches code named Sword, Gold and Juno, between Courseulles sur Mer and Ouistreham). So, until then, I'll forget about my nice races in forests and plains, and concentrate on interval trainings, boring sequences of 400m or 1000m runs, etc, etc. That will be the first time I'm doing this but everybody keeps telling me I need to do that if I want to lower my PB (and every marathoner wants to!).

15 April 2012

Richard Hartmann: All aboard the Choo Choo train!

All aboard the Choo Choo train! As some of you know, we are planning our trip on the Trans-Siberian (well, Trans-Mongolian to be exact) railway, at the moment. Given the wall of text my post about Svalbard turned out to be, I am trying to split early and often. The plan Our three-week itinerary is pretty much fixed, by now: Visa issues Not wanting to just book any random package with any random travel agent, we are picking and mixing our own, as usual. Given visa requirements, language barriers and things that are plain weird, it's been an interesting ride before we even pack our bags a month or two from now. Being European, visas are something that do not concern us a lot, normally. I am starting to realize how incredibly lucky we are... barring India, we never needed to do much to be allowed to enter any foreign country. Not so on this trip... While Mongolia has rather lax, or let's just say reasonable, terms, China and Russia are different. To get a visa for Russia, you need: For China, you need to provide a night-to-night itinerary of hotel stays. This may be a remnant of Mao's philosophy of restricting free movement of the population within China, I don't know. On the plus side, they don't require an invitation, health insurance or means of industrial espionage. With processing times ranging from one to four weeks per visa, you will wait one to two months until your passports are back with you and properly visa'ed by everyone. Tickets, please Purchasing tickets in advance, but not through a travel agent, is turning out to be rather complicated. As you can not book train tickets with RZD, the Russian rail company, more than 45 days in advance, you end up with nice circular dependencies. We could pay a premium for faster processing of our visas and buy via travel agencies which sell train tickets from their special contingents, but that's kinda boring, innit? Also, rzd.ru offers Russian order forms, only. With the help of Google translate, ##russian on irc.freenode.net, and a lot of guess-work, we were able to hammer out the connections from Moscow to Novosibirsk and from there on to Irkutsk. Yet, it's quite a different experience not to not be able to fall back to English. No matter where we went in the past, English was the lingua franca, at least to some extent. That's not the case in any of our three intended destinations so those phrase books will get used a lot, I suspect... We are still not sure if the 4-people cabins in second class have a door, let alone one that can be locked, but we decided that this is not too much of an issue. Generally speaking, people are nice and there are train attendants so crime should not be an issue. And even if things do disappear, as long as we stay healthy and I still have one of my several copies of all photos I will take, I will be reasonably happy. As to getting from Irkutsk to Ulan Bator... even a Russian travel agent we contacted claimed you can only purchase tickets on site, not online. From Ulan Bator to Beijing, it's a different booking system. And from Beijing to Shanghai, it's yet another one. This is the most fragmented booking process we ever went through. Other than a few moments of stress and frustration, the hunting for information, cross-referencing and having the occasional eureka moment is tons of fun, though; no complaints here. When in Rome... According to Lonely Planet, the Trans-Siberian is more or less one large picnic where everyone shares whatever they have with them with everyone else. I poked a few Russians about what typically German food they like and they came up with Frankfurters, mustard, and beer. Guess that's what we will be stuffing our backpack with once everything else is packed. And snuff; apparently it's common courtesy to offer your snuff to other men you meet in Mongolia and then snuff from their bottle. When in Rome... German snuff tends to be mixed with menthol, I suspect that will raise some eyebrows ;) No idea if there is any non-obvious social grease to bring to China; still working on that. Initially, we wanted to tour Mongolia's main attractions, but we would either have to sit in a car for days on end or go by plane. Driving for days in between a solid week(!) spent on trains is not very appealing and neither is breaking our personal ground travel record by cheating flying. Thus, we decided to take the eco-tourism route: After a two-hour orientation course in Mongolian behaviour and manners, we will travel a few hundred kilometers by public bus, be picked up by a local, driven out into the outback and ride from ger to ger by camel and horse before returning to Ulan Bator by bus. The stays at the gers are immersive, if short; we will live with the families, help them do their daily chores, and visit local places of interest in between. Oddly enough, I am looking to forward to the bus ride even more than to the gers. At the gers, at least one person is able to speak basic English. Not so with the local bus. People on the various trains will be used to strangers, as are the families at the gers. People on the bus are, most likely, not. Communicating with hand, foot, phrasebook, and a smile will, hopefully, be a lasting experience. If time permits, we will try to visit kiva.org borrowers while in Ulan Bator; this is something we never had a chance to do before and it's been on the bucket list for some time, now. As an aside, the local agent in Mongolia asked us if our sleeping bags are rated down to -26 degrees Celsius. I do hope he was joking... On the privileges of living in a first-world country... Travelling by horse means travelling light. Not a problem for us as we pack light by default and still manage to bring everything from Swiss Tool to water bottles, zip ties, medkit, and everything in between. Still, there's one major thing I have been taking for granted all my life: electricity. Being without any source of power for four to five days is... challenging. Even more so as every single gadget I rely on uses a different type of battery. Flashlight: CR123A and AA; GPS: Ni-MH AA rechargeables and Lithium AAA primaries respectively; Laptop, cameras, cell phone, ebook reader, Nintendo DS and MP3 player: proprietary. I am starting to truly understand why NATO has a hard rule of allowing AA-powered devices, only; planning spares is a pain. Obviously, I am focusing on flashlights, GPS, and, above all, cameras. Battery grips with AA adapters to the rescue! Another even more unsettling realization came when I asked if it would be possible to have boodog (vegetarians/vegans: don't click). Mongolians do not usually prepare boodog in spring as the animals which survived the harsh winter need to fatten and breed before being killed for food. All my life, I have never even once considered the remote possibility of not being able to slaughter domesticated animals due to outside constraints. Mongolian nomads need their animals. This is not about wanting to do things one way or another; this is about survival. Quite fascinating, and humbling, to think about. I am incredibly privileged and, as you are reading this, so are you. An exercise for the reader If you have any information regarding: please let me know. As usual, but just a little bit more than usual, comments are appreciated.

5 April 2012

Michal &#268;iha&#345;: Why weblate?

I still get lot of questions why I wrote Weblate instead of staying with some existing solution. Most of the reasons were already addressed in previous posts, where I looked for Pootle alternative or announced Weblate, but still it might be worth to summarize reasons once more. Git integration None of existing tools integrate with distribute version control, while I find this integration somewhat obvious and necessary. Having translation system as one of remote branches makes things really easy to handle for developers. Merging translations is nothing more than git merge. Attribution to translator I think it's fair to request that translator gets appropriate attribution for his contribution. Once we rely on Git, it is quite obvious that he will be author for Git commit doing changes. This way you can find author purely with Git and don't need to look for other tools. Handling of several branches In phpMyAdmin we heavily use branches to maintain stable releases. On the other side we want to be able to push translations to both stable and testing branches without translators having to do the work twice. Weblate does automatically push changes to all branches. Well this is not limited to branches but to subprojects as they don't have to be only branches, they can be also just related projects like with Gammu and Wammu. Displaying context Most of tools I tried fail to provide way to show related source code even when Gettext po files contain such references. With Weblate this is just matter of configuring URL of viewer, what nicely integrates with existing services like Github. Sane merging of uploaded translations With Pootle, it is possible to upload incomplete po file and it will luckily take it and replace existing one. This is quite dangerous as you would not notice it is incomplete unless you do some translation validation before release. Weblate does merge uploaded translations unit by unit, what makes it slower, but generates minimal diff and works also across branches. Consistency checks There are always things which can go easily wrong in the translation like format strings and translation tool should catch these as early as possible. Weblate semi-enforces checks - if check has failed, you will be redirected back to previous translation to possibly correct it (or mark check for ignoring). At current state checks are not as good as I would like to, but they are completely modular and can be easily extended and improved. Bonus points While working on Weblate I realized that some things could be done even better than I originally anticipated. For example with Github web hooks, it is possible to receive notifications on upstream repository updates and merge changes as they appear, leaving to really live translation process. Git allows to write custom merge drivers and this can be utilized for less merge conflicts on po files (they are usually caused by timestamps in headers), so that's another handy you get with Weblate. Of course there is much more built in, just check the documentation.

Filed under: English Weblate 3 comments Flattr this!

1 April 2012

Joey Hess: ls: the missing options

I'm honored and pleased to be the person who gets to complete ls. This project, begun around when I was born, was slow to turn into anything more than a simple for loop over a dirent. It really took off in the mid and late 80's, when Richard Stallman added numerous features, and the growth has been steady ever since. But, a glance at the man page shows that ls has never quite been complete. It fell to me to finish the job, and I have produced several handy patches to this end: The only obvious lack now is a -z option, which should make output filenames be NULL terminated for consuption by other programs. I think this would be easy to write, but I've been extermely busy IRL (moving lots of furniture) and didn't get to it. Any takers to write it? Due to the nature of these patches, they conflict with each other. Here's a combined patch suitable to be applied and tested.
diff -ur orig/coreutils-8.13/src/ls.c coreutils-8.13/src/ls.c
--- orig/coreutils-8.13/src/ls.c    2011-07-28 06:38:27.000000000 -0400
+++ coreutils-8.13/src/ls.c 2012-04-01 12:41:56.835106346 -0400
@@ -270,6 +270,7 @@
 static int format_group_width (gid_t g);
 static void print_long_format (const struct fileinfo *f);
 static void print_many_per_line (void);
+static void print_jam (void);
 static size_t print_name_with_quoting (const struct fileinfo *f,
                                        bool symlink_target,
                                        struct obstack *stack,
@@ -382,6 +383,7 @@
    many_per_line for just names, many per line, sorted vertically.
    horizontal for just names, many per line, sorted horizontally.
    with_commas for just names, many per line, separated by commas.
+   jam to fit in the most information possible.
 
    -l (and other options that imply -l), -1, -C, -x and -m control
    this parameter.  */
@@ -392,7 +394,8 @@
     one_per_line,      /* -1 */
     many_per_line,     /* -C */
     horizontal,            /* -x */
-    with_commas            /* -m */
+    with_commas,       /* -m */
+    jam            /* -j */
    ;
 
 static enum format format;
@@ -630,6 +633,11 @@
 
 static bool immediate_dirs;
 
+/* True means when multiple directories are being displayed, combine
+ * their contents as if all in one directory. -e */
+
+static bool entangle_dirs;
+
 /* True means that directories are grouped before files. */
 
 static bool directories_first;
@@ -705,6 +713,10 @@
 
 static bool format_needs_type;
 
+/* Answer "yes" to all prompts. */
+
+static bool yes;
+
 /* An arbitrary limit on the number of bytes in a printed time stamp.
    This is set to a relatively small value to avoid the need to worry
    about denial-of-service attacks on servers that run "ls" on behalf
@@ -804,6 +816,7 @@
    "escape", no_argument, NULL, 'b' ,
    "directory", no_argument, NULL, 'd' ,
    "dired", no_argument, NULL, 'D' ,
+   "entangle", no_argument, NULL, 'e' ,
    "full-time", no_argument, NULL, FULL_TIME_OPTION ,
    "group-directories-first", no_argument, NULL,
    GROUP_DIRECTORIES_FIRST_OPTION ,
@@ -849,12 +862,12 @@
 static char const *const format_args[] =
  
   "verbose", "long", "commas", "horizontal", "across",
-  "vertical", "single-column", NULL
+  "vertical", "single-column", "jam", NULL
  ;
 static enum format const format_types[] =
  
   long_format, long_format, with_commas, horizontal, horizontal,
-  many_per_line, one_per_line
+  many_per_line, one_per_line, jam
  ;
 ARGMATCH_VERIFY (format_args, format_types);
 
@@ -1448,6 +1461,9 @@
       print_dir_name = true;
      
 
+  if (entangle_dirs)
+      print_current_files ();
+
   if (print_with_color)
      
       int j;
@@ -1559,6 +1575,7 @@
   print_block_size = false;
   indicator_style = none;
   print_inode = false;
+  yes = false;
   dereference = DEREF_UNDEFINED;
   recursive = false;
   immediate_dirs = false;
@@ -1644,7 +1661,7 @@
      
       int oi = -1;
       int c = getopt_long (argc, argv,
-                           "abcdfghiklmnopqrstuvw:xABCDFGHI:LNQRST:UXZ1",
+                           "abcdefghijklmnopqrstuvw:xyABCDFGHI:LNQRST:UXZ1",
                            long_options, &oi);
       if (c == -1)
         break;
@@ -1667,6 +1684,10 @@
           immediate_dirs = true;
           break;
 
+   case 'e':
+          entangle_dirs = true;
+     break;
+
         case 'f':
           /* Same as enabling -a -U and disabling -l -s.  */
           ignore_mode = IGNORE_MINIMAL;
@@ -1697,6 +1718,10 @@
           print_inode = true;
           break;
 
+   case 'j':
+     format = jam;
+     break;
+
         case 'k':
           human_output_opts = 0;
           file_output_block_size = output_block_size = 1024;
@@ -1765,6 +1790,10 @@
           format = horizontal;
           break;
 
+   case 'y':
+     yes = true;
+     break;
+
         case 'A':
           if (ignore_mode == IGNORE_DEFAULT)
             ignore_mode = IGNORE_DOT_AND_DOTDOT;
@@ -2510,7 +2539,7 @@
       DEV_INO_PUSH (dir_stat.st_dev, dir_stat.st_ino);
      
 
-  if (recursive   print_dir_name)
+  if ((recursive   print_dir_name) && ! entangle_dirs)
      
       if (!first)
         DIRED_PUTCHAR ('\n');
@@ -2526,7 +2555,8 @@
   /* Read the directory entries, and insert the subfiles into the  cwd_file'
      table.  */
 
-  clear_files ();
+  if (! entangle_dirs)
+     clear_files ();
 
   while (1)
      
@@ -2615,7 +2645,7 @@
       DIRED_PUTCHAR ('\n');
      
 
-  if (cwd_n_used)
+  if (cwd_n_used && ! entangle_dirs)
     print_current_files ();
  
 
@@ -3464,6 +3494,10 @@
       print_with_commas ();
       break;
 
+    case jam:
+      print_jam ();
+      break;
+
     case long_format:
       for (i = 0; i < cwd_n_used; i++)
          
@@ -4418,6 +4452,24 @@
   putchar ('\n');
  
 
+static void
+print_jam (void)
+ 
+  size_t filesno;
+  size_t pos = 0;
+
+  for (filesno = 0; filesno < cwd_n_used; filesno++)
+     
+      struct fileinfo const *f = sorted_file[filesno];
+      size_t len = length_of_file_name_and_frills (f);
+
+      print_file_name_and_frills (f, pos);
+      pos += len;
+     
+  putchar ('\n');
+ 
+
+
 /* Assuming cursor is at position FROM, indent up to position TO.
    Use a TAB character instead of two or more spaces whenever possible.  */
 
@@ -4627,11 +4679,13 @@
   -D, --dired                generate output designed for Emacs' dired mode\n\
 "), stdout);
       fputs (_("\
+  -e, --entangle             display multiple directory contents as one\n\
   -f                         do not sort, enable -aU, disable -ls --color\n\
   -F, --classify             append indicator (one of */=>@ ) to entries\n\
       --file-type            likewise, except do not append  *'\n\
       --format=WORD          across -x, commas -m, horizontal -x, long -l,\n\
                                single-column -1, verbose -l, vertical -C\n\
+                               jam -j\n\
       --full-time            like -l --time-style=full-iso\n\
 "), stdout);
       fputs (_("\
@@ -4667,6 +4721,8 @@
   -i, --inode                print the index number of each file\n\
   -I, --ignore=PATTERN       do not list implied entries matching shell PATTERN\
 \n\
+  -j                         jam output together, makes the most of limited\n\
+                             space on modern systems (cell phones, twitter)\n\
   -k                         like --block-size=1K\n\
 "), stdout);
       fputs (_("\
@@ -4733,6 +4789,7 @@
   -w, --width=COLS           assume screen width instead of current value\n\
   -x                         list entries by lines instead of by columns\n\
   -X                         sort alphabetically by entry extension\n\
+  -y                         answer all questions with \"yes\"\n\
   -Z, --context              print any SELinux security context of each file\n\
   -1                         list one file per line\n\
 "), stdout);
It remains to be seen if multi-option enabled coreutils will be accepted into Debian in time for the next release. Due to some disagreements with the coreutils maintainer, the matter has been referred to the Technical Committee (Flattr me) Traditionally new ls contributors stop once enough options have been added that they can spell their name, in the best traditions of yellow snow. Once ls -richard -stallman worked, I'm sure RMS moved on other other more pressing concerns. The current maintainer, David MacKenzie, was clearly not done yet, since only ls -david -mack worked. But he was being slow to add these last few features, and ls was very deficient in the realm of spelling my name (ls -o -hss .. srsly?), so I took matter into my own hands in the best tradition of free software.

26 March 2012

Kees Cook: keeping your process unprivileged

One of the prerequisites for seccomp filter is the new PR_SET_NO_NEW_PRIVS prctl from Andy Lutomirski. If you re not interested in digging into creating a seccomp filter for your program, but you know your program should be effectively a leaf node in the process tree, you can call PR_SET_NO_NEW_PRIVS (nnp) to make sure that the current process and its children can not gain new privileges (like through running a setuid binary). This produces some fun results, since things like the ping tool expect to gain enough privileges to open a raw socket. If you set nnp to 1 , suddenly that can t happen any more. Here s a quick example that sets nnp, and tries to run the command line arguments:
#include <stdio.h>
#include <unistd.h>
#include <sys/prctl.h>
#ifndef PR_SET_NO_NEW_PRIVS
# define PR_SET_NO_NEW_PRIVS 38
#endif
int main(int argc, char * argv[])
 
        if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0))  
                perror("prctl(NO_NEW_PRIVS)");
                return 1;
         
        return execvp(argv[1], &argv[1]);
 
When it tries to run ping, the setuid-ness just gets ignored:
$ gcc -Wall nnp.c -o nnp
$ ./nnp ping -c1 localhost
ping: icmp open socket: Operation not permitted
So, if your program has all the privs its going to need, consider using nnp to keep it from being a potential gateway to more trouble. Hopefully we can ship something like this trivial nnp helper as part of coreutils or similar, like nohup, nice, etc.

2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

20 March 2012

Russell Coker: An Introduction to Android

I gave a brief introductory talk about Android at this month s LUV meeting. Here are the slides with a brief description. All the screen-shots were made on a Samsung Galaxy S running Cyanogenmod version 7.1 [1] (Android version 2.3.7). With that build of Cyanogenmod you can press the power button for about 1.5 seconds to get a menu which gives an option to take a screen shot. The aim of the talk was to give an overview of what Android can do. I also gave some random commentary about Android such as explaining why it doesn t make a good phone. Most of the pictures in this post have links to the Android applications in question. Essential and Important Apps picture of root shell access running df I started by explaining why having root access to your system is really important, including the issue of backing up an Android phone [2]. Cyanogenmod includes a terminal program which allows you to run su - . Running a shell as root isn t generally that useful, what you really want is to be able to run programs such as Titanium Backup which can only work properly if given root access. When you run an OS that allows root access you can run su - at a terminal prompt and you can also have an application use a GUI to request root access. I recommend rooting and modding an Android phone immediately after buying it. However that takes some time which is somewhat equivalent to money and is a significant hidden cost to purchasing an Android phone. picture of 3g watchdog bandwidth monitor The business models of telephone companies seem to involve hidding users for unexpected fees and extra fees for excess bandwidth can be really expensive. 3G Watchdog is one app that can monitor bandwidth and disable data transfers if too much is used. Onavo is an alternative that allows tracking data use on a per-application basis, but it only runs on Android 2.3.x while 3G Watchdog works on Android 2.1. Official EBay app searching for Samsung Android phones EBay has an official app which is handy for searching for items. So far I ve only used it to get price estimates and have used a PC for buying. K9 viewing my SE Linux mailing list email K9 seems to be the best MUA for Android. The MUA that ships with Android 2.1 isn t nearly as good and K9 is good enough that I didn t even bother testing the MUA from Cyanogenmod. The above picture shows a list of mail in my SE Linux folder. graph by opticron grapher The Opticron Grapher is a good graphing calculator. I won t claim it s the best because I didn t seriously test such programs, but for the basic tests I ve done it has worked well. Google map of LUV location The Google Maps client comes with every Android system, the above shows the location of the LUV meeting. Open Street Map location of the LUV meeting Osmand is an Android client for the Open Street Map project. Here is the web site for the Open Street Map project [3]. One significant advantage of OSM over the Google Maps is that OSM is free, the data is all contributed by users like Wikipedia. Another significant advantage is that you can download as much data as you need to your phone, for example the entire dataset for Australia is about 200M. Storing 200M of data on your phone is no big deal when you consider the availability of phones with more than 16G of storage and the ability to use a map when offline is a real benefit. So far I ve used Osmand while waiting for a train at an underground station and I plan to use it to track my progress the next time I m on a cruise. serval mesh networking and VOIP Serval is a mesh networking application for Android that supports VOIP phone calls and distributing messages and files. It s designed to be used in disaster areas, but there are lots of other potential uses of the technology. The Serval Project blog has an article about the presentation they gave at LCA 2012 [4]. periodic table
details about Titanium Periodic Droid is my favorite Periodic Table viewing program. tomfusion Au Weather forecast app The Tomfusion AU Weather forecast app seemed to be better than the one from the BoM last time I checked. It s probably the best weather app for Australia. screen-shot of LUV web site
LUV web site zoomed in The Opera Mini browser is often faster than other browsers because it uses a compressing proxy run by Opera. It s not so good for privacy though LUV page on Wikipedia I have been using Wapedia for browsing Wikipedia. Since giving the talk I discovered the Official Wikipedia browser from the Wikimedia foundation [5] which is a better fit for my needs and I ve uninstalled Wapedia. As an aside modern phones have 16G of storage or more and could easily have a copy of the entire English text of Wikipedia on internal storage. It would be good if someone like Jason King (who is known for work on stand-alone DVD images for Wikipedia) was to write an Android program to do this. Handy Apps Androsensor showing GPS, accelleration, and light intensity
Androsensor showing magnetic field, orientation, and battery Androsensor is a program to display output from most (all?) of the sensors on your phone. The results aren t as accurate as one would hope, for example Earth s gravity is 9.81m/s^2 not the 10.26m/s^2 my phone registered. But they are a useful indication. picture of Coke can, scanning the barcode
Google search on Coke can barcode
QR Code lookup of Facebook page The Zxing Barcode scanner is one of many programs that will scan barcodes with the camera in an Android phone. It can launch a Google search on a product code or open a URL from a
QR Code. The above pictures show it scanning a Coke can (the can and other background was displayed on the full screen before the screen capture program activated), doing a Google search on the can barcode, and looking up a QR code that was on an advertisment outside the LUV venue. picture from the bridge of the Dawn Princess in Tauranga NZ Cruise Cams allows you to download pictures from cruise ships. Some cruise ships have several cameras on different parts of the ship uploading pictures regularly so that people around the world can see what s happening. list of geo-caches near the park where I prepared most of my notes The c:geo opensource program allows you to get information on Geocaches and see a compass or map showing the location. This program has been getting an increasing number of features to do everything you might want to do related to Geocaching. The above picture shows some caches that are close to where I made the screen-shot, in a park a couple of Km from the meeting location. picture of Google Sky Map in the direction of Andromeda The Google Sky map uses augmented reality techniques to display stars in the direction that your phone is pointing along with their names and the names of the constellations. Marine Traffic showing ships near me
Marine Traffic showing the Pacific Sun highlighted on a Google Map
Marine Traffic showing details of the Pacific Sun
Photos of the Pacific sun taken by fans and shown by Marine Traffic The Marine Traffic program shows the locations of ships as well as lots of information about them. The above pictures show me discovering that the Pacific Sun was nearby, viewing it s location on Google Maps, seeing the details, and then viewing fan pictures. The developer s web site allows viewing all the same data without an Android phone [6]. Anyone can join the project by buying an Automatic Identification System (AIS) receiver and configuring a PC to take data from AIS and send it to the MarineTraffic.com servers. As an aside they seem to be missing coverage in western Victoria, so it would be good if someone near Apollo Bay or Warrnambool could install an AIS receiver and help out. Satellite map from Satellite AR Satellite AR uses augmented reality to show the location of satellites and other things in spare. Unfortunately the screen capture process turned off the camera as I had a sign advertising fast food positioned in an amusing location in the background. Shipmate overview of Dawn Princess
Shipmate map of Dawn Princess Shipmate publishes a set of programs giving information on cruise ships, they have one program for each cruise line. Above is the program for the Princess cruise line, the above pictures give information on the Dawn Princess. Unfortunately the program wasn t usable without net access when I tried to use it on a cruise ship. Games Air Attack HD game, clone of 1942 Air Attack HD is an entertaining game that demonstrates the capability of Android phones to run action games. Like many Android games it has a free version and paid versions if you want more. picture of Angry Birds Angry Birds is one of the most well known games for touch-screen devices. It has also spawned a huge line of merchandise. Labyrinth Lite
Labyrinth Lite from a different angle Labyrinth Lite is one of the many Android games based on the old mechanical game where you tilt a toy to roll a ball-bearing through a maze. It s free and is better than most of the free games in that genre. Minecraft Pocket Edtion Minecraft Pocket Edition allows you to play Minecraft on your phone. The demo version doesn t allow saving the game, you have to buy the game for about $5 if you want to do that. It also lacked the full features of the game last time I checked, it didn t have monsters. Paradise Island overview
Paradise Island details of Bungalow Paradise Island is one of many business simulation games for Android. It s more playable than most and has very detailed graphics, but the down-side is that it s a memory hog and will crash if you don t have enough RAM. It s one of the games that are free to download but encourage you to pay money to level up, for some players it s probably a very expensive game. Tower Raiders 2, Pratt was eaten by a Grue Tower Raiders 2 is one of the better tower defense games for Android. Related posts:
  1. Choosing an Android Phone My phone contract ends in a few months, so I m...
  2. Galaxy S vs Xperia X10 and Android Network Access Galaxy S Review I ve just been given an indefinite loan...
  3. Standardising Android Don Marti wrote an amusing post about the lack of...

14 March 2012

Richard Hartmann: Open Source Days 2012, the aftermath

Open Source Days Open Source Days were nice, especially the meating(sic) of and talking to old friends and new people. I got to discuss Mercurial with two of its developers which was very interesting indeed. It's funny how similar git and Mercurial are in some regards and how different in others. Overall, it feels a bit as if Mercurial is not quite as distributed as git. Its local, sliding revision index feels like disaster waiting to happen, to me. On the other hand, Mercurials ability to not check out large files to your local repository sounds very git-annex-ish, which is nice. Choice is good :) My talk about how to gitify your life (slides, no video) went extremely well. Within my time-slot, there were a total of six talks, and lunch(!), in parallel. About 300 people attended OSD on Sunday and of those, many were at booths, stands, eating, etc. My personal turnout was 50-70 listeners so I was well above mathematical average and from what I heard, my talk was the most-visited one during that time. Add the fact that several people asked me to put up the resource links up again after the talk so they could take pictures and I would say this talk really was a success. Ego-stroking? Yes. Vanity? Most likely. Being able to see that people can really want to learn about what you have to say? Priceless. It's a real pity that there was no video recording, but it's obviously too late to do anything about that. Joey Hess asked me to at least make an audio recording on my laptop, but that turned out to be a lot of useless white noise with faint mumbling in between. Maybe I should at least get a small microphone or recorder so I will always have an audio copy. I already did order a presentation clickie with built-in laser pointer so I won't have to rely on possibly non-existent conference infra, any more. If there's any interest, I may re-record the talk as audio, else I will just try to get another speaker slot at another conference, with video recording, and link that. Denmark Barring the airport, central station, bus, hotel, venue, and metro, I did not see anything of Copenhagen. That's a pity, but I didn't want to leave the conference early. To continue my recent tradition of dumping random observations about places I visit onto my blog, here goes:

Next.

Previous.