Search Results: "nate"

20 December 2012

Wouter Verhelst: Installing Debian to the network with NBD

As I've blogged about before, wheezy will be the first release since I've started maintaining NBD for Debian to feature NBD support in the installer. That is, you'll be able to install wheezy to the network, simply by adding "modules=partman-nbd" to the kernel command line: obligatory screenshot While I'm happy and, in fact, a bit proud about that fact, there are still a few minor issues that make the whole process a bit less smooth than it could be: In my plan to take over the world of network storage, I plan to work on that last point during the jessy release cycle, at least for x86 architectures; I envision a system which would mount /boot on NFS, and write pxelinux.cfg files, where the name of this config file would be configurable (so it can be configured to only boot the system on which the file was written, or to boot any system on the local network). If done well, it will also fix the point about installing the bootloader to local hard disk when doing a network installation, since we'll have a "install netbootloader" or some such option.

Russell Coker: Samsung Galaxy S3 First Review with Power Case

My new Samsung Galaxy S3 arrived a couple of days ago. Kogan added to the weirdness in their shipping that I documented in a previous post [1] by shipping the two phones on consecutive days with separate invoices for $789 each (as opposed to the $449 that I paid). This means that if customs were to track this down they could bust me for import duties (which apply whenever you order more than $1000 of electronics from another country) as multiple shipments in a small period of time are aggregated for tax purposes. It also means that it would be easy to defraud the tax office or an employer about the cost of the devices if I didn t blog about it Phone and Case Review back of power case with S3 installed next to back plastic of S3S3 and case next to each other To install the Samsung Galaxy S3 in the power case I bought from Kogan the back of the S3 has to be removed. One annoying implication of this is that when using the phone in it s case I will either have to carry the back of the phone with me or have no option of removing the case during the day. The phone has to slide in to the case from a small angle and if it s removed carelessly there seems to be a risk of breaking the power connector on the case. S3 on power case standedge view of S3 in power case stand The above photos show the kick-stand on the power case in use, I haven t yet had an opportunity to use the stand for anything other than photography and as it s made of thin plastic there s a reasonable chance that it will break off before I ever use it properly. One major deficiency of the power case is that it has no protection for the screen. The typical gel case will extend by about 2mm above the screen which means that you can place the phone screen-down on a table without a risk of scratching the screen. Also if a phone in a gel case is dropped on a hard surface it seems likely that the chance of the screen hitting a bump and cracking will be lower although I m not aware of any research regarding how well this theory works in practice. This combined with the fact that the power case is smooth and difficult to grip makes it quite unsuitable for regular use. I am concerned that the added mass of the power case would increase the risk of serious phone damage if it s dropped screen-down. I plan to buy a gel case ASAP which I will use most of the time. I will swap the gel case for the power case on the occasions when I expect to spend an extended period of time away from home or an office where I can charge my phone. The power case really should have been designed with a flange that extends above the level of the screen on all sites. It also shouldn t be smooth on the outside. Phone Review The Galaxy S3 equals the Galaxy S for the title of the most slippery phone I ve ever owned. This combined with the fact that the Galaxy S3 is slightly thinner and also 10% wider and longer makes it a more difficult phone to hold. It s a phone that really needs a case with a lot of grip. The camera is reasonable given the fact that any device that is 8.6mm thick will have a very small focal length and also a very small CCD for receiving light. The laws of physics just don t allow something the size of the Galaxy S3 to have a half-decent camera. My ideal mobile phone would be about the size of the Galaxy S3 with the power case, it would use the extra thickness for a big battery as well as a much longer focal length for the camera. I d like a phone to be about 16mm thick over most of it s area with a section that s 20mm for the camera. The burst shot mode (which allows you to take 20 pictures rapidly) is good for capturing certain types of relatively slow movement (EG someone s mouth moving when they talk). But it s no good at all if you try to pan the camera while doing burst mode. When I first started using the phone it gave me lots of hints about using various functions. This wasn t particularly useful for me as most things are obvious to anyone who has much experience with computers and the non-obvious things are probably going to be forgotten if you are doing something else when the help system prompts you. It will be useful for my sister in law who s getting the other phone, but the down-side of this is that I can t configure her phone before delivering it. She needs to use it first to see all the tips. The Galaxy S3 has a screen that s 4.8 diagonally which is slightly larger than the 4.0 screens of my previous Android phones (Sony Ericsson Xperia X10 and Samsung Galaxy S). That gives 44% more screen area which is a good thing. Unfortunately many apps seem to display the same amount of data, I had hoped that 44% more area and 140% more pixels would result in most apps displaying more data by default. Fortunately ConnectBot (my SSH client of choice) gives me 60 rows and 120 columns on the Galaxy S3 as opposed to the mere 31 rows and 80 columns on the Xperia X10. I don t use ConnectBot that often (a phone is a really bad ssh client no matter how you do it), but when I do it s really important and the extra space matters. The down-side of the larger screen for me is that the distance from the center of the palm of my hand to the end of my thumb is slightly less than 4.8 . This means that I can t hold the Galaxy S3 with one corner in the palm of my hand and still reach the far corner of the screen. So I am forced to hold the sides of the phone for one-handed use which means that it would be easier for it to slip out of my hand. A further complication is that the settings and back buttons are very close to the edge of the phone and can be hit by accident very easily. Hopefully a gel case will alleviate that. But I have never owned a hand-held device which was so difficult to hold in a stock configuration. According to AndroSensor (one of my favorite demo programs for showing useless things that a phone can do) the Galaxy S3 lacks support for measuring humidity and external temperature but it has all the other sensors. I think I can survive without knowing how hot and humid my pockets are, but it would still be nice to have every possible feature. The default partitioning of the phone storage is to have all internal storage used for applications and none used for a VFAT filesystem accessible over USB. It s nice that I won t run out of application storage space (a problem I ve had with every other Android device) but it does mean that I ll have to buy a micro SD card before I can use the phone properly. Micro SD storage costs about $1.50 per gig in the size range from 8G to 32G, it s not particularly expensive but it s annoying to have the phone not be fully usable right from the start. Without using a SD card I have to use gphoto2 to download pictures from the phone so it s fully usable in every way. But the cost of the power case, the gel case I m about to buy, and an SD card make the phone less of a bargain. The phone has more RAM, a faster CPU, and Android 4.1. These are all really good things, but nothing worth writing about in a review. Conclusions The power case is very poorly designed, I probably wouldn t buy it if I knew then what I know now. For one-handed phone use by someone like me with longer than average fingers a 4.0 phone is about the largest possible. This doesn t mean that phones should stay small, but it does mean that Apple weren t being entirely silly when they released all the 3.5 iPhones. Presumably a large portion of the population would find a 3.5 phone to be as usable as a 4.0 phone is for me. It now seems to me that as a 4.8 screen is going to be impractical for one-hand use by almost everyone the phone designers should concentrate more on larger phones similar to the Samsung Galaxy Note. I think that the 4.8 screen size may be ideal for some young children who could use it the way an adult might use a 7 tablet an Android phone seems to be a relatively common present for children in the 6-12 age range. Generally I think that no-one should ever buy a new phone or tablet without trying one of the same size. While I am happy with the S3 it really doesn t work in the way I expected. Having 2.9* as many characters on screen for ConnectBot is a massive benefit. If I ever write another magazine article on a hand-held device the extra size and resolution will really help if I found a suitable editor (please make suggestions in the comments). For the WordPress client to write long blog posts on it then the screen could potentially be reasonably useful, but so far it seems that it s more suited to reviewing posts and comments than to writing posts. Generally the Samsung Galaxy S3 is a good phone and I think it s great value for money when it s less than $500.

18 December 2012

Russ Allbery: Review: Liars and Outliers

Review: Liars and Outliers, by Bruce Schneier
Publisher: John Wiley
Copyright: 2012
ISBN: 1-118-14330-2
Format: Hardcover
Pages: 285
One of the perils of buying a book written by a blogger one reads regularly is that the book may be little more than a rehashing of their blog, with insufficient original material to warrant the time investment. Sometimes it's still nice to support them financially, but it may not make sense to read the book. I've been following Schneier's blog for years (as should anyone with an interest in security), including through the entire process of writing Liars and Outliers, and was a bit worried that might be the case here. Thankfully, I can reassure any other worried potential readers that is not the case. This is substantial new material establishing a firm framework for thinking about incentives and controls in any society or organization. Liars and Outliers talks about security mechanisms, but it's not, at its core, a book about security. Rather, it's a book about incentives, order, and how order is established. It's a comprehensive reductionist analysis of how societies create predictability and compliance to allow us to trust other people who we have never met before and will never meet again. It's a unique (at least in my experience) combination of anthropology, sociology, security analysis, and political science. Schneier cuts across fields in an idiosyncratic but illuminating way that reminded me of (on an entirely separate topic) Jane Jacobs. This is not a prescriptive book, nor is a collection of answers, solutions, or even deep analysis of particular problems. Rather, it's an attempt to construct a general framework for analyzing societal dilemmas: conflicts between individual desires and social good, how those conflicts are resolved, and how societies can weigh the scales and influence the statistical outcomes. The closest Schneier comes to telling the reader how to solve problems is a checklist, at the end of the book, for designing effective societal pressures. Its primary contribution is vocabulary and structure. It also passes one of my litmus tests for any book about human behavior: Schneier complicates, broadens, deepens, and expands understanding, and points out complex interactions and complex feedback in effects we're inclined to consider simple, rather than simplifying or eliminating human complexity. One point I found refreshing about this book is that Schneier is scrupulous in refusing to define either society or individuals as good or bad, to the point of carefully defining terminology used in all of the social dilemmas. Following societal rules is called compliance; not following those rules is called defection. In some cases, defection is morally correct (Schneier's most frequent example is in the civil rights struggles of the 1960s in the United States). In all cases, social pressures are tools, which can be used to encourage compliance with moral or immoral systems, and which are deployed by totalitarian dictatorships and utopian communes alike. Schneier explicitly puts out of scope for this book the questions of how societal goals should be determined, how they change, and whether any given societal rule or interest is moral or immoral. He focuses, rather, only on the mechanisms, with a primary goal of informing and deepening debates over how best to encourage behavior that societies want to encourage and discourage behavior societies want to discourage. He also emphasizes discourage, as opposed to eliminate. Early on, Schneier shows some of the results of game theory, as well as basic common sense, that indicate that no healthy society can totally eliminate defection. Not only would that stifle valuable and important reform, such as changes in civil rights, but the degree of pressure required is immense. Defectors are natural and will always exist, and are sometimes valuable and necessary. Rather, the goal of a society is to reduce defectors to a level where most people can ignore their existence most of the time, a state that leads to the level of risk and trust required to have a functioning and healthy society. The word society, similarly, is intentionally broad, and can refer to just about any collection of people, from a circle of friends to a corporation, institution, or country. However, as Schneier points out early in the book, small societies rarely need much in the way of formal pressure and appear almost magically self-governing. That's the first property that he disassembles, resulting in a general classification of societal pressures into four categories: moral, reputational, institutional, and security systems. The last is an odd category that's partly orthagonal to the other three. Moral pressure is internalized conscience: the normal tendency of nearly everyone to follow their own moral code, a code that's at least partly constructed and certainly heavily influenced by the surrounding society. Reputational pressure is, in a sense, externalized morality: it's the informal reactions of others around one to one's past actions. Included in reputational pressure is shunning of every kind, from cutting off a friendship to boycotts against corporations, but it also includes confrontation from another member of one's society, the more subtle effects of our individual desires to be liked and respected, and all the various aspects of "face", honor, and respect within a community. Small communities are frequently self-governing, in Schneier's model, because they're small enough that moral and reputational pressures are sufficient and no other pressures are required. We're so used to applying moral and reputational pressure to other humans almost unconsciously that we sometimes don't even notice its existence, leading to that "magical" self-governing property. But Schneier puts pressures in a sequence: pressures that work extremely well with small groups often don't scale. Moral pressure works best with small groups and reputational pressure with somewhat larger groups, but when societies scale beyond the limits of reputational pressure when, for example, one frequently interacts with people whose reputations are unknown to you and whose subsequent opinion of you will not be relevant institutional pressure is required to force compliance. Institutional pressure is the sort of pressure that we all tend to think of first when we look for ways to enforce rules: laws, policies, contracts, and other codes of behavior that carry with them formal punishments and some enforcement mechanism. But even in societies so large that institutional pressures are frequently required, such as whole countries, moral and reputational pressures still exist and are extremely important. One of Schneier's most interesting points is his analysis of how institutional pressures can paradoxically undermine reputational and moral pressures, resulting in more defection than if the institutional pressure hadn't existed. This is just the basic framework of Schneier's analysis, hopefully giving you a feel for the structure of the book. He goes much deeper into the complicated interactions between the various levels of pressure, and then dives into an extensive look at competing societal dilemmas: cases where there is more than one society in play simultaneously, possibly demanding contradictory actions. Liars and Outliers also includes a wonderful analysis of organizational entities within the same framework, including their much-different reactions to moral, reputational, and institutional pressures. One of the most cogent analyses of the difficulties of regulating both corporations and governmental institutions falls out of that analysis, once one looks at them in light of Schneier's basic framework. Pressures quickly become complex and multi-layered, and human reactions to pressures are frequently counter-intuitive. Schneier draws extensively on game theory to show that some counter-intuitive responses are actually emergent properties of logical analysis of the situation, but that others are more uniquely human and have little or nothing to do with a mathematical cost-benefit analysis. I haven't even mentioned his discussion of security systems, and how they can extend moral, reputational, and institutional pressures, as well as add a new type of pressure (making a defecting action impossible) that scales even better than institutional pressures. Liars and Outliers has all of the supporting infrastructure you would expect in a scholarly book: notes, extensive references, and a good index. I suspect it will end up being used as at least additional reading in college classes. The notes are, unfortunately, end notes, making the full context of the book much harder to read than was necessary, but at least Schneier does separate the notes from the references so that one doesn't chase notes for further explanation and find a simple citation. As with any book like this, one always wishes it could end in a simple prescription to fix everything, but of course it doesn't. But that's also a measure of a good scholarly work. Human and organizational motivations are complex and tricky, and any framework for analyzing them needs to be able to represent that complexity. Schneier here has constructed a very powerful one, one that I started using in discussions before I'd even finished the book. Perhaps the most valuable contribution of Liars and Outliers to public discussion is clear terminology and categories, which can be of great help in finding the core components of a problem. Liars and Outliers can be slow going, particularly early on when Schneier is still defining terms and setting up the background of his analysis. One can get a bit tired of the analysis matrices of societal dilemmas. But stick with it through the groundwork, since the analyses of competing societal dilemmas and of the impact of societal pressures on organizations are exceptional. Highly recommended, particularly for anyone who is designing or implementing societal pressures: managers, political activists, or anyone in a security-related field. Rating: 8 out of 10

17 December 2012

Vincent Sanders: In fact, we started off with two or three different shells and the shell had life of its own.

With apologies to Ken Thompson, this is a list derived from the wafflings residents of a certain UK IRC channel. The conversation happened several years ago but I was reminded of it again today from a conversation on that same channel.

No names have been kept to project the guilty.


/bin/bush - a shell that steals and lies?
/bin/cash - displays adverts before each prompt
/bin/crash - Microsoft shell
/bin/mash - requires libsausage.so
/bin/hush - terse output
/bin/irish - only shell that finds jerkcity funny
/bin/rush - ber-optimised for speed, but might not actually work correctly
/bin/flash - proprietary shell that displays adverts and cartoons and hangs periodically
/bin/welsh - whinges about people not using it, but then steals features from other shells in order to actually make sense
/bin/rehash - never actually runs any program, just uses markov chains to construct output from stuff it has seen before.
/bin/wash - will remove lint in your scripts if you let it
/bin/parish - prays for the successful exit of every command
/bin/punish - symlink to /bin/csh
/bin/sheepish - apologises when $? is not zero
/bin/diminsh - decrements all result codes by one
/bin/lavish - and you thought bashisms were bad...
/bin/brainwash - once you've tried it, you'll believe it's the only shell in existence
/bin/hoggish - written in Java
/bin/reversepolish - arguments order different in go must
/bin/ganesh - no subprocess dares exit zero for fear of being removed from the system
/bin/roguish - every day is April 1st as far as it is concerned
/bin/macintosh - it's shiny, has *loads* of things you think you can poke at, yet it only actually responds to a single key on the keyboard and has no useful features for fear of confusing the user
/bin/lush - garbles output from processes like a drunkard
/bin/snobbish - you're not good enough to use it
/bin/thrush - it itches
/bin/vanquish - kills processes mercilessly
/bin/tush - pert
/bin/fetish -> /bin/zsh
/bin/skirmish - multi-user shell
/bin/whiplash - gets invoked if you break too hard
/bin/newsflash - BREAKING NEWS pid 1234 terminated
/bin/mulish - DJB does Shell
/bin/hsilop - arguments order different in go must
/bin/whitewash - government approved
/bin/trish - for tri-state hardware
/bin/jdsh - it sucks...
/bin/flesh - optimised for viewing p0rn

Ian Campbell: Foreign Chroots with schroot and qemu

Since I've been working on qcontrol I've had need of a way to build packages in an ARM environment. Since all my native ARM hardware is "production" (at least so far as something can be production in a home environment) I prefer to avoid installing development tools on them. When I started looking into this all my ARM systems ran from relatively small flash devices which also ruled out the possibility of a development chroot. After search the web a bit I discovered that it is possible to use qemu-user-static together with [[!binfmt-support ]] to create cross architecture chroots which can be entered mostly transparently. It is also possible to combine this with schroot and sbuild which makes the day to day use pretty much the same as a native or biarch chroot. Since I've now had to rediscover how to do this twice (once for Sid and then again when I realized I also needed a Wheezy environment) I thought I should write it down for the benefit of my future self. First things first lets install the various packages which we need. As well as debootstrap to create the chroot we need qemu-user-static and binfmt-support to allow us to run ARM binaries:
$ sudo apt-get install debootstrap qemu-user-static binfmt-support
I'm using schroot's lvm-snapshot mode so we need an LVM volume, formatted with a filesystem and mounted in a temporary location:
$ sudo lvcreate -L8G -n sbuild-wheezy-armel /dev/VG
$ sudo mkfs.ext3 /dev/VG/sbuild-wheezy-armel
$ sudo mount /dev/VG/sbuild-wheezy-armel /mnt/
Next we need to create the chroot. For this we use debootstrap in foreign mode which does a 2 stage setup, firstly by using the --foreign option:
$ sudo debootstrap --variant=buildd --include=fakeroot,build-essential \
        --arch=armel --foreign \
        wheezy /mnt http://<DEBIAN-MIRROR>/debian/
Now we (temporarily) copy the qemu-arm-static binary into the chroot (at the place where binfmt-support expects it), change into the chroot and run debootstrap's second stage to finish the basic chroot setup:
$ sudo cp /usr/bin/qemu-arm-static /mnt/usr/bin/
$ sudo chroot /mnt
(chroot)# ./debootstrap/debootstrap --second-stage
(chroot)# exit
Now that we have a basic chroot in place it needs a little bit of tailoring. I've found the easiest way to do this is to use sbuild-createchroot and then tweak the result:
$ sudo sbuild-createchroot --arch=armel --foreign --setup-only \
    wheezy /mnt http://<DEBIAN-MIRROR>/debian/
If we were building a native chroot then we could have skipped the debootstrap stuff and let sbuild-createchroot handle things. Unfortunately this doesn't seem to play nicely with --foreign. Now we can remove the qemu-arm-static binary from the chroot. schroot will automatically bind mount the version from the host into the chroot which ensures it remains up to date. We can also unmount the chroot since we will let schroot manage that from here on:
$ sudo rm /mnt/usr/bin/qemu-arm-static
$ sudo umount /mnt
Next we need to tweak the resulting schroot/sbuild configuration since sbuild-createchroot assumes the use of schroot's directory mode. First remove the symlink to /mnt which it created:
$ sudo rm /etc/sbuild/chroot/wheezy-armel-sbuild
Lastly we need to edit /etc/schroot/chroot.d/wheezy-armel-sbuild-<SUFFIX> (where is randomly generated) and reconfigure it to use lvm-snapshot mode, by apply the following changes:
 [wheezy-armel-sbuild]
-type=directory
-directory=/mnt
+type=lvm-snapshot
+device=/dev/VG/sbuild-wheezy-armel
+lvm-snapshot-options=--size 2G
 description=Debian wheezy/armel autobuilder
 groups=root,sbuild
 root-groups=root,sbuild
 profile=sbuild
That's it, we now have a chroot which we can trivially invoke via sbuild to build armel packages:
$ sbuild --arch=armel --dist=wheezy <... other options...>
We can also use sbuild-update to keep the chroot up to date:
$ sudo sbuild-update -udcar wheezy-armel
You can also use schroot to login to the chroot directly, although for that case you'll likely want to create a different chroot using schroot's directory or block-device mode instead lvm-snapshot and drop some of the sbuild specific setup. I expect I'll blog about this configuration in some more detail at some point in the future since I'll want to document it to give a more convenient build environment for Xen on ARM than the build farm which we have at work.

Dominique Dumont: Be wary of optimised for devices

Hello This story began with linphone having a weird behavior: every now and then, the mouse pointer would become stuck: the pointer moves, but clikcking on it always activate the same widget. Since this always happened in the middle of phone conference done for work, this was infuriating. Long story short, I finally found that my plantronics USB headset was responsible for this weird behavior. This headset is connected to the USB bus and is seen by the computer as a USB sound card. The bug was triggered by pressing the mute button provided by the headset. Suspicious of this usb device, I used lsusb to find the features provided by the heaset: $ lsusb -v -s 1:3 2>&1 grep InterfaceClass
bInterfaceClass 1 Audio
bInterfaceClass 1 Audio
bInterfaceClass 1 Audio
bInterfaceClass 1 Audio
bInterfaceClass 1 Audio
bInterfaceClass 3 Human Interface Device
Audio devices were expected, but why a HID device ? This device is a headset, not a keyboard or a mouse Testing further, I also saw some number poping up on my screen whenever I pressed the the mute button. The only way to get the mouse back was to unplug the headset. A quick google search gave a solution to setup X11 to ignore the headset. Problem solved. But this did not answer the question regarding the HID device. Then I got a hint in the form of a sticker glued on the headset USB plug: Optimized for Microsoft Lync . This page gave the answer: an optimised for Lync device provides mute/unmute across PC and device . I can only guess that when the mute button is pressed, some data is sent from the HID interface. Unfortunately, the window manager does not like to be on the receiving end of this data. The moral of this story is: optimized for something actually means not standard .
ok. There s a bug somewhere. The comments of this blog have convinced me that I went too far with the moral of this story. It s now overstriked instead of plainly remvoed so the comments still make sense. Thanks everybody for the constructive comments.
All the best
Tagged: wtf

Jaldhar Vyas: Heads Up: Latest Dovecot Package Loses Features

Sometimes it's easy to forget that Debian's unstable branch is...unstable. Unfortunately today I have to remind you of that unpleasant fact. Dovecot 2.1.7-6 which I just uploaded has removed some features present in -5. The reason for this is that I am trying to get dovecot in shape for Wheezy. Unfortunately, as I have been severely pressed for time these past few months, the process did not take place in quite as orderly a fashion as I had hoped. My -5 packages were deemed by the release managers to have too drastic changes for this stage of the pre-Wheezy freeze and this is an attempt to make things more acceptable to them. Hopefully this version will pass into testing without incident and as soon as it does, I will prepare -7 which will put these features back. So if you need any of them, you are advised to skip -6 and stay at -5 until that happens. If you are following testing there is another possible problem you might encounter. When -1 of this version was uploaded, certain extra conf files were accidently omitted from the package. This is also true of -2 which is currently in testing. It was fixed in -3 but that version (or -4 or -5) never made it into testing so -6 is potentially the first time you might be dealing with the conf file fix. If you have manually made changes to files in /etc/dovecot/conf.d make sure you have backups because they may get overriden (as dpkg will be unaware at that point that they are conffiles.) This is a one time only packaging error and will not be a problem going forward but I apologize in advance for any inconvenience caused. And this is a good time to ask anyone interested in Dovecot on Debian (and Ubuntu etc.) to volunteer to help maintain it. It is getting more and more popular at a time when I can give it less and less attention. It is vital that an important package like this needs is team-maintained and it kind of is but so far other people who have expressed an interest in joining the team have been similarly strapped for time so we still need more recruits. All the code is publically accessible at git://git.debian.org/git/collab-maint/dovecot.git and I would be happy to sponsor any non-DD/DMs who need it.

16 December 2012

Joachim Breitner: Circle Packing

For an upcoming introductory Haskell talk of mine (next Tuesday in Karlsruhe please come and join if you will) I was looking into data visualization as a example application. With libraries like gloss, getting some nice vector graphics up and running within one talk is quite possible. For some reason, I want to arrange multiple circles of varying size so that they do not overlap and use as little space as possible. My first approach was to use general purpose optimizers such as cmaes, Numeric.GSL.Minimization from the hmatrix package and force-layout. I especially liked the interface of cmaes, which is able to minimize a function of, say, type [(String, Double, Double)] -> Double by automatically finding the values of type Double in the input, as described by Takayuki Muranushi. That would have looked great in the talk. Unfortunately, none of these libraries gave sufficient good results in a reasonable amount of time for this problem. I then reviewed some of the academic literature on the circle packing problem and there were some algorithms proposed, but no simple one and none of the papers had a concise pseudo-code description that I could transfer to Haskell. So eventually I set out to implement a relatively dump and slow greedy algorithm myself: I place one circle after another, starting with the largest, and among the valid positions where a new circle touches two already placed circles, I choose the positions closest to the center of mass. The result, available in the circle-packing package, looks surprisingly well, here visualized using the diagrams library (See the documentation of Optimisation.CirclePacking for the code used to generate that image):
Now showing just this image is not a very good way to demonstrate my code. A few years ago, I might have created a CGI script that would dynamically generate images based on your input. But not in 2012: Since my code is pure Haskell without fancy features like type classes, I can use the fay compiler to generate JavaScript code from it. Add a little Haskell code to interact with the HTML5 canvas and now you can interactively try out circle-packing in your browser. Oh, and while I am talking about neat tricks: You can put vector graphics in your haddock documentation, as I have done for Optimisation.CirclePacking, using the syntax <<<data:image/svg+xml;base64,PD94bWwgdmV...c3ZnPg==>>.

12 December 2012

Vincent Sanders: What man's mind can create, man's character can control.

I have a project that required me to programmatically control power to several devices. I have done this before using a Velleman vm8090 board which is relatively easy to control. However they are relatively expensive.

I turned to ebay and found a similar module at a substantially reduced cost. Upon receipt however I discovered that instead of being a simple serial USB interface it presented USB HID and the Debian system I was running it on has loaded the hiddev driver for me but it did not implement any of the standard HID Usage Pages leaving me with no way to control the device.
I did the obligatory
sudo lsusb -d 12bf:ff03 -vvv

Bus 003 Device 019: ID 12bf:ff03  
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x12bf
idProduct 0xff03
bcdDevice 1.00
iManufacturer 1 Matrix Multimedia Ltd.
iProduct 2 Flowcode USB HID
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 41
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 50mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 3 Human Interface Device
bInterfaceSubClass 0 No Subclass
bInterfaceProtocol 0 None
iInterface 0
HID Device Descriptor:
bLength 9
bDescriptorType 33
bcdHID 1.10
bCountryCode 0 Not supported
bNumDescriptors 1
bDescriptorType 34 Report
wDescriptorLength 54
Report Descriptors:
** UNAVAILABLE **
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 5
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x01 EP 1 OUT
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 5
Device Status: 0x0980
(Bus Powered)
This simply showed me what I already knew and surprised me that lsusb did not dump HID report descriptor items. Some searching revealed that teh device had to be unbound so lussb could access the descriptor.

Thus a simple
echo -n 3-1.1.4:1.0   sudo dd of=/sys/bus/usb/drivers/usbhid/unbind
resulted in lussb dumping the descriptor items:

Item(Global): Usage Page, data= [ 0xa0 0xff ] 65440
(null)
Item(Local ): Usage, data= [ 0x01 ] 1
(null)
Item(Main ): Collection, data= [ 0x01 ] 1
Application
Item(Local ): Usage, data= [ 0x02 ] 2
(null)
Item(Main ): Collection, data= [ 0x00 ] 0
Physical
Item(Global): Usage Page, data= [ 0xa1 0xff ] 65441
(null)
Item(Local ): Usage, data= [ 0x03 ] 3
(null)
Item(Local ): Usage, data= [ 0x04 ] 4
(null)
Item(Global): Logical Minimum, data= [ 0x00 ] 0
Item(Global): Logical Maximum, data= [ 0xff 0x00 ] 255
Item(Global): Physical Minimum, data= [ 0x00 ] 0
Item(Global): Physical Maximum, data= [ 0xff ] 255
Item(Global): Report Size, data= [ 0x08 ] 8
Item(Global): Report Count, data= [ 0x08 ] 8
Item(Main ): Input, data= [ 0x02 ] 2
Data Variable Absolute No_Wrap Linear
Preferred_State No_Null_Position Non_Volatile Bitfield
Item(Local ): Usage, data= [ 0x05 ] 5
(null)
Item(Local ): Usage, data= [ 0x06 ] 6
(null)
Item(Global): Logical Minimum, data= [ 0x00 ] 0
Item(Global): Logical Maximum, data= [ 0xff 0x00 ] 255
Item(Global): Physical Minimum, data= [ 0x00 ] 0
Item(Global): Physical Maximum, data= [ 0xff ] 255
Item(Global): Report Size, data= [ 0x08 ] 8
Item(Global): Report Count, data= [ 0x08 ] 8
Item(Main ): Output, data= [ 0x02 ] 2
Data Variable Absolute No_Wrap Linear
Preferred_State No_Null_Position Non_Volatile Bitfield
Item(Main ): End Collection, data=none
Item(Main ): End Collection, data=none

By consulting the device class definitions document I determined the device was using the "Vendor defined" Usage page (0xff00 to 0xffff) so I would definitely have to write a program to control the device.

Linux provides a really easy interface to deal with HID devices called hiddev (gosh, such adventurous naming) which I already had to unbind to get my descriptors decoded so I am fairly sure it works ;-)

The kernel documentation and header for hiddev provide the absolute basic mechanics of the interface but no example code or guidance. The obligatory web search turned up very little and even that had to be retrieved from the internet archive. So It seems I would be forced to work it through myself.

It seems the hiddev interface is orientated around HID devices generating reports which the program is expected to read. Numerous ioctl() are provided so the program can obtain the descriptor information necessary to control and process the received reports.

However in this case we need to be able to send reports to the device, all the descriptor information revealed was that there were eight (Item Report Count = 8) values with eight bits each (Item Report Size = 8) with logical and physical values representing the whole range of the octets.

Fortunately the seller provided a website with some control programs and even source. After some time rummaging through the Visual Basic program I finally found (in FrmMain.vb:2989) that the eight bytes were largely unused and the first was simply a bitmask of the eight relays coil status, set for energised clear for off. With bit 0 controlling relay labelled 1 through to bit 7 for relay 8.

To send a report to a HID device the hiddev interface uses the HIDIOCSREPORT ioctl where the report data is first set using HIDIOCSUSAGE .

The HIDIOCSUSAGE ioctl is passed a hiddev_usage_ref structure which must be initialised with information about the report descriptor identifier (constructed from the Usage Page and Usage as set by the items in the descriptor), the index of the item (named usage) we wish to set in the report (in this case the first which is 0) and the value we actually want to set.

After a great deal of debugging the final program is very short indeed but does the job, my main problem now is that if I switch too many (more than one) relays at once the whole device resets. The scope says the supply rails are behaving very badly when this happens, looks like I need to add a load of capacitance to the power well to stabilise it during the switching events.

Oh and add in the fact Relay 1 LED doest work unless you push on it and I do wonder about the wisdom of the economy in this case. Though yet again Linux makes the software side easy.

11 December 2012

Russell Coker: Links December 2012

Steven Johnson gave an interesting TED talk about where good ideas come from [1]. He starts by attributing coffee replacing alcohol as a standard drink for some good ideas and then moves on to how ideas develop. Erez Lieberman Aiden and Jean-Baptiste Michel gave an interesting and amusing TED talk about the ngram analysis of books that Google scanned [2]. Here is the link for the Google Books ngram search [3]. Clay Shirky gave an insightful TED talk about how the Internet is changing the world [4]. He cites Open Source programmers as the modern day equivalent to the Invisible College based on our supposed great ability to get along with other people on the Internet. If we really are so much better than the rest of the Internet then things must be bad out there. He ends with ways of using Git to draft legislation. Hans Rosling gave an interesting TED talk about religion and the number of babies that women have [5]. His conclusion is that it s more about income and social stability and that the world s population can stabilise at 10 billion if we provide family planning to everyone. Alexis C. Madrigal wrote an interesting interview with Genevieve Bell about her work at Intel and the way people use technology [6]. Indigogo is raising funds for the Cuddle Mattress , it s a mattress with foam slats and a special fitted sheet to allow your arm to slide between the slats [7]. So you could have your arm underneath your SO for an extended period of time without risking nerve damage. They also show that when sleeping on your side your shoulder can go between the slats to avoid back problems. Nate Silver (who is famous for predicting US elections gave an interesting TED talk about racism and politics [8]. One of his main points is to show the correlation between racism and lack of contact of members of other races. Sociological Images has an interesting article by Lisa Wade about whether marriage is a universal human value [9]. In regard to historical marriage she says women were human property, equivalent to children, slaves, servants, and employees . The general trend in the comments seems to be that there are so many types of marriage that it s difficult to make any specific claims to traditional marriage unless you count a tradition of a short period in a single geographic region. Plurality is an excellent sci-fi short movie on youtube [10]. TED has an interesting interview with Hakeem Oluseyi about his research about astrophysics and how he achieved a successful career after being a gangster as a teenager [11]. He has some good ideas about helping other children from disadvantaged environments become successful. Paul Dwerryhouse wrote an interesting blog post about his work in designing and implementing a filesystem based on a Cassandra data store with FUSE [12]. Paul also wrote a post about using Apache Zookeeper to lock metadata to permit atomic operations [13]. The documentary Monumental Myths provides an interesting and insightful analysis of the processes of creating, maintaining, and explaining monuments [14]. It focusses on some significant monuments in the US and explains both sides to each story. Howard Zinn makes the insightful point that when people present a certain point of view of history it s not controversial, as soon as you present the other side they call it controversial . That happens even in debates about current issues. Howard also says to criticise whatever the government does is not anti-America, it s anti-government, it s pro-America, it s pro the people, it s pro the country . The song that plays during the closing credits is interesting too. The music video Same Love is one of the best presentations of the argument for marriage equality [15]. Chris Samuel wrote an interesting post about systems locked down for Windows 8 and options for purchasing PCs for running Linux [16]. His solution is to buy from ZaReason. I saw his laptop in action at the last LUV meeting and it looks really nice. Unfortunately a byproduct of the extremely thin form factor is the fact that it lacks a VGA port, this meant that Chris had to use my Thinkpad T61 (which is rather clunky by comparison) for his presentation.

9 December 2012

Niels Thykier: Wheezy release progress (December)

Wheezy has been frozen for 5-and-(almost-)a-half months now. Last month, Neil sent a mail to d-d-a stating that there were 403 RC bugs left and today UDD claims there are 348 left. My pre-coffee^Wtea math gets this to an average of approximately 1.8 RC bugs a day. This average is only the change in total RC bugs between the 8th of Nov and today. It does not account for RC bugs being filed and fixed between these two days. However, there is another number that worries me. Namely the number of RC bugs fixed in sid, but not Wheezy. Today it is 139. According to Richard s weekly updates, that number has generally remaining in the interval 143-148 for 6 weeks (except week 45 where it went down to 134). Those 139 are generally just an unblock (and a couple of days) from reaching Wheezy (or a tpu upload). It does also include unblocked, but not old enough to migrate packages. Unfortunately the 112 unread emails in my d-release inbox suggests most of these 139 still needs attention. Most of these unread emails are (unfortunately) unblock/tpu requests that none of us (to my knowledge) has had time to respond to. So, if you cannot find an RC bug to fix, I hope you will consider doing a bit of peer review and help us bring down the number of unanswered unblock requests. DDs can access the same tools we use (See Neil s mail). Otherwise, the diff is just a:
$ dget -d http://.../pkg_testing-version_arch.dsc
$ dget -d http://.../pkg_sid-version_arch.dsc
$ debdiff pkg_testing-version_arch.dsc pkg_sid-version_arch.dsc


8 December 2012

Eddy Petri&#537;or: Consent is needed, but not the point

Russ makes a good point about a nuance of practical jokes that makes them special, the characteristic of involving a third party which is the target of the joke, and how that relates to consent. He is right, consent is an important fact, I acknowledge that. Russ also says at the end of his post that being an asshole doesn't make you responsible for somebody else's decisions, I ackowledge that, too. I even somewhat agree to the position that the employer might fire you on assholery reasons, but in this particular case, I think that's a hypocrite position. If assholery was an issue, it should have been before, too, and I don't think the employer thought of it before this sad even, as long as it meant more money for them. I think this only shows how cynical they are, the cost analysis for them showed them they now might lose more money by not firing, than by keeping them. It's just cynical.

In spite of that, my initial post, just as I wrote, was not about the DJs, the people involved, the assholery, the taste or lack thereof of the joke. That's why I was intentionally leaving out names or specific context. It was about the bigger picture, irrational reactions and how in the face of a tragedy we surrender our thinking capabilities and react in an irrational manner to the least significant things.

I have had quite recent unfortunate events happening to people close to me, and, invariably, I observe this loss of rational faculties in the face of sad events. Even when this kind of disproportionate response is shown for what it is and how it is harmful in itself, people still refuse to give up on being irrational or they do after a long struggle with the facts.

I'll finish my article with a similar question: can we, as a species, go beyond our condition and make a conscious effort to keep calm and think things through before making decisions in the face of tragedy, or are we incapable to do that?

And no, I am not advocating to drop feelings and affection, I find it sad that I have to say that again, but I am almost sure that in spite of it, I will be wrongly understood by somebody.

Russ Allbery: Review: Peacekeeper

Review: Peacekeeper, by Laura E. Reeve
Series: Major Ariane Kedros #1
Publisher: Roc
Copyright: December 2008
ISBN: 0-451-46245-9
Format: Mass market
Pages: 324
Ariane Kedros is a pilot of a second-wave prospecting ship. Second-wave prospectors are the first explorers into new systems after slower-than-light generation ships haul in the beacons that add them to the FTL network. She's also a reserve major in the Consortium military, which is a point of ongoing tension with her business partner and employer. She disappears on missions for the intelligence branch that she can't talk about, and usually ends up drinking herself into a stupor when she returns. In actuality, Kedros is not her real name. It's a fake identity given her after the last war, a war in which humans detonated a temporal distortion warhead for the first time and possibly destroyed an entire solar system. Kedros was part of the team that launched the bomb and is, according to the Terran Expansion League (the other side of the all-human conflict), a war criminal. She's in the equivalent of a witness protection program, as are all the other people involved in the act. But someone has broken their cover and is picking them off, one by one, and her latest secret mission from the Consortium is to protect another member of the group. Peacekeeper is Reeve's first novel, and unfortunately this shows badly at the start of the book. The writing is choppy and awkward, the paragraph breaks are a bit off, and the sentence structure kept making my teeth itch. I'm not a good enough technical critic to put a finger on what's wrong, but the writing doesn't flow. I also found myself anticipating and completing sentences in my head, which is a sign of too many low-level cliches. A few dozen pages into the book, the writing was bothering me, I hadn't warmed to any of the characters, and I wasn't sure I was going to like it. Ariane grew on me, though, particularly once she got into her mission and started balancing a cover assignment and her actual work. She's hurt, suffering from what's probably post-traumatic stress disorder and self-medicating with alcohol, but she's smart and competent and the plot is sufficiently tricky to provide a lot of meat. Not only is someone after her old team, but she and her partner discovered something potentially major on their last trip, an old friend of her partner has been murdered, the Consortium and the Terrans are negotiating a peace treaty, and all this is happening under the watchful eye of a superior alien race. The Minoans are probably Reeve's best creation here (if a little too similar to Vorlons at times). They're absolutely neutral, but they do not want humanity messing with temporal distortion weapons and are trying to nudge (or push) the humans into peace and disarmament. They are sticklers for not imposing their cultural practices on others, attempting to judge humanity solely by its own rules, but they're utterly literal-minded about rules. They expect everyone to follow any stated rules exactly and to the letter, making them excellent but frustrating overseers of disarmament treaties. They also have a fascinating tendency to interact with people in roles rather than as individuals. This is clearly the first book in an ongoing series, and a lot of the setup is not resolved here. We get only glimpses of what Ariane and her partner may have found, and only a few interactions with the Minoans, while Ariane tracks down who is responsible for killing her former team. There's a bit too much torture for my taste, and the subplot of Ariane's struggle with alcoholism wasn't really what I wanted to read about, but the writing seemed to get substantially better and the plot kept me turning the pages. There's a quality to military SF written by people who have actually been in the military that I find hard to describe but that helps a book tremendously. It didn't surprise me at all to learn that Reeve served in the US Air Force, since Peacekeeper has that quality. It's less about technical accuracy and more about a feel, a sense of what it's like to be inside a military routine and interacting with other military personnel. The culture feels coherent, despite not being described in detail. I think it makes for good SF in general, since it conveys to someone like me (who has never been in the military) a semi-alien but completely coherent culture. It certainly improves this book; in general, I thought Reeve's military characters were more interesting and more believable than her non-military characters. With the rocky beginning, the unpolished writing, and a villain who wasn't entirely satisfactory, I can't quite recommend Peacekeeper, but it's not a bad read. I'm not sure if I'll pick up the sequels, but I did leave it curious about the unresolved plot threads and willing to learn more about Reeve's world. Followed by Vigilante. Rating: 6 out of 10

5 December 2012

Russell Coker: Ingress

Today Google sent me an invite for Ingress their latest augmented reality game [1]. The fact that they sent me the invitation while the Google Play store page for Ingress [2] tells me that it s not available in my country (Australia) is interesting. Google obviously aren t using their Big Brother powers effectively! The way to install Ingress if you are in Australia, New Zealand, or other countries where it s not supported is to do a Google search on the words Ingress and APK and take the highest available version (1.08 at the moment). Then you will find a web site that offers it with no authentication and the potential of getting a trojan version. Forcing people to install the software in an insecure manner doesn t seem to be in the best interests of Google. I first installed the game on my Sony Ericsson Xperia X10 which has a 1 GHz Qualcomm Snapdragon QSD8250 CPU and runs Android 2.3.3 compiled by Sony-Ericsson and performance was terrible. It often would fail to respond to the UI effectively, it would process touch actions after I had repeated them because they didn t seem to have registered, scrolling text at the same speed as playing audio was apparently impossible. Then I briefly tried running it on my Samsung Galaxy S which has a 1 GHz (ARM Cortex A8) CPU and runs Android 2.3.7 with a CyanogenMod-7.1.0-GalaxyS build. It seemed to be a bit faster but the difference was small enough that I could have imagined it. I tried it on the cheap Onix tablet I bought from Aldi this morning [3] but it refused to work every time I tried and eventually crashed the tablet and forced me to use the reset button. Finally I tried it on a Kogan Agora 10 tablet running Android 4.0.4 with what Kogan describes as a 1GHz ARMv7 CPU and it seemed a lot more usable. I haven t yet tried actually playing the game on the Kogan Agora tablet, but the fact that I can read the messages from other players is a significant improvement over the experience on the phones. On the phones the poor performance of the UI made it almost impossible to read messages from other players, now it s merely extremely annoying. I m very disappointed with this. Almost three years ago I reviewed the Seer augmented reality software from IBM that performed tasks that are more demanding and did it well [4]. The IBM Seer software ran on a HTC Hero which had a 528 MHz Qualcomm MSM7600A CPU, it s really disappointing that Ingress can t run well on modern phones particularly as the augmented reality part of the game which I ve seen so far is not much different to what Google Maps, Osmand, and other mapping programs do. The IBM Seer software was good enough to drive the purchase of new phones. When I first got an Android phone almost two years ago I wanted to run such augmented reality software and it was a factor that determined my choice of phone. Unfortunately I still haven t found anything to live up to that promise. IBM s software was tailored to the Australian Open even a tennis fan would find it to be of little interest for about 360 days of the year and I haven t found any other augmented reality software that is useful and works well (please let me know if there s something I ve missed). I find it difficult to imagine that anyone would be inspired to change their phone purchase plans after seeing a demo of Ingress on any hardware that I own. It seems that it will probably be usable on my Xperia X10 and all features should work on the Kogan Agora tablet, but I don t think that any of them will allow the game to live up to the hype.

Russell Coker: Cheap Android Tablet from Aldi

back of the onix 7 inch tabletfront of the onix 7 inch tablet and cover of manual I ve just bought a 7 Onix tablet from Aldi. It runs Android 4.0.4, has a 1GHz Cortex A8 CPU, 512M of RAM, 16G of flash storage, and a 800*480 display. They are selling rapidly and I don t know how long they will last probably you could get a returned one next week if you can t get one today. But if you like pink then you may be able to get one (the black ones are selling out first). The tablet seems like a nice piece of hardware, solid construction and it feels nice to hold. Mine has a minor screen defect, but that s the sort of
thing you expect from a cheap device, apart from that the display is good. The Wifi doesn t seem to have as good a range as some other devices (such as my phones and the more expensive 10 tablet I got from Kogan). This isn t a
problem for me (the data intensive uses for this device will be in the same room as the AP) but could be a killer for some people. If you have your phone or a dedicated 3G Wifi AP in your pocket while using the tablet then it should be fine, but if you have an AP at the wrong end of your house then you could be in trouble. I found Youtube unusable due to slow downloading even when sitting next to my AP but I can play videos downloaded from iView that are on my local web server (which is more important to me). I expect that I will be able to play local copies of TED talks too. The camera is bad by phone camera standards, fortunately I have no interest in using a tablet as a camera. I had no real problems with the Google Play store (something that caused problems for some users of an earlier Aldi Android tablet). Generally the tablet works well. The people who build Android for modern devices seem remarkably stupid when it comes to partitioning, every device I ve seen has only a small fraction of the storage usable for apps. This tablet is the worst I ve ever seen, it has 16G of storage of which there is 512M partitioned for apps of which only 400M is free when you first get the device! It comes pre-installed with outdated versions of the Facebook client and Google Maps (which isn t very useful on a Wifi device) and some other useless things. If you upgrade them to the latest versions then you ll probably lose another 100M of the 512M! Fortunately the Android feature to run apps from the VFAT partition works so I haven t been prevented from doing anything by this problem yet. In conclusion, it s not the greatest Android tablet. But you don t expect a great tablet for $100. What I hoped for was a somewhat low spec tablet that works reasonably well and that s what I got. I m happy.

4 December 2012

Michal &#268;iha&#345;: PyGtk in virtualenv on Travis

Weblate is using Travis CI for continuous integration. That in turn uses virtualenv to test against different Python version. Unfortunately one of dependencies in Weblate are PyCairo and PyGtk (actually Pango for text rendering) and these does not support installation using distutils. Fortunately this can be workarounded, but it took me some time to make all this properly. Ffirst you need to choose correct versions not as I did mistakenly mixed PyGObject 3 with PyGtk 2. The next step is to manually run configure and make with correct paths to install within virtualenv. For example for Gtk it looks like following:
wget http://ftp.gnome.org/pub/GNOME/sources/pygtk/2.24/pygtk-2.24.0.tar.bz2
tar xf pygtk-2.24.0.tar.bz2
cd pygtk-2.24.0
export PKG_CONFIG_PATH=/home/travis/virtualenv/python$TRAVIS_PYTHON_VERSION/lib/pkgconfig
./configure --prefix=/home/travis/virtualenv/python$TRAVIS_PYTHON_VERSION/
make
make install
The complete Travis configuration file can be found in Weblate's Git repository.

Filed under: English Weblate 0 comments Flattr this!

1 December 2012

Rapha&#235;l Hertzog: My Free Software Activities in November 2012

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (692.20 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Misc packaging I updated the publican package (a tool for publishing material authored in DocBook XML) with version 3.0, a major new upstream version. As with any important update, it had its share of problems and I created two patches that I sent upstream. I uploaded the package to experimental since we re in freeze. The Debian Administrator s Handbook Since the translation teams have been working for a few months, I wanted to put the result of their work online. I did it and I blogged about it on debian-handbook.info. By the way, we have a Polish translation that just started. This took quite some time because many translators were not well versed with Docbook XML and its structure. So I fixed their mistakes and asked the Weblate developer (Michal Cihar) to implement new checks to avoid those basic XML mistakes. I also added a couple of build scripts to the git repository to make it easier to rebuild translations in multiple formats. I used this opportunity to file a couple of bugs I encountered with Publican (concerning ePub output mainly, and custom brands). I also blogged about our plans to update the book for Wheezy. Roland started to work on it but I did not have the time yet. Debian France The officers (president, treasurer, secretary) have just changed and we had to organize the transition. As the new president, I got administrator access on our Gandi virtual machine (france.debian.net) as well as access to our bank account. I got also got a bunch of administrative papers retracing the history of the association. Carl Chenet (the former president) gave them to me during the mini-debconf that was organized in Paris. Indeed, Sylvestre Ledru and Mehdi Dogguy organized our second mini-debconf Paris and they did it very well. It was a great success with over 100 attendants each of the 2 days it lasted (November 24-25th). Carl managed a merchandising booth that was well stuffed (Luca Capello also brought goodies of Debian.ch) I gave small lightning talk to present the ideas behind my Librement project (it s about funding free software developers). BTW I have not been very good at it, it was only my second lightning talk and I have been a bit too verbose. The talk did not fit in my 5 minutes time slot ;-) Back from the mini-debconf, I have been trying to delegate some projects (like get a real website, improve the work-flow of members management, update our server which was still running Lenny). Julien Cristau was willing to upgrade the server did not exactly knew how to upgrade the kernel (it s a bit special since Gandi manages the kernel on the Xen hypervisor side). So I took care of this part and also did some cleanup (adding a backup with its associated remote disk, tweaking the email configuration). And Julien completed the upgrade on November 30th. Alexandre Delano volunteered to have a try at the website and Emmanuel Bouthenot has been looking a bit to see if there was something better than Galette to handle our members. It looks like we ll stay with Galette but have to take care of upgrading it to a newer version. I also processed the first membership applications and organized a vote to extend the board of administrators (since we have two vacant seats). On Monday, we should be back to 9 administrators. Librement Except for the talk during the mini-debconf, I did not do much on this project. That said I got an answer from the Autorit de Contr le Prudentiel saying that I might be eligible for the exemption case (see discussion of last month) and that I should fill out a form to get a confirmation. I also contacted Tunz.com who might be able to provide the services I need (their E-money manager product in particular). They have the required accreditation as a banking/credit institution and are willing to partner with enterprises who setup platforms where you must manage flows of money between several parties. I m now waiting for details such as the cost of their various services. I expect to have much more to show next month I m working with two developers to implement the first building blocks of all this. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

29 November 2012

Bernhard R. Link: Gulliver's Travels

After seeing some book descriptions recently on planet debian, let me add some short recommendation, too. Almost everyone has heard about Gulliver's Travels already, so usually only very cursory. For example: did you know the book describes 4 journeys and not only the travel to Lilliput? Given how influential the book has been, that is even more suprising. Words like "endian" or "yahoo" originate from it. My favorite is the third travel, though, especially the acadamy of Lagado, from which I want to share two gems: " His lordship added, 'That he would not, by any further particulars, prevent the pleasure I should certainly take in viewing the grand academy, whither he was resolved I should go.' He only desired me to observe a ruined building, upon the side of a mountain about three miles distant, of which he gave me this account: 'That he had a very convenient mill within half a mile of his house, turned by a current from a large river, and sufficient for his own family, as well as a great number of his tenants; that about seven years ago, a club of those projectors came to him with proposals to destroy this mill, and build another on the side of that mountain, on the long ridge whereof a long canal must be cut, for a repository of water, to be conveyed up by pipes and engines to supply the mill, because the wind and air upon a height agitated the water, and thereby made it fitter for motion, and because the water, descending down a declivity, would turn the mill with half the current of a river whose course is more upon a level.' He said, 'that being then not very well with the court, and pressed by many of his friends, he complied with the proposal; and after employing a hundred men for two years, the work miscarried, the projectors went off, laying the blame entirely upon him, railing at him ever since, and putting others upon the same experiment, with equal assurance of success, as well as equal disappointment.' " "I went into another room, where the walls and ceiling were all hung round with cobwebs, except a narrow passage for the artist to go in and out. At my entrance, he called aloud to me, 'not to disturb his webs.' He lamented 'the fatal mistake the world had been so long in, of using silkworms, while we had such plenty of domestic insects who infinitely excelled the former, because they understood how to weave, as well as spin.' And he proposed further, 'that by employing spiders, the charge of dyeing silks should be wholly saved;' whereof I was fully convinced, when he showed me a vast number of flies most beautifully coloured, wherewith he fed his spiders, assuring us 'that the webs would take a tincture from them; and as he had them of all hues, he hoped to fit everybody s fancy, as soon as he could find proper food for the flies, of certain gums, oils, and other glutinous matter, to give a strength and consistence to the threads.'"

28 November 2012

Alberto Garc a: QEMU and open hardware: SPEC and FMC TDC

Working with open hardware Some weeks ago at LinuxCon EU in Barcelona I talked about how to use QEMU to improve the reliability of device drivers. At Igalia we have been using this for some projects. One of them is the Linux IndustryPack driver. For this project I virtualized two boards: the TEWS TPCI200 PCI carrier and the GE IP-Octal 232 module. This work helped us find some bugs in the device driver and improve its quality. Now, those two boards are examples of products available in the market. But fortunately we can use the same approach to develop for hardware that doesn t exist yet, or is still in a prototype phase. Such is the case of a project we are working on: adding Linux support for this FMC Time-to-digital converter.
FMC TDC
This piece of hardware is designed by CERN and is published under the CERN Open Hardware Licence, which, in their own words is to hardware what the General Public Licence (GPL) is to software . The Open Hardware repository hosts a number of projects that have been published under this license. Why we use QEMU So we are developing the device driver for this hardware, as my colleague Samuel explains in his blog. I m the responsible of virtualizing it using QEMU. There are two main reasons why we want to do this:
  1. Limited availability of the hardware: although the specification is pretty much ready, this is still a prototype. The board is not (yet) commercially available. With virtual hardware, the whole development team can have as many boards as it needs.
  2. Testing: we can test the software against the virtual driver, force all kinds of conditions and scenarios, including the ones that would probably require us to physically damage the board.
While the first point might be the most obvious one, testing the software is actually the one we re more interested in. My colleague Miguel wrote a detailed blog post on how we have been using QEMU to do testing. Writing the virtual hardware Writing a virtual version of a particular piece of hardware for this purpose is not as hard as it might look. First, the point is not to reproduce accurately how the hardware works, but rather how it behaves from the operating system point of view: the hardware is a black box that the OS talks to. Second, it s not necessary to have a complete emulation of the hardware, there s no need to support every single feature, particularly if your software is not going to use it. The emulation can start with the basic functionality and then grow as needed. The FMC TDC, for example, is an FMC card which is in our case connected to a PCIe bridge called SPEC (also available in the Open Hardware repository). We need to emulate both cards in order to have a working system, but the emulation is, at the moment, treating both as if they were just one, which makes it a bit easier to have a prototype and from the device driver point of view doesn t really make a difference. Later the emulation can be split in two as I did with with TPCI200 and IP-Octal 232. This would allow us to support more FMC hardware without having to rewrite the bridging code. There s also code in the emulation to force different kind of scenarios that we are using to test if the driver behaves as expected and handles errors correctly. Those tests include the simulation of input in the any of the lines, simulation of noise, DMA errors, etc.
Tests
And we have written a set of test cases and a continuous integration system, so the driver is automatically tested every time the code is updated. If you want details on this I recommend you again to read Miguel s post.

24 November 2012

Russ Allbery: Review: Dune

Review: Dune, by Frank Herbert
Series: Dune #1
Publisher: Ace
Copyright: 1965
Printing: September 1990
ISBN: 0-441-17271-7
Format: Mass market
Pages: 537
The Atreides family, distant cousins to the imperial family, have ruled the planet of Caladan for twenty generations. Caladan is a wet farming world, comfortable and pleasant, but not horribly important. But House Atreides is feuding with House Harkonnen, and, at the start of Dune, that feud maneuvers Duke Leto into giving up his holdings and moving his family to take possession of Arrakis. Arrakis is a desert planet, previously controlled by Baron Harkonnen. It is unrelentingly hostile, home to smugglers and dangerous local desert dwellers called Fremen. But it's also one of the most important planets in the galaxy, since it's the sole origin of the chemical called melange, or spice. Spice permits a limited form of prescience, which allows the navigators of the Spacing Guild to successfully steer ships across the interstellar void. Arrakis's production of spice is what makes interstellar travel, and therefore all of interstellar civilization, possible. Dune is the story of Paul Atreides, son and heir to Duke Leto Atreides. His mother, Lady Jessica, is one of the Bene Gesserit, a secretive order of women devoted to mental and physical discipline and to the long-term genetic improvement of mankind. He is not supposed to exist; Lady Jessica was supposed to only bear a daughter of Leto. But he may be something special, the long-sought (but also dangerous) Kwisatz Haderach who can unite male and female Bene Gesserit powers. The Bene Gesserit take great interest in him from the start of Dune. More surprisingly, so do the Fremen of Arrakis; from the moment he arrives there, he seems to be fulfilling prophecies of theirs that are partly, but not entirely, ones planted by the Bene Gesserit long ago. The feud with the Harkonnens, the unstable place of Arrakis in galactic politics, the dreams of the Fremen and the Imperial ecologist on Arrakis of terraforming, Bene Gesserit plans, Paul's abilities, and the legends of the Fremen all combine in a complex mix of politics, battle, and clashes of culture. Dune is an acknowledged SF masterpiece, one of the best-known classics of the genre. It's usually found in short lists of the best SF novels ever written. It spawned five sequels by Frank Herbert (about which more in a moment), as well as numerous additional sequels and prequels by Kevin J. Anderson and Brian Herbert. It's been adopted for the screen twice, not to mention board games, video games, and numerous other projects. This is my second reading, the first in about twenty years, but the story was still immediately familiar from having seen films and having discussed and read about the universe. This is not science fiction in any strict sense. Dune is science fiction in the same way that Star Wars is: a futuristic gloss on top of power structures inspired by feudalism, heavily mixed with mysticism, mental powers, magic, and implausible but convenient science that creates the story effects the author wants. Both Bene Gesserit powers in general and Paul's abilities in particular are effectively magic. There is some hand-waving explanation of their ability to verbally control other people as taking advantage of specific pitches and intonations that people are vulnerable to, but it's effectively spell-casting (and is a direct inspiration for Jedi mind tricks). All of the mysticism (and there's quite a lot of it in Dune, including race memory, precognition, and even molecular transformation) resembles the Force from Star Wars more than anything scientific. Dune is epic fantasy told on a science fiction stage, complete with a young protagonist coming into his powers and dangerous and sometimes hostile mentors. What Dune gets right, and what has put it so high in the pantheon of great science fiction, is the world building. Herbert sets the story tens of thousands of years into the future of humanity and then effectively projects the feeling of deep history over everything in the novel. This is the kind of book that has appendices with more background information; more to the point, it's the kind of book where you may actually read them out of curiosity. Mankind has a vast interstellar empire (Herbert's universe, like Asimov's Foundation universe, admits no aliens) governed by a system akin to the early British monarchy. An emperor rules in balance with the Great Houses, who meet in a sort of parliament. But against both is a third force: the Spacing Guild, who maintains a monopoly over all interstellar travel. (And the Bene Gesserit form an underground, secretive fourth power base.) Herbert plays with vast swaths of time and great forces of history as well as very good epic fantasy and better than nearly all SF I've read. The detailed world-building is equally good. Nearly all of Dune takes place on the desert planet of Arrakis, which has a lovingly-described ecology and local culture built entirely around scarcity of water. (The details of that ecology are much of the plot and mystery of the book, so I won't spoil them further.) While I doubt the precise details hold up to close scientific scrutiny, this is an obvious precursor to the great ecological stories of later SF, such as Kim Stanley Robinson's Mars trilogy. The details all feel right and hang together in satisfying ways, while also generating the great Sand Worms of Arrakis, a key ingredient in several of the best set pieces in the history of SF. This is the sort of book where the fascinating details and discoveries about the world do as much to keep one turning the pages as the plot, although the plot is also satisfyingly twisty and tense. Unfortunately, Dune doesn't get everything right. The amount of mysticism involved is a bit much, and at times the drug-trip mystical experiences of viewpoint characters turn into excessively purple prose and nearly incomprehensible descriptions. Those mystical experiences also involve race and genetic memory, a concept that's just scientific enough to be unbelievable. A few of the other scentific cheats are also rather blantant; for example, Herbert constructs an elaborate, artificial technology of shielding that seems designed primarily as an excuse to add sword combat to a futuristic story, and I have always struggled to suspend disbelief about the way lasers and shields interact in Dune. The Spacer Guild's monopoly on interstellar travel can be explained; their monopoly on local orbital space, or even the high stratosphere, both vital to allow certain things on Arrakis to remain secret, are much more dubious. Herbert mostly doesn't try to explain these things, and as with Star Wars the less explained the cheats are, the better they work as part of the story. But the technological background doesn't hold up against much examination. Worse, for me, is the general quality of the writing. Herbert does some things very well, such as world-building, and avoids awkward infodumps. Characterization and pacing are both fairly solid; he does a good job with Paul and Jessica in particular, and I've always liked the Fremen. But he wants to put the reader in everyone's head, frequently by giving character thoughts as italicized dialogue, and to enable that he uses a perspective that I always find distracting. Most fiction is written in tight third person. This means that the viewpoint character for any given section of the book is referred to in the third person, like all the other characters, but the reader has special access to their thoughts and emotions. We get to know what they're really thinking and feeling, not just the impressions they give to others, while the non-viewpoint characters are shown only from external appearances and the thoughts of the viewpoint character. Some books hold to the same viewpoint character throughout, but more commonly books move between viewpoint characters at scene breaks to provide more angles on the book's events. First person, in which the story is told by a specific character as if they were telling a story or writing it down, is the most common alternative. Third person objective, in which we don't get any special insight into the internal thoughts of any of the characters, is less common but still unsurprising. Dune does not use any of those perspectives. Instead, Dune uses wandering third-person omniscient, in which we get the inner thoughts and emotions of a character in a scene and then a few lines later the inner thoughts and emotions of a different character. This is the sort of thing that may or may not bug you depending on how much you've read, how deep the expectations of perspective are ingrained, and how much you notice perspective. It drives me nuts. I subconsciously align with the viewpoint character of a section, and pay attention to the ways that authors indicate which character will be the viewpoint character at the start of a scene. Herbert's constant flitting from character to character makes me dizzy. We get the verbatim thoughts of everyone almost indiscriminately, making me feel like I'm randomly hopscotching through the scene. For me, this does two things: it hurts my ability to get engrossed in the story, since I'm constantly thrown out of my normal reading mode when the viewpoint unexpectedly shifts, and it makes the writing feel repetitive. One keeps hearing about the same thing from multiple perspectives, and at times the story bogs down in everyone's internal dialogues rather than showing character reactions and letting the reader draw their own conclusions. I think it tries for a cinematic perspective, but ends up making the story feel muddled. The other flaw, which I didn't notice originally but which leaped out at me during this re-read, is that Herbert's world-building uses quite a few stereotypes. The most notorious, and most widely discussed, is of course the Fremen. Herbert draws heavily on Arab and Islamic culture even beyond the obvious similarities of people living in a harsh, arid climate. He borrows some rather loaded terms and cultural markers, such as jihad, to construct a culture of potential religious fanatics. This is not all bad; the Fremen are clearly portrayed as the good guys, which is a refreshing change from more typical current portrayals of Islam. But it becomes clear that they have aligned their entire culture around influences from outside, and the whole plot of Dune can be fairly characterized as an instance of "what these people need is a white man." Paul (and Kynes before him) joins their culture as well, but Paul becomes a better native than the natives, while simultaneously bringing his outside perspective. It's the sort of plot that is more widely noticed today than it would have been in 1965. Another major example of this, and one that I found more blatant, is that Herbert turns the Harkonnen into hissable, one-sided villains and uses some nasty stereotypes to do it. The insane torturer is consistently and repeatedly described as effeminate, fat is used as a marker of moral inferiority and evil, and the primary villain is homosexual and prefers drugged young male slaves. Here too, this sort of characterization short-cut was more common in 1965, but it's not appealing and makes the (already rather camp) scenes set among the Harkonnen even less enjoyable. Less clear-cut is the way women are handled throughout Dune. I do have to give Herbert some credit, particularly for the era in which he was writing. There are powerful female characters in Dune, including both Jessica and Alia, who have their own independent power and successfully pursue their own agendas throughout. The effectively all-female Bene Gesserit is a major political power in the story and is treated by the other players with respect as well as fear. But it's hard not to also notice the general position of women as subservient to men, not only in the general culture of the Great Houses but also in the more positively-portrayed Fremen culture. Indeed, the subservience of women is even worse in Fremen culture, where they're treated like property and where being killed by a woman is a sign of shame. Again, Herbert deserves some credit for doing better than a lot of 1960s fiction, but the sexism fairy has still been at work here. None of these flaws change the fact that Dune is a masterpiece. Herbert brings together history, world building, ecology, politics, and a compelling coming-of-age story about a messiah figure into a fast-paced, sweeping epic with a thoroughly satisfying conclusion. I think they do make it a flawed masterpiece, but it's still one of those SF novels that everyone should read at least once. Sadly, it's also a masterpiece that I think has suffered from its own success in the form of sequels, prequels, and a ton of supporting material. This is one of the problems that truly excellent world building can lead to. Human history is fractal: any specific detail can be examined in more depth and will usually lead (provided that information is available at all) to even more fascinating detail. The best world building conveys that impression of depth. That's what Herbert achieves here with hints, notes, and asides: the sense that galactic history is a vast ediface with the same fractal complexity as real human history. It makes for a compelling background, but it also inspires people to dig into that background and flesh out all of the details the way that we do with human history. But this doesn't actually work; invented history created by one person simply cannot be fractal in the same way. Human history is endlessly complex because it was generated by the complex interactions of many people. Invented history is an illusion that hints at complexity by building the same surface, but one mind, or even a small number of minds, cannot generate the same depth. The result is that if one digs too deep, one removes that convincing surface and ends up with a mundane, simplistic, and unsatisfyingly fake set of events. I think that's what's happened with all of the supporting material that's been written around Dune since its original publication. Dune is of a piece, a single story that's deeply enjoyable on its own terms and leaves the reader with a satisfying impression of complexity. The systemic excavation of that complexity lessens it and reveals too much of the illusion. Yes, I want to know more about the Butlerian Jihad, but that's the point: the wanting is the sign of succesful crafting of imagined history. Reading the definitive account is more likely to leave me unsatisfied than to lead to the recursive curiosity that human history can create. The sequels to Dune written by Herbert himself are, for me, another matter. Some reviewers level the same criticism at them: that Herbert dives too far into background best left unexplored. But they have the advantage of moving forward, telling more of the story set off by Paul, and the end of Dune is a clear setup for a sequel. One of Paul's goals throughout most of the book has been left unaccomplished. I don't think Herbert dove too deep into his creation; rather, my problem with his sequels (all of which I've read, although it's been some years now) is that he took the story in a direction that I actively disliked and found painful to read. Regardless, the general consensus is that the sequels aren't as good as the original, and while Dune doesn't fully resolve its story, it's complete enough that it's possible to stop here. Stopping is the general recommendation, although I still may re-read and review the sequels at some point. Followed by Dune Messiah. Rating: 8 out of 10

Next.

Previous.