Review: Lockstep, by Karl Schroeder

Publisher:	Tor
Copyright:	March 2014
ISBN:	0-7653-3726-6
Format:	Hardcover
Pages:	351

Toby McGonigal's family fled an Earth dominated by trillionaires and vicious class conflict, and attempt, instead, a very risky and precarious settlement on a trans-Neptunian object. It's the last unclaimed but marginally habitable space left in the solar system, and securing their claim requires constant bureaucratic hoop-jumping. That's what sent Toby on a solo mission to a distant moon of their home to claim it and solidify their title. But, while in cold sleep, his craft hits a small chunk of rock, and he wakes up near another world: cold, silent, dead... but apparently with lifeless cities. Another trip through suspended animation in a desperate attempt to conserve resources against the distant hope of rescue has him awakening in a society both utterly foreign and yet strangely familiar. Schroeder is one of the better big idea writers in science fiction, but I found Lockstep to be a mixed bag. He does a surprisingly good job with the core conceit of the novel (more on that in a moment), even though it's a tricky one to make believable. Surrounding that, though, are a lot of less convincing bits that I kept having to not think about too hard, such as Toby crossing the path of another trans-Neptunian object after the accident (space is really, really big and really, really empty), or the implications of later revelations about the time scales involved in parts of the plot. Some parts of the world building, even if scientifically plausible, struck me as sloppy; for example, a religion plays a prominent role in the plot, but the nature of that religion was not particularly believable, nor was its interaction with the plot climax. I won't go into details about the religion, since it's a major plot point, but the short version is that religions tend to mature from the concrete to the abstract, not the other way around. That said, the core conceit is both surprising and considerably better-defended in the book than I thought it could be. The world into which Toby awakes is the world of the locksteps: a society that uses suspended animation in a universal and coordinated way to build a functional society on the far outskirts of the solar system. Humans emerge for some short period of time, like a month, and build, trade, interact, and consume. They welcome ships traveling from other trans-Neptunian worlds and prepare for their own journeys. And then they go into suspended animation for an extended period 15 years and 30 years are common choices while robots slowly gather more resources and energy, and repair and replace what's consumed in the month of active life. Ships travel with passengers in suspended animation, allowing the vast distances between these cold worlds to be reached during the sleep period. And, since all members of the lockstep sleep and wake on the same schedule, there is no wrenching desynchronization with the surrounding society during travel. One may spend thirty years in transit, but no time passes for anyone else in the lockstep while you're traveling either. A world can effectively trade with all other worlds within a thirty year travel radius without noticing the elapsed time. This is a technological system that at first doesn't sound like it would work, but Schroeder does a great job defending it and chasing down implications. The lockstep civilization serves as a sort of anchor and time capsule separate from the more frantic pace of the so-called fast worlds. The trans-Neptunian lockstep worlds are remote enough and poor enough to not attract too much unwanted attention, and are thus tolerated by fast societies that may have more available resources. (Although, to be sure, automated robot defenses have to carry a lot of weight here given how helpless the locksteps are during a sleep cycle.) I'm not sure I completely bought the sociology, but it works well enough to carry the weight of the story, and I've never seen a science fiction construction that uses the common technology of suspended animation in quite this way before. Schroeder sets up some nice stabilizing tensions between resources and time, adds some believable political reasons for this fragile society to survive, and uses some of its obvious political vulnerabilities as story drivers. The plot, unfortunately, isn't as good as the big idea, and particularly suffers at the start of the book. Toby is a teenager without much experience (Lockstep is marketed as young adult), and is immediately thrown into a strange and quickly hostile environment. This means that he spends the first half of the book reacting to a blizzard of new information, and the plot tends to revert to a tour of Schroeder's constructed world. Toby is also a bit of a cipher and a bit of an everyman, without many feelings or opinions beyond the obvious feelings that any teenage boy would go through in this situation. That makes the sense of a world tour even stronger. This flaw does not persist through the whole book. Toby does eventually start making decisions and doing things, some of the supporting characters add additional depth, and I found the unwinding of the plot surprisingly satisfying. The nature of time in this world lets Schroeder have both epic sweep and personal connection at the same time, which lets him pull off some neat contrasts between the personal and the political. I also liked Toby's gradual piecing together of what actually happened while he was asleep, both in the construction of society as a whole and in the personal conflicts between people he knew well. Some of the ease of grand manipulation seemed dubious to me, but, in Schroeder's defense, people do develop a reverence for and stories about things that have lasted a very long time, and Schroeder's setup gives him quite a lot to work with in that department. So, a mixed bag. The core concept is thought-provoking and up to Schroeder's usual standards. The surrounding world-building isn't as much, and I think Schroeder reaches for some too-easy explanations and still underplays just how many disruptive things can happen over the span of time that this book covers. The characterization I found weak and unsatisfying for much of the book, but it gets moderately better in the end. There's a bit too much tour, and a bit too much world exploration instead of plot, but it's a fascinating world and I still enjoyed the tour. With stronger characters and a few fewer dubious supporting pillars in the world background, I think this would have been an excellent book. As is, it's an enjoyable novel set firmly in the big idea and deep future end of science fiction, and I'm always happy to see more stories like that. It's not the best that Schroeder has done, but I still recommend it. Rating: 7 out of 10

We currently use completely different methods and tools of building test beds and running tests for Debian vs. Click packages, for normal uploads vs. CI airline landings vs. upstream project merge proposal testing, and keep lots of knowledge about Click package test metadata external and not easily accessible/discoverable. Today I released autopkgtest 3.0 (and 3.0.1 with a few minor updates) which is a major milestone in unifying how we run package tests both locally and in production CI. The goals of this are:

• Keep all test metadata, such as test dependencies, commands to run the test etc., in the project/package source itself instead of external. We have had that for a long time for Debian packages with DEP-8 and debian/tests/control, but not yet for Ubuntu s Click packages.
• Use the same tools for Debian and Click packages to simplify what developers have to know about and to reduce the amount of test infrastructure code to maintain.
• Use the exact same testbeds and test runners in production CI than what developers use locally, so that you can reproduce and investigate failures.
• Re-use the existing autopkgtest capabilities for using various kinds of testbeds, and conversely, making all new testbed types immediately available to all package formats.
• Stop putting tests into the Ubuntu archive as packages (such as mediaplayer-app-autopilot). This just adds packaging and archive space overhead and also makes updating tests a lot harder and taking longer than it should.

So, let s dive into the new features! New runner: adt-virt-ssh We want to run tests on real hardware such as a laptop of a particular brand with a particular graphics card, or an Ubuntu phone. We also want to restructure our current CI machinery to run tests on a real OpenStack cloud and gradually get rid of our hand-maintained QA lab with its test machines. While these use cases seem rather different, they both have in common that there is an already existing machine which is pretty much only accessible with ssh. Once you have an ssh connection, they look pretty much the same, you just need different initial setup (like fiddling with adb, calling nova boot, etc.) to prepare them. So the new adt-virt-ssh runner factorizes all the common bits such as communicating with adt-run, auto-detecting sudo availability, doing SSH connection sharing etc., and delegates the target specific bits to a setup script . E. g. we could specify --setup-script ssh-setup-nova or --setup-script ssh-setup-adb which would then get called with open at the appropriate time by adt-run; it calls the nova commands to create a VM, or run a few adb commands to install/start ssh and install the public key. Then autopkgtest does its thing, and eventually calls the script with cleanup again. The actual protocol is a bit more involved (see manpage), but that s the general idea. autopkgtest now ships readymade scripts for these two use cases. So you could e. g. run the libpng tests in a temporary cloud VM:

# if you don't have one, create it with "nova keypair-create"
$ nova keypair-list
[...]
  pitti   9f:31:cf:78:50:4f:42:04:7a:87:d7:2a:75:5e:46:56  
# find a suitable image
$ nova image-list 
[...]
  ca2e362c-62c9-4c0d-82a6-5d6a37fcb251   Ubuntu Server 14.04 LTS (amd64 20140607.1) - Partner Image                           ACTIVE    
$ nova flavor-list 
[...]
  100   standard.xsmall    1024        10     10                 1       1.0           N/A        
# now run the tests: please be patient, this takes a few mins!
$ adt-run libpng --setup-commands="apt-get update" --- ssh -s /usr/share/autopkgtest/ssh-setup/nova -- \
   -f standard.xsmall -i ca2e362c-62c9-4c0d-82a6-5d6a37fcb251 -k pitti
[...]
adt-run [16:23:16]: test build:  - - - - - - - - - - results - - - - - - - - - -
build                PASS
adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.

Please see man adt-virt-ssh for details how to use it and how to write setup scripts. There is also a commented /usr/share/autopkgtest/ssh-setup/SKELETON template for writing your own for your use cases. You can also not use any setup script and just specify user and host name as options, but please remember that the ssh runner cannot clean up after itself, so never use this on important machines which you can t reset/reinstall! Test dependency installation without apt/root Ubuntu phones with system images have a read-only file system where you can t install test dependencies with apt. A similar case is using the null runner without root. When apt-get install is not available, autopkgtest now has a reduced fallback mode: it downloads the required test dependencies, unpacks them into a temporary directory, and runs the tests with $PATH, $PYTHONPATH, $GI_TYPELIB_PATH, etc. pointing to the unpacked temp dir. Of course this only works for packages which are relocatable in that way, i. e. libraries, Python modules, or command line tools; it will totally fail for things which look for config files, plugins etc. in hardcoded directory paths. But it s good enough for the purposes of Click package testing such as installing autopilot, libautopilot-qt etc. Click package support autopkgtest now recognizes click source directories and *.click package arguments, and introduces a new test metadata specification syntax in a click package manifest. This is similar in spirit and capabilities to DEP-8 debian/tests/control, except that it s using JSON:

    "x-test":  
        "unit": "tests/unittests",
        "smoke":  
            "path": "tests/smoketest",
            "depends": ["shunit2", "moreutils"],
            "restrictions": ["allow-stderr"]
         ,
        "another":  
            "command": "echo hello > /tmp/world.txt"
         
     

For convenience, there is also some magic to make running autopilot tests particularly simple. E. g. our existing click packages usually specify something like

    "x-test":  
        "autopilot": "ubuntu_calculator_app"
     

which is enough to do what I mean , i. e. implicitly add the autopilot test depends and run autopilot with the specified test module name. You can specify your own dependencies and/or commands, and restrictions etc., of course. So with this, and the previous support for non-apt test dependencies and the ssh runner, we can put all this together to run the tests for e. g. the Ubuntu calculator app on the phone:

$ bzr branch lp:ubuntu-calculator-app
# built straight from that branch; TODO: where is the official" download URL?
$ wget http://people.canonical.com/~pitti/tmp/com.ubuntu.calculator_1.3.283_all.click
$ adt-run ubuntu-calculator-app/ com.ubuntu.calculator_1.3.283_all.click --- \
      ssh -s /usr/share/autopkgtest/ssh-setup/adb
[..]
Traceback (most recent call last):
  File "/tmp/adt-run.KfY5bG/tree/tests/autopilot/ubuntu_calculator_app/tests/test_simple_page.py", line 93, in test_divide_with_infinity_length_result_number
    self._assert_result("0.33333333")
  File "/tmp/adt-run.KfY5bG/tree/tests/autopilot/ubuntu_calculator_app/tests/test_simple_page.py", line 63, in _assert_result
    self.main_view.get_result, Eventually(Equals(expected_result)))
  File "/usr/lib/python3/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: After 10.0 seconds test failed: '0.33333333' != '0.3'
Ran 33 tests in 295.586s
FAILED (failures=1)

Note that the current adb ssh setup script deals with some things like applying the autopilot click AppArmor hooks and disabling screen dimming, but it does not do the first-time setup (connecting to network, doing the gesture intro) and unlocking the screen. These are still on the TODO list, but I need to find out how to do these properly. Help appreciated! Click app tests in schroot/containers But, that s not the only thing you can do! autopkgtest has all these other runners, so why not try and run them in a schroot or container? To emulate the environment of an Ubuntu Touch session I wrote a --setup-commands script:

adt-run --setup-commands /usr/share/autopkgtest/setup-commands/ubuntu-touch-session \
    ubuntu-calculator-app/ com.ubuntu.calculator_1.3.283_all.click --- schroot utopic

This will actually work in the sense of running (and succeeding) the autopilot tests, but it will fail due to a lot of libust[11345/11358]: Error: Error opening shm /lttng-ust-wait... warnings on stderr. I don t know what these mean, just that I also see them on the phone itself occasionally. I also wrote another setup-commands script which emulates read-only apt , so that you can test the unpack only fallback. So you could prepare a container with click and the App framework preinstalled (so that it doesn t always take ages to install them), starting from a standard adt-build-lxc container:

$ sudo lxc-clone -o adt-utopic -n click
$ sudo lxc-start -n click
  # run "sudo apt-get install click ubuntu-sdk-libs ubuntu-app-launch-tools" there
  # then "sudo powerdown"
# current apparmor profile doesn't allow remounting something read-only
$ echo "lxc.aa_profile = unconfined"   sudo tee -a /var/lib/lxc/click/config

Now that container has enough stuff preinstalled to be reasonably fast to set up, and the remaining test dependencies (mostly autopilot) work fine with the unpack/$*_PATH fallback:

$ adt-run --setup-commands /usr/share/autopkgtest/setup-commands/ubuntu-touch-session \
          --setup-commands /usr/share/autopkgtest/setup-commands/ro-apt \
          ubuntu-calculator-app/ com.ubuntu.calculator_1.3.283_all.click \
          --- lxc -es click

This will successfully run all the tests, and provided you have apt-cacher-ng installed, it only takes a few seconds to set up. This might be a nice thing to do on merge proposals, if you don t have an actual phone at hand, or don t want to clutter it up. autopkgtest 3.0.1 will be available in Utopic tomorrow (through autosyncs). If you can t wait to try it out, download it from my people.c.c page . Feedback appreciated!

Suppose you had spawned a process in TCL and knew its PID and wanted to kill it? Sounds simple enough thing to do, right? This problem has plagued me for many months because some things you can assume on a normal system do not hold true on strange environments, such as build deaemons. Seems simple enough, I started off with: Except.. not every environment has the kill binary, and with that piece of code exec has to be a binary and not a shell builtin. The funny thing is that /bin/kill is in the procps package, which is the package having the buildd problems. So next idea was to use command -v to check for the existence of kill and skip those tests that needed kill if not found. Good idea except, so I found out later, it also finds built-ins. That means we are back to problem #1. There is a kill command in tcl, but it requires tclx. That seems excessive for just one little simple command. How can I run a shell out of TCL that runs the kill builtin? On the command line, something like below would do it. I was closer, but then hit TCL quoting hell. No matter what I (initially) did I d either get the shell to complain or my variable to not be evaluated. In the end, I had to write it to a separate variable for the command line then apply that to the exec. Not perfect but at least it works now. The resulting code (found in testsuite/config/unix.exp) looks like: Perhaps there is a more elegant way, I m certainly no star TCL programmer, but of all the combinations I saw this was the only that worked.

LAVA has arrived in Debian unstable. No need for third party repositories (unless you want to), a simple apt install lava-server. What happens from that point isn t so simple. I ve made the single instance installation as straightforward as I could but there is a lot more to do to get LAVA working in a useful manner. First, a little history to explain some of the hysterical raisins which may appear later. Validation So, you ve made a change to the code, good. It compiles, yay! Before you ship it, does it pass the unit tests? Right, now you have an improved patch which does what you want and keeps the unit tests running. Validation is all about asking that awkward question: does your change work? Does it introduce side effects on other systems / services in the same code? Does it break programs which use services exported by the code? LAVA is one step along the road to system testing, starting at the bottom. Automation Well you could do all that yourself. Write the image to the device yourself, apply power yourself, connect to serial, copy test scripts to the booted image and pull the results off, somehow. Much better if this is automated maybe every commit or every push or as often as the tests can be run on the number of devices available. LAVA Linaro Automated Validation Architecture. Linaro is a not-for-profit building the future of the Linux kernel on ARM devices. Lava was built for and by Linaro, so the initial focus is clearly on validating the Linux kernel on ARM devices. There are likely to be gotchas in the code for those wanting to use LAVA for other kernels Linaro can t support lots of different kernels, but if there are changes which make it easier to use other kernels without impacting on validation of Linux, those would likely be accepted. The experience with using LAVA is all with ARM but if there is interest in extending LAVA to work with devices of other architectures, again, patches would be welcome. The development of the packaging to make LAVA suitable for Debian has also meant that LAVA will run on hardware other than x86. e.g. armv7.com. I m running a mini lab at home based around an arndale board, I ve also got a mini lab at work based on a cubie2. Requirements for such setups are described in the documentation and in previous entries on this blog (principally you will need SATA, lots of RAM and as many CPU cores as you can find. If you want to run LAVA on other architectures, go ahead and let us know if there are issues. Available versions The versions uploaded to Debian will (from now on) be production releases. The same code as is running on http://validation.linaro.org/ development builds and test builds are supported using helpers in the lava-dev package. Releases to unstable will automatically migrate into Ubuntu Unicorn. I ll continue building interim versions at the former locations on http://people.linaro.org/~neil.williams/, including builds for Ubuntu Trusty 14.04LTS as well as providing packages for Jessie until the uwsgi package can migrate. LAVA is looking to work with anyone who can prepare and maintain packages for other distributions like Fedora. Bugs LAVA is migrating from Launchpad bugs to http://bugs.linaro.org which is a bugzilla interface. Now that LAVA is also in Debian, anyone is welcome to use the Debian BTS which does not require any login or account setup. The maintainers (including me) will then forward those bugs as appropriate. Documentation The immediate task is to update the documentation in the lava-server-doc package (and the Debian wiki) to reflect the availability of LAVA from Debian unstable and how to choose which release of LAVA to use in your systems. However, there is a large amount of documentation already available click the Help link in the menu bar of any current LAVA instance. As with many projects, the docs have been written by the development team. If there are things which are unclear or if sections need to be moved around to make it easier for people new to LAVA to pick it up, please file bugs. Hardware It isn t easy to run a LAVA lab, there is a lot more to it than simply installing lava-server. LAVA would not be possible without the lab team and any other LAVA instance is going to need the same level of excellence in system administration, device support and cooperation. I ve covered a little bit of that in previous entries on this blog about my home lab but that is nothing compared to the work required to provide a working lab like the one in Cambridge. Access to that lab is restricted to people within Linaro but LAVA is also reaching out to the community and if there are tests or hardware you want to see within a LAVA instance, not necessarily in the main lab, then talk to us. Contact Bug reports are preferable but you can also find LAVA people on #linaro-lava on OFTC or contact us on the linaro-validation mailing list. Future There is a lot more work to do on LAVA yet. There are assumptions about partition layout within images (hysterical raisins) and issues with unused software being required for remote worker installations. Both are now part of the next major development work within LAVA. DebConf14 There is lot more to talk about with LAVA if you are attending DebConf14 then there will be talks on LAVA and plenty of time to answer questions.

Review: Ancillary Justice, by Ann Leckie

Series:	Imperial Radch #1
Publisher:	Orbit
Copyright:	2013
ISBN:	0-316-24662-X
Format:	Trade paperback
Pages:	416

As Ancillary Justice opens, Breq is on an icy planet trying to track down a person who has gone rather thoroughly to ground. As her search continues, the reader slowly learns the context of that search, which is only the latest step of a long and quixotic exercise in private determination. She was set on her path by events twenty years previous, told in interleaved flashbacks, that have left her greatly diminished and thrust outside of any of the context of her over thousand years of life. Breq was Justice of Toren, a starship and AI that possessed thousands of ancillaries, human bodies slaved to her mind. Now only this one remains, and nothing exists of her former life other than this goal. It's always hard to write a review of a book that I loved this much. I want to find some way to grab the reader and shake them and say "you have to read this!" without giving away any of the delicious details. That's particularly difficult with Ancillary Justice, since one of the delights of this book is the slow unfolding of not only the plot but the background and motivations behind the plot. There is so much beneath the surface of Breq's methodical intent, and discovering all the nuances is utterly delightful. This is also a book that is very deeply concerned with identity, and which does one of the best jobs I've seen in science fiction of showing a non-human first-person protagonist: close enough to human to permit identification and comprehension, but far enough away for delightful sparks of unexpected insight or thought-provoking difference. Breq is a ship, and AIs are central to this story, so the comparison that comes immediately to mind is with Iain M. Banks. Leckie's role for ship AIs is much different than the structure of Banks's Culture, but I think this is still an apt comparison. Like Banks, Leckie is writing large-scale space opera dominated by a meddling empire that follows some familiar human patterns but not others. Banks's Culture takes a less direct approach with its meddling; the Radch has a more Roman attitude towards preventative conquest and citizenship. But both deal with large-scale issues of politics, culture, and conquest. Both also write excellent AIs, but I think Leckie is more successful than Banks at giving her AIs inhuman properties and creating a sense of eerie alienness that sometimes fades almost entirely and sometimes comes sharply to the surface. The first-person perspective helps considerably there. But where Ancillary Justice truly shined for me is in the interpersonal relationships, and in the insight they provided into character and motive. At the start of the book, Breq finds a drug addict dying in the cold and rescues her, bringing her along on her search for lack of a better option. Breq's relationship with Seivarden is complex, difficult for both of them, badly asymmetric at the start, and develops into something brilliant. At the start of the book, Seivarden is easy to dislike, and Breq's tone is refreshingly bracing and forthright. But through the course of the book Seivarden grows into something much more, in a way that I found both believable and incredibly compelling. And Leckie does this without falling into any of the typical relationship patterns, without introducing artificial romance (or, indeed, any romance at all, which is an excellent choice here), and without compromising the personalities of either character. It's masterfully done. One of the most amazing things about this book to me is that it's a first novel. I never would have guessed that from reading it. It's beautifully paced, the characterization is deep and compelling, and Leckie avoids any sign of the typical first-novel problem of stuffing the book with too much Stuff. Ancillary Justice is a capable and confident novel that builds a compelling world and even more compelling characters. I liked it more than I like most of the Culture novels, which is saying quite a lot, but Leckie offers much of the same scope with deeper and more personal characterization and a tighter plot. I haven't yet remarked on one aspect of this book that every other review seems to remark on: its treatment of gender. The Radch do not recognize or care about gender distinctions, and therefore Breq struggles throughout the book with proper gender labeling in much the same way that a native English speaker tends to struggle with grammatical gender when learning a Romance language, except with more social consequences. Leckie has chosen to represent this in the novel by having Breq refer to everyone uniformly as "she." This has its pluses and minuses: it still supports a binary gender concept where a gender-neutral pronoun might not, but given all the negative reaction this book got just for using "she," a gender-neutral pronoun might be a bridge too far for a lot of readers. I thought it created a nice bit of alienation, a way of forcing the reader to look at gender markers from the outside and a way to point out how arbitrary many of them are. It was also interesting to see how surprised I was at various points in the book when it became obvious that some person Breq had been calling "she" in the first-person narration turned out, from story context, to probably be male. That said, I think this part of the book is overblown in reviews. It's often the first thing people mention, but while it was a nice side bit of world-building, I don't think it's that central to the story. I'm particularly baffled by the handful of people who complained about it, since it's not intrusive and it quickly fades into the background apart from occasional necessary shifts of mental image. (It does create the impression of a world containing only women, but I found that a nice change from the more common impression in space opera of a world containing only men.) Ancillary Justice has already won the Nebula and Arthur C. Clarke awards and tied for the BSFA award for best novel, and I'm happy to report that it deserves all of those. I haven't yet read all of the other Hugo nominees, but it's hard to imagine a world in which it won't top my ballot. This is a fantastic novel, by far the best thing I've read so far this year. I'm delighted that it's the first book of a trilogy, since I'm not done with either the world or the characters yet, but it stands well on its own and reaches a satisfying conclusion. I recommend it to everyone, but particularly to anyone who likes Banks, intelligent ships, or who is looking for thoughtful and complex space opera. Followed by Ancillary Sword. Rating: 10 out of 10